Just a quick update. NWS is now experimenting with additional XML feeds:

http://weather.gov/xml/

This is seven day digital forecast data for 5x5km grids.

http://weather.gov/data/current_obs/

This are current observations in xml and rss formats.

and some more.

http://rssweather.com/

Well, they're getting data from somewhere, and then someone built a Drupal (http://www.drupal.org) module for it:
 http://drupal.org/project/weather

Here's a demo site that uses it:
 http://www.badgergoose.com/drupal/weather

Just more sources to get to know about.


--
Boris Mann
http://www.bmannconsulting.com

Tony:

On Feb 12, 2004, at 2:08 PM, QT - Tony Collen wrote:

> Not sure if there's still interest, but I've picked up
> openWeather a little bit more lately. I've got a program
> parsing the data at
> http://weather.noaa.gov/pub/data/forecasts/city/ into XML, and I
> will have forecasts put up on the openweather site within the
> next couple days I would imagine.

Nice work.

> Here's the sample "raw" xml format I am using. As you can see,
> it's pretty well marked up. Any input on the intermediate format
> (which will also be provided) is welcome. I apologize if this
> does not come through correctly:

I actually think it should be MORE marked up. Also, the DTD would be more useful than sample output...

Comments on potential additional markup in-line:

> <?xml version="1.0" ?>
> <fc>
> <header>
> <city>Anchorage</city>
> <state>AK</state>
> <issued>
> <time>Wednesday morning</time>
> <date>Feb 4</date>
> <year>2004</year>

Is there an existing date format that can be used? Like, dc:date ? Transformations should be done earlier rather than later, IMHO.

Don't know why it has to be wrapped in "header"...why not:

<location>
U
</location>
<issued>
U
</issued>

> <forecast>Cloudy, high 22, 0% chance of
> precipitation.</forecast>

<forecast>
   <description>cloudy</description> <-- normalize case
   <high>22</high> <-- what about units? e.g. <high units="celsius"> <low>18</low>
   <precipitation>0</precipitation>
</forecast>STRIPMIME_JOINLINES

Hey guys,

Not sure if there's still interest, but I've picked up openWeather a little bit more lately. I've got a program parsing the data at http://weather.noaa.gov/pub/data/forecasts/city/ into XML, and I will have forecasts put up on the openweather site within the next couple days I would imagine.

Here's the sample "raw" xml format I am using. As you can see, it's pretty well marked up. Any input on the intermediate format (which will also be provided) is welcome. I apologize if this does not come through correctly:

<?xml version="1.0" ?>
<fc>
  <header>
    <city>Anchorage</city>
    <state>AK</state>
    <issued>
      <time>Wednesday morning</time>
      <date>Feb 4</date>
      <year>2004</year>
    </issued>
  </header>
  <outlooks>
    <outlook>
        <period>Wednesday</period>
        <forecast>Cloudy, high 22, 0% chance of precipitation.</forecast>
    </outlook>
    <outlook>
        <period>Wednesday night</period>
        <forecast>Low 29, 0% chance of precipitation.</forecast>
    </outlook>
    <outlook>
        <period>Thursday</period>
        <forecast>Cloudy, high 23, 0% chance of precipitation.</forecast>
    </outlook>
    <outlook>
       <period>Thursday night</period>
       <forecast>Low 32.</forecast>
    </outlook>
    <outlook>
      <period>Friday</period>
      <forecast>High 18.</forecast></outlook>
  </outlooks>
</fc>

>I see this actually has the full information, which may be >the easiest way to work with it if people want the more >detailed info.

Yes... this allows some additional parsing for some fields as well as additional information (like the geocode tag - the county FIPS code in our case - usefull if you are doing GIS stuff).

>Are you aware of an XML format for describing standard, >i.e. >non-alert, weather forecasts?

Not really... just some very informal ones here and there... some discussed in this thread. Was surfing today looking (again) when I stumbled across this thread.

The Weather community has had some very weather specific formats in place for many years (like METAR) that work great if you are an expert in those formats. Several attempts within the community to jump start some xml standards have apparently not seen much progress. That's one reason why we focused on these already existing formats - it made this a low hanging fruit project to help show how xml and weather data can work together.

However, for me, the key is to keep it simple. An all encompassing xml schema for "weather" will be so complex (IMHO) as to be unusable. That's why, in my own mind, I tend to break it up into several areas; forecasts, alerts, current obs, hydro/water, climate, historical, etc.

Also, keep your eye on this: http://weather.gov/ndfd/

NWS forecasts are now being produced for 5x5km grids (not large "zones" that are somewhat based on counties) every three hours instead of every 12 hours. This digital data is mostly raw numbers from which you can build a worded forecast from using a program.

Cheers,

Bob


>Boris Mann
>http://www.bmannconsulting.com

On Nov 12, 2003, at 4:45 PM, QT - Bob Bunge wrote:

> Interesting thread.
>
> Here at NWS we are starting to experiment with RSS feeds:
>
> http://weather.gov/alerts/

Thanks for the information, Bob. This looks like some really useful stuff. It would be more useful to have the entire announcement included in the RSS feed, rather than just the "header" information. I'll note it in the feedback form.

> We are also producing data in another xml format, CAP (Common
> Alerting Protocol), which is more tailored towards the hazard
> messages we are working with.

I see this actually has the full information, which may be the easiest way to work with it if people want the more detailed info.

Are you aware of an XML format for describing standard, i.e. non-alert, weather forecasts?

--
Boris Mann
http://www.bmannconsulting.com

Awesome!

Bob, I must compliment you and the NWS on the work you've done. It's nice to see a data format that is descriptive and useful, finally.

Interesting thread.

Here at NWS we are starting to experiment with RSS feeds:

http://weather.gov/alerts/

We are also producing data in another xml format, CAP (Common Alerting Protocol), which is more tailored towards the hazard messages we are working with.

If you have comments, filling out the survey/feedback form is the best way to ensure your comment is in the official record and will be taken into account when it's time to decide if the rss feeds should become an official product or not.

Bob

Wow. I just had a totally awesome idea, and I think I can pull it off.

I had thought about the problem of decoding raw METAR data. I found WeatherEase Toolkit, which is written in Java, and up until about 15 minutes ago I thought it was the best way to decode METAR.

In one of my classes, we've been talking about grammars, and how they relate to parsing a program. It's all interesting, but we weren't really applying it much in the class.. it would be better off as a project for a class on compilers and interpreters.

Fast forward to tonight. I was randomly poking around the site for Chaperon, a parser which "converts structured text to XML." You define a grammar, and Chaperon will produce XML output based on that. I didn't think about it in the context of how it's used.

Suddenly I put 2 and 2 together, and whammo, I figured it out. I'll write a grammar that describes a METAR report, and let Chaperon parse it. The METAR format is well defined, the problem is the order that report elements come in. Hoefully Chaperon can fix this.

So what do you guys think.. shall I forge ahead into this new territory of grammars, or should I try to shoehorn something like the WeatherEase Toolkit into openWeather?

Tony

I hit 'reply' but not 'reply all', which might have messed it up. Strange. A blog or even a wiki would be good, but I guess I got a handle on the QT so it's all good for now.

On Wednesday, September 3, 2003, at 01:33 PM, QT - Tony Collen wrote:
> Dang, this quicktopic stuff is all messed up for me, so I'll
> resend the message I wrote but the list didn't seem to get:

Should just be able to hit "reply" and it just works.

> openWeather is me!

I almost suspected as much when I hit somewhere by mistake and it gave me a pretty Cocoon error page :p

> I still get tons and tons of weather rss searches, I just wish I
> had all the time to devote to it. I need a week off work and
> school or something.

Put me down in the same request for time off.

> I'm contemplating setting up a weather-rss mailing list in
> mailman, would you guys be interested in a real list by any
> chance?

For now, this Quicktopic seems to meet the needs -- especially with only three people. I sent an email to Chris Heathcote, inviting him to subscribe.

If there is to be a mailing list, there needs to be a common purpose -- like developing WeatherML or something.

And actually, a blog with multiple posting permission might serve just as well...

Dang, this quicktopic stuff is all messed up for me, so I'll resend the message I wrote but the list didn't seem to get:

Boris wrote:

> Since I'm in the process of aggregating, I should enter the
> other service that Chris pointed out OpenWeather.

openWeather is me!

> They serve up RSS as well as "raw" XML, but the XML is pretty
> useless -- for instance, the location (loc) field has the entire
> thing in one big string, instead of semantically tagging with
> city, state, etc. etc.
>
> The units as well are mixed in with data.

I know, I know, it was a super sloppy implementation that I did quickly due to time limitations. Last semester was hell on my body.

I've been busy doing some administrative things on the web server, like switching the main web server to port 80, along with reorganizing the other virtualhosts on this web server. All this is so I can use mod_proxy and point it at Cocoon.

Another thing I need to do is to rewrite my data downloader in Java, since Cocoon recently got a scheduler component, eliminating the need for a crontabbed python script.

Oh, and I might as well integrate that METAR decoder, too :)

I still get tons and tons of weather rss searches, I just wish I had all the time to devote to it. I need a week off work and school or something.

I'm contemplating setting up a weather-rss mailing list in mailman, would you guys be interested in a real list by any chance?

Tony

Since I'm in the process of aggregating, I should enter the other service that Chris pointed out OpenWeather.

They serve up RSS as well as "raw" XML, but the XML is pretty useless -- for instance, the location (loc) field has the entire thing in one big string, instead of semantically tagging with city, state, etc. etc.

The units as well are mixed in with data.

I'm still getting lots of referrer hits from search engines for people looking for RSS weather info.

Chris Heathcote at anti-mega has made a screen scraper (I think?) that uses worldweather.org as its source.

I just wanted to say sorry for being so quiet. I suggested this write before getting on the road for 2+ weeks. Great info so far. Hopefully I'll have some time to respond next week.