Forums (Forecast Builder)

Back

Add configuration for the observed database?

RSS (Opens New Window)(Opens New Window)
DE
David Eversole, modified 7 Years ago.

Add configuration for the observed database?

Youngling Posts: 6 Join Date: 6/12/17 Recent Posts

Hi,

I'm working on getting ForecastBuilder set up for our office here at WFO Mobile and was wondering if a future release for the ForecastBuilder package could include a configuration for what the observed database is?  Currently, there are a bunch of places in ForecastBuilderUtility that are hard coded to use the "Obs" database.  For example, in the quickObsUpdate method starting at line 604-ish.  Our SOO (now an MIC at Binghamton) recently assigned a COMET course called "Verification Methods in the NWS National Blend of Global Models" which was really enlightening about how URMA/RTMA compares with the old MOA software.  Here is what the course has to say:

MatchObsAll (MOA) is a locally-configured and generated gridded analysis. As a result, the analysis creates discontinuities at CWA boundaries when the local grids are joined together. Analysis systems like MatchObsAll force the analysis to exactly match every observation, but those observations are subject to errors. While “blacklists” of stations exclude consistently bad data to reduce observation error, other aspects of MOA are still problematic:
It is not a flow-following analysis, which makes for poor wind analyses.
It is based on an older objective analysis scheme.
It has no procedure to minimize analysis error like NWP data assimilation systems.
It simplifies terrain effects.
RTMA and URMA have characteristics that address some of the limitations of MOA. The NWS selected RTMA/URMA as the analysis of record for the Blend because:

It covers all domains in the NDFD continuously.
Bad observations are removed using an automated quality control method.
It runs centrally on the Weather and Climate Operational Supercomputing System.
It is a flow-following analysis, which improves the wind analysis.
It better accounts for terrain effects.
Though MOA has a long history of use in the field, RTMA/URMA has been identified as the best candidate for verification in the Blend. 

 So, it seems like the Obs database has been succeeded by the RTMA/URMA.  I've modified our ForecastBuilder package to use RTMA, but a configuration for this would be easier?

Have a good one!

- Dave

DB
David Barjenbruch, modified 7 Years ago.

RE: Add configuration for the observed database?

Youngling Posts: 16 Join Date: 9/2/14 Recent Posts

 

Hi David,

Thank you for your feedback.  There are clearly advantages and disadvantages of each scheme and what one considers ground truth as a starting point for observations.  We are looking for a consistent and sound starting point, and the CRGMAT team will be addressing this very thing in the coming months as we move toward ESTF enhancements.  

On a personal note, URMA indeed has a more realistic flow following analysis for wind.  That said, there are inherent biases in sharp terrain where I've seen and documented URMA analysis with 10-15 degree errors and 20-25% errors on RH which has severe detrimental impacts to not just p-type forecasts, but fire weather observations/forecasts as well.  We, as a team, believe that more feedback like this can and will improve URMA/RTMA analysis schemes, and be able to create a sound and consistent observation database across the nation in the future.

In the interim, the team may be able to discuss the potential for making adjustments to the configurations.

Thanks again,

Dave

JM
Joseph Moore, modified 7 Years ago.

RE: Add configuration for the observed database?

Youngling Posts: 72 Join Date: 11/1/13 Recent Posts

Hey David,

There was a lively conversation on the RTMA-URMA AOR email list in mid/late August about this subject. I feel passionately about this topic, and in the field the two main reasons we might be against switching to RTMA at this time is (1) latency and (2) extremes. In the case of latency, because RTMA doesn't arrive until :47ish while Obs (MatchObsAll) runs around :10 or so. This isn't critical, but important when conditions are fast-changing and especially in winter weather when surface temps will be part of what drives the precip type. For extremes, I think it partly is a culture/mindset issue, but Obs can capture extremes at sites better than RTMA. This has improved greatly from the first versions of RTMA in which an actual observation gets thrown out because it's not close enough to the HRRR, but it does still happen in extreme circumstances. This is well known and like I said, improving with each new version.

One last reason: Local and responsive QC. If we see a bad ob we can fix it in the Obs grid in minutes. On the RTMA/URMA scale they'll soon have the ability to update the blacklist dynamically (right now it is fixed with each new version), but even then it's unclear to me how fast we'll be able to clear a bad ob or force a good ob to be integrated into the analysis.

In summary, RTMA isn't perfect but I sure hope we can all adopt it to replace Obs operationally within the ESTF framework soon - if only to get more feedback to the RTMA folks so we can truly make that our agency-wide "analysis of record" that works well across all of the United States and her territories.

 

AJ
Andy Just, modified 7 Years ago.

RE: Add configuration for the observed database?

Youngling Posts: 89 Join Date: 6/2/15 Recent Posts

Dave,

  Following up on what Joe said, yeah we definitely have a goal of switching over to the RTMA, so that all forecaster feedback and technological improvements can go in improving that data source.  Same holds true with NBM.  This is nicely stated in our latest training video on an Introduction to ForecastBuilder.

  Your post, however, got me thinking we need to get that ability in place for switchover. So with the next ForecastBuilder update (in the CrGfeTools/trunk directory), I have added a configuration option in ForecastBuilderConfig titled ObservationalDatabase. You can specify either "Obs" or "RTMA" and it will copy that dataset in.

DE
David Eversole, modified 7 Years ago.

RE: Add configuration for the observed database?

Youngling Posts: 6 Join Date: 6/12/17 Recent Posts

Thanks Andy!

That will help a lot!  I'm starting to work on going through the training.  Right now, ForecastBuilder is up and running and seems to be working great.

 

- Dave