Re: [aor-rtma] Daily QC of RTMA/URMA All - Home - RTMA/URMA Discussion Group

First, "yay" for the good news Jeff about the 15 min RTMA coming soon! Any rough guess timeline or is that a dangerous thing to predict?

Second, I also believe that we need to move away from ObsGrids for initiating the ESTF ( we do not using it any more for BOIverify bias correction), and the improved timeliness of RTMA I hope will make that possible, but to tag onto comments from Joe and Andy...we REALLY will need an easy way to QC blacklist bad obs we know about, AND what about forcing ones back in that RTMA QC is dropping but we KNOW to be valid (say behind a thunderstorm or near some sharp thermal boundary)?  I don't suppose that any local blacklisting of bad obs, or forcing back in of good ones QC'd out of RTMA for whatever reason, could be applied to some locally-clipped version of the RTMA re-created locally, but instead would only apply to future RTMA grids generated at NCEP.

Steve K @RNK

Another item of interest is the upcoming RU RTMA capability.  Rapid Updates every 15 minutes will begin in the near future.  I don't have quick access to which elements are included, but this is moving in the right direction of having more timely RTMA available to serve the WFO needs in Enhanced Short Term Forecasting.

JPC

Jeff Craven

Chief, Statistical Modeling Branch

National Weather Service, W/STI-12

Meteorological Development Laboratory (MDL)

Room 10410, SSMC2

Silver Spring, MD 20910

(301) 427-9475 office

(816) 506-9783 cell/text

@jpcstorm

https://www.weather.gov/mdl/nbm_home

On Thu, Aug 17, 2017 at 9:36 AM, Andy Just - NOAA Federal <andy.just@noaa.gov> wrote:

All,

  Just want to say loving this thread - which likely came from me when I brought up the concept to really start looking at the URMA on the ForecastBuilder webinar yesterday morning.   URMA has definitely become much more important:

• Bias correction for NationalBlend and SuperBlend
• Verification for the BOIVerify related pages/programs across CR - and any office that uses them from the CrGfeTools package on the SCP. 

  Currently ForecastBuilder uses the Obs database for the Enhanced Short Term Forecast process because of:

• Timeliness vs the RTMA.  
• Offices can "blacklist" portions or all of of observations - something that cannot be done with the RTMA at this time (though is expected in the near future)
  

  However, given the importance to verification and bias correction - a push should begin to discontinue use of Obs in favor of RTMA/URMA.  Having the "blacklist" feature for RTMA will really help in that regard.

Andrew Just

ForecastBuilder and HazSimp WSW Developer

Information Technology Officer (ITO)

National Weather Service La Crosse WI (ARX)

608-784-8275 ext 486

On Thu, Aug 17, 2017 at 8:04 AM, Joseph Moore - NOAA Federal <joseph.moore@noaa.gov> wrote:

I honestly couldn't tell you what the automated BOIVerify stats are using, though I'd guess that Tech Order was completed so we must be using it. (The automated emails and web page don't indicate what's being used for verification.) Apologies for any confusion!

However, I can say with certainty that we're still using the Obs grid for ESTF purposes since that the tool default. We're also constantly adjusting the Obs grid using the ObsGridQC tool, and I can say with certainty we're not the only office dedicating time to QC'ing the Obs grid as a regular shift duty.

-Joe

On Thu, Aug 17, 2017 at 7:49 AM, Jeffrey Craven - NOAA Federal <jeffrey.craven@noaa.gov> wrote:

Interesting.  In CR the BOIVerify ground truth was changed via a Tech Order from "obs" to URMA.  This occurred back in Spring 2016.  

Are you saying that your office is still using "obs" as the ground truth?  Or are you using "obs" for short term Forecast purposes owing to it being available at HH + 10 versus HH + 40 (RTMA).

JPC

On Thu, Aug 17, 2017 at 1:30 AM Joseph Moore - NOAA Federal <joseph.moore@noaa.gov> wrote:

Hi James,

I'll just add my thoughts and experience as a forecaster with a strong interest in this topic: Most offices (at least in CR) are focusing a lot of energy on the MatchObsAll "Obs" grid quality control, doing quality control multiple times a day. The primary reason for this is that within the ESTF paradigm the Obs grid generated by MatchObsAll is used for the current hour then interpolated to the forecast for numerous weather elements, so having this grid be representative is important. We have forecasters at every office combing over the obs, usually noticing bad obs getting through within a few hours. In my personal experience, knowledge of the RTMA/URMA is low, and the quality control/feedback is usually limited to communicating with the SOO. I don't think many doing this QC know about the RTMA/URMA AOR list and if/how passing along QC notes would be helpful.

Another issue, which I experience, is seeing so many conversations on this listserv seem to end in questioning whether a ground-truth observation is really correct and/or representative for the 2.5x2.5km grid box. I've seen a number of back-and-forth conversations ending with discussion about how the ob was thrown out because it was too far off the HRRR... which should not happen if the observation is from a quality source! From reading discussions I think this might get better eventually, but it's still a deterrent for me to report QC issues.

Finally, we gotta eat our own dog food. If we're serious about making the RTMA/URMA the Analysis of Record, as this email list is named, then we need to be using it in operations to replace the Obs grid. There's simply to excuse - this is a mature production model under active development, and at the WFO level we should be supporting this to replace the Obs grid. I think if that's going to happen and we want buy-in at the ground level there needs to be clear instructions on how forecaster can share QC feedback. With the upcoming dynamic blacklist (where the developers (and maybe SDM?) can adjust the blacklist more responsively) this feedback will be able to make meaningful adjustments. There's still going to be issues of latency (Obs grid comes in at :10 to :15, RTMA around :44 last I used it), but with plans for a rapid-update RTMA in the pipeline that challenge should be temporary.

-Joe

On Wed, Aug 16, 2017 at 11:35 AM, James V Clark <james.v.clark@noaa.gov> wrote:

Is anyone/are any WFOs doing daily QC of the RTMA/URMA and how you are going about it?  I am running a "daily/hourly" QC, but I typically only look at it seasonally due to time constraints.  Basically I pull MaxT/MinT, T and Td out of our BV archive and then run comparisons with ASOS/AWOS.  I eventually need to expand to mesonets as well, however due to various issues associated with our myriad of mesonet sites, I haven't gone there yet.  We do perform a QC of our Obs grids multiple times a day.  It was mentioned on a call today and I think it's a great idea as it would help the development of the RTMA/URMA and the NBM, it's just a question of implementation.

Thanks,

JV

--

Joseph J. Moore

Meteorologist | Social Media Program Leader | Open Source GIS Evangelist

NOAA/National Weather Service Duluth, MN

--

Jeff Craven

Chief, Statistical Modeling Branch

National Weather Service, W/STI-12

Meteorological Development Laboratory (MDL)

Room 10410, SSMC2

Silver Spring, MD 20910

(301) 427-9475 office

(816) 506-9783 cell/text

@jpcstorm

https://www.weather.gov/mdl/nbm_home

 

--

Joseph J. Moore

Meteorologist | Social Media Program Leader | Open Source GIS Evangelist

NOAA/National Weather Service Duluth, MN

 

 

 

--