All - RE: Re: case assignments - Forecast Performance Blog

Hey Geoff, and team,

I had "hit the ground running" on evaluating one of my cases, and got a start on #6, by grabbing images and making comments in a document.   So far, I'm finding the process to be PRETTY subjective, but it seems to show the v12 being better, by a day or two, I think.

I was pondering making use of a prior public-facing (to explore community engagement) VLab space exploration I was testing to post the same as in the document, as a place that conceivably we could use, not only to review and comment on these case findings, but perhaps as a space that other MEG and V&V like efforts could use to engage broader modeling community folks to join in assessments. 

The test Forums area is here, if you want to join/view/post to see if you think it has any merit.  In fact, I've BCCd the email address one can use to auto-send Forum posts to that VLab space, so this email ought to end up there.  If you were Subscribed to that VLab community, you would then get the standard email notification VLab sends to members who Subscribe:  case-study.vlab@noaa.gov 

Lastly, as I was poking around the GEFS eval site, I was perusing this NHEM AC slide.  Lots of overlap, but I can see in the overall the RETRO AC score is higher, so that's good.  What I wondered about is the choice below says ALL RETROS 6/17-11/19.  I assume, given the large number of dates across the X axis, that EVERY day in that 2.5 year range is reflected on the chart, and it's not just more sparse RETRO cases? 

On Tue, Feb 25, 2020 at 2:22 PM Geoffrey Manikin - NOAA Federal <geoffrey.manikin@noaa.gov> wrote:

  hi GEFSv12 SOO evaluation team-

         I've finished drafting the assignments spreadsheet for all of the GEFSv12 retrospective cases.    The assignments are based on the relevance to your agency's mission or the event having significant impacts within your NWS region. (And thanks to Emily for offering to review some of the low skill cases which didn't necessarily cause major impacts in any region).    If I've incorrectly assigned any cases, please let me know. 

          For each case for which you are listed first or are the only person listed, I'd like you to go through each of the 10 cycles for review and assign ratings on the GEFSv12 Case Assessment Form, as we discussed at our last tag-up.      Adding comments on the form is optional.    Some points to note:

   1)    If you are listed second or third (with no slashes between the names), this case will be of interest to you, but please don't make it one of the cases you go through first;  it's optional.     If, however, names are separated with a slash, I think that all listed people should assess the case.  (Example:  Hurricane Harvey is of equal interest to NHC and WPC).

   2)       As we discussed on the Friday call, the criteria for your ratings is up to you   You have plenty of parameters to review, and you have means, probabilities, spreads, and  low/spaghetti plots.      Consider in each case whether the run of GEFSv11 or GEFSv12 provided more useful information, from the viewpoint of a forecaster.

   3)      This is a big assignment.     If you can't get through all of your cases, let me know, and I'll find someone to take the extra ones.    This team, however, was advertised as a high intensity project with a significant commitment, so I'm going to keep my expectations high.    :)

  4)        That said, if you get through all of your cases and would like to do a few more, go for it!   Feel free to choose any case of interest.

  5)        I have no objection to you dividing up the cases with other people in your region/center.

  6)        In addition to submitting ratings, all participants will be asked to generate some slides showing important examples and overall impressions (good and bad), so I recommend extracting images and making notes as you go through the cases.

  7)         Ratings will be due by COB April 3.       We'll expect some short presentations from team members on Friday afternoon calls starting in a few weeks.

            Let me know if you have any questions or concerns, and thank you.

                                     -Geoff

--

Jack Settelmaier

Digital Techniques Meteorologist

NOAA/NWS, Southern Region HQ

Fort Worth, TX 

Work: 682 703 3685

--
Jack Settelmaier Forecast Performance Blog Virtual Lab Forum http://vlab.noaa.gov/web/forecast-performance-blog/discussions-forums-/-/message_boards/view_message/9233768VLab.Notifications@noaa.gov