I love honest and open discussions amongst our STI community.
It's been a great thread, with lots of diverse opinions.facts, data, shared.
Late last week, it occurred to me that the event that lead to
this LONG thread, had now been observed. So, as I'll often do, I
assessed my ability to access the various forecast and observed
data, and visualize it, either via static maps or in a GIS
interface I got as far as the former, to make a few static
graphics. I AM looking into the ability to make comparative GIS
views of some of the datasets in play (I'm most interested in
creating some useful NBM trend views), but it will take a bit more
effort and thought, to gather and organize the data feeds, since
I'm doing this outside AWIPS. Please let me know if you see
anything that interests you below, and I'll share more, especially
as I liven up the ability to create GIS displays.
So, what I did was to look back at available sources of NBM,
NDFD, and I used the MRMS for observed.
Below, is the observed (MRMS 12-hr Gauge-Corr) QPE
values for the 12-hr period ending at 00UTC Apr 18th.
So, in light of this "new information" about how the
forecast verified, there are potentially another set of answers to
JJ's probing questions about what the various offices had in their
Day 7 PoP12 forecast. I'm not saying that "an NDFD
quilt" is EVER justified, however, it does lend some support
to some of the comments that came in, about forecasters wanting to
deviate away from an offered NBM solution. That is, perhaps the
offices that decided to make use of an alternative population
choice for this Day 7 forecast and lowering the PoPs were correct,
from a strict verification sense, in doing so, since not very much
rain fell over the bullseyed area centered on eastern AR.
Or, in looking at the below image, of the same MRMS QPE, but
for the 12-hr period JUST AFTER 00UTC (ending at 12UTC Apr 18),
once can see that it more so a timing issue, since the precip
basically just held off AFTER the 00UTC time that day/evening.
While I was in there, exploring THREDDS locations
from which to gather data, to recreate views, I DID want to
recreate the NDFD view that JJ posted. I was able to find the
same data that his interface made use of, and, from that, I was
able to then choose
my own coloring, filters (only show
PoP>20%), and add labels, as the NCEI Weather and Climate
Toolkit allows for. The NDFD "quilt" forecast
recreation is below.
Also possible, I now have the potential to export out data, in
GIS-friendly formats, from which to create geospatial
displays/services that may be of assistance for us to be able to
readily compare guidance sources, trends, etc. on a regular basis.
During my exploration, I was able to discover that only ~4
days of NBM data is available from MDL's "archive"
location, so that, if one wanted to regularly explore Day 7
forecasts, and watch the lead-up to an event, one would have to
capture the data in another location for it to be around.
Obviously, places like WAVE and the MDL 4-panel viewer itself
can provide SOME of the kinds of views one might like to peruse,
but they currently do not offer the raw data links.
Lastly, since I had access to some prior forecasts at the
various, I did go ahead and create static images of both the
NDFD and NBM, not from ~06Z on Apr 11th like JJ showed, but from
the more recent Apr 14th day. In the below two images, one can
see how the NBM not only pulled west its AR bullseye to now be
more over east TX, but it also made larger the POP forecast over
northern NM. Similarly, the NDFD for the same time period also
changed to a) not be as "quilty" (a bit more
consistent), but b) also shift a tad west (not much though).
Oh, and just on the off-chance we wish to make use of VLab's
capabilities to share/post NBM (and other models--FV3?)
examples, as we continue our evaluations, I have placed this
email thread response in a VLab area I've been exploring setting
up in case we wish to make use of it to gird other Dataset
Evaluations, of the kind the MEG covers weekly. Making use of
such a space not only opens up the discussion to a broader STI
community, should we wish to, but also can be opened to select
external (university/big data/etc.) collaborators, of which
there are several on the weekly MEG calls, as we work community
modeling efforts.
Thanks for listening.
FWIW, the images I've created, as well as some of the GIS
data I work on, is written to NAS drive locations like the
following, where at least SR folks can see and even write to collaboratively.
\\sr-s-nas\SR-Public\GIS\Data\NBM
---------- Forwarded message ---------
Greetings all,
This is a tough conversation to start, but I think it's
important we as the SOO community engage in open/honest
discussion. You are welcome to share this with whom you see
fit, but I want to be 100% clear that I am not the grid
police. I am however going to ask a hard question below.
Given the image below showing NDFD (top) and NBM (bottom)
12 hour PoP forecasts valid on day 7 (hour 168), help me
understand why there is such a stark contrast between NDFD
and NBM. Is there a significant difference in the
"IDSS message" as Andy Edman would say? In other
words, did we feel the NBM forecast was so far removed from
our message that we had to adjust it? I'm not being
rhetorical. I really do want to understand reasoning.
During the SOO/DOH meeting, I heard, "The NBM
represents a threat to forecasting. Forecasters are afraid
the NBM will remove them from the forecast process."
So I'd like to better understand why selecting an entire
grid area and assigning a single PoP value for the entire
area is considered "forecasting", but leaving the
NBM alone in a case like this is considered, "taking
the forecaster out of the process."
Again, I'm trying to understand reasoning here so we
can determine if these forecasts are based on:
1. Office grid philosophy (we only do blanket PoPs in
the extended)
2. Science (we don't have the skill to show such
precision on day 7 - blanket PoPs verifies better)
3. Desire to show something different than the NBM (I
have to make some kind of edit or else I'm not doing my job)
As an FYI, we (SRH) are going to focus more on these
kinds of scenarios in the future. We need to start
providing that accountability you all asked for at the meeting.
I also want to know if there are real scientific
concerns so we can address them with data/evidence.
I'm starting with just messaging the SOOs because I
want to talk science first. Eventually though, I want to
promote transparency and send these thoughts to a larger audience.
Thanks all!
JJ
--
John
J. Brost
Science and Training Branch
Chief
SRH, Fort Worth, Texas
(682) 703-3767
--
Jack Settelmaier
(NRAP) Technical Lead, NOAA Big Data Project
Digital Techniques Meteorologist
NOAA/NWS, Southern Region HQ
Fort Worth, TX
Work: 682 703 3685
