The NWS Meteorological Development Laboratory (MDL) has conducted a series of tests to evaluate potential impacts of the upcoming NMM-b implementation on the performance of the existing NAM MOS system. Specifically, forecasts from the currently-operational NAM MOS system were compared to MOS forecasts obtained by evaluating the NAM MOS equations as applied to output from parallel runs of the NMM-b over the same time period. We verified operational and parallel MOS forecasts of 2-m temperature and dewpoint, local maximum/minimum (max/min) temperature, probability of precipitation (PoP), and wind speed and direction at 335 selected sites over the CONUS, Alaska, Hawaii, and Puerto Rico. These elements were chosen because experience has shown that MOS guidance for these elements are generally the most affected by changes to the underlying Numerical Weather Prediction (NWP) model. These tests were conducted for complete, 6-month warm and cool seasons comprising the periods of April-September 2010, and October 2010-March 2011, respectively. The results are detailed in the slide presentations below:

Cool Season Evaluation (PDF)
Warm Season Evaluation (PDF)

In general, output from a new configuration of an NWP model cannot be substituted for output from the original model (as it existed during MOS system development) without degrading the accuracy of the MOS guidance. However, these impacts seem to be mitigated if the stochastic properties of the new model version are similar to the original, or if the MOS dependent sample is at least partly comprised of data from the model's currently-operational configuration. (See Antolik and Baker 2009 (PDF) for a more complete discussion of issues pertaining to MOS and evolving NWP models.)

The current NAM MOS system is a "hybrid" in the sense that dependent data from three different mesoscale model configurations (eta-coordinate, and the final two operational configurations of the NMM) were used to develop the statistical relationships. After NMM-b implementation, the NAM MOS system will no longer contain dependent data from the operational configuration of the model. Not surprisingly, our results indicate that the performance of the NAM MOS will be degraded as a result. However, these effects are not nearly as dramatic as those seen during the transition from the NCEP eta-coordinate model to the NMM, likely owing to the greater similarity between the NMM and NMM-b.

Cool- and warm-season parallel test results are similar, with the magnitude of the effects somewhat greater in the cool season, as would be expected given the overall synoptic conditions. NAM and NMM-b parallel MOS forecasts of wind and PoP suggest that the impact of NMM-b implementation on NAM MOS guidance for these elements will be minimal. However, analysis of the results for temperature, max/min, and dewpoint suggests that the NMM-b implementation will introduce a slight cool, dry bias relative to the behavior of the existing NAM MOS system. Overall, this effect is generally less than 1° F, but this seems to be enough to eliminate its current performance advantage over the GFS MOS, which is readily seen in the verifications. In our opinion, these impacts are not severe enough to warrant delay of the NMM-b implementation. However, MDL may consider future redevelopment of the NAM MOS temperature and dewpoint equations to include parallel output from the NMM-b, resources permitting.

Mark Antolik
Allison Monarski