2008 NAM Prototype Evaluation

Overview

New MOS forecast equations based on output from NCEP's North American Mesoscale Model (NAM) have been developed for selected weather elements, namely Temperature, Dewpoint, Max and Min Temperature, Probability of Precipitation (POP), and wind speed and direction. The tables below give the results of cool- and warm-season comparative verications for three configurations - the original eta MOS system (ETA), the eta MOS equations applied to the current NAM model (ETAonNMM), and new NAM MOS system (NMM). In most cases the new NAM MOS system performed best.

Configuration and Comparison Details

We have used 2-season dependent data samples for these new NAM MOS elements. However, since the NAM had been running in the current WRF-NMM configuration for only a few months prior to system development, both the warm- and cool-season dependent samples for the NAM MOS contain a mixture of data from the older, eta-coordinate model and the WRF-NMM in its latest two operational configurations. Despite this, the new NAM MOS system performs favorably when compared to both the original eta MOS system (applied to eta-model output) and a "hybrid" system whereby the older, eta MOS equations were applied to output from the current configuration of the NMM.

The tables below give the results of cool- and warm-season comparative verifications performed for the above three systems. Verification scores are presented at our standard set of 335 verification sites for the weather elements listed. However, because of the scarcity of available developmental data, these withheld-data tests generally were limited to a period of 30-35 days for each season. Verification results on more extensive samples of independent data will be available as we progress through the spring and summer months of 2008.

Evaluation Results

Element Cool Season Plots (Images) Warm Season Plots (Images)
Maxiumum Temperature All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
Minimum Temperature All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
2-m Temperature All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
Dew Point Temperature All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
All Sites MAE
CONUS MAE
Alaska MAE
All Sites Bias
CONUS Bias
Alaska Bias
12-hr PoP All Sites Brier Score [ Missing ]
Wind Speed All Sites MAE
CONUS MAE
Alaska MAE
All Sites Heidke Skill Score
CONUS Heidke Skill Score
Alaska Heidke Skill Score
All Sites MAE
CONUS MAE
Alaska MAE
All Sites Heidke Skill Score
CONUS Heidke Skill Score
Alaska Heidke Skill Score
Wind Direction All Sites MAE
CONUS MAE
Alaska MAE
All Sites % of errors < 30 deg.
CONUS % of errors < 30 deg.
Alaska % of errors < 30 deg.
All Sites MAE
CONUS MAE
Alaska MAE
All Sites % of errors < 30 deg.
CONUS % of errors < 30 deg.
Alaska % of errors < 30 deg.

MOS