LAMP Verification FAQ - MDL
LAMP Verification FAQ
Q: How were the GFS-LAMP elements verified?
A: The verification techniques used to generate these plots varied by cycle. The reason for the change is due to the fact that once it was determined that the GFS-LAMP development process produced useful and reasonable results, the verification process was streamlined to allow for more efficient development of future cycles. The following notes provide more specific information on the verification process and the differences between the elements and the cycle times.
03Z, 09Z, 15Z, and 21Z
These cycle times were the first to be developed for GFS-LAMP. Equations were created using all but the most recent season of available data. This was done so that each element would be verified on a sample independent from what was used to create the implemented forecast equations. Besides IFR (see below), there are three exceptions to this method: warm season precipitation type and 03Z/15Z temperature/dewpoint. With warm season precipitation type, the sample size was too small to come up with useful verification statistics over only one season. With 03Z and 15Z temperature/dewpoint development, there was no season that was independent for both the GFS-MOS and GFS-LAMP. It was therefore decided to compare the verification scores on the dependent sample for these elements.
All Remaining Cycles
Each element was verified using the implemental equations, which utilized data from the entire sample. The equations were then verified over a season that was part of the dependent sample for both GFS-MOS and GFS-LAMP. This is true for each element except warm season precipitation type and winds. Since it was developed for GFS-MOS well before GFS-LAMP, the verified sample includes independent seasons for GFS-MOS with this element. This means that the improvement of GFS-LAMP over GFS-MOS may not accurately reflect what would be observed in real time for precipitation type. The verification results for wind speed, direction, and gust will always reflect independent data for both GFS-MOS and GFS-LAMP.
IFR was verified in a manner unique from the other elements. The implemented equations were used to create forecasts for an independent season (where possible). The 15Z cool season was verified over a dependent sample, because no independent season was available. Additionally, at times, an abbreviated season had to be used because of the inavailablity of data. Be sure to take note of the season length on the second line of the plot titles.
Q: How many stations were used to verify the data?
A: This number varied by element. As a general rule, wherever a forecast was made, the station was included in the verification.
Q: Which elements were verified and what were the scores used?
- Temperature/Dewpoint: Mean Absolute Error
- Wind Speed/Direction: Mean Absolute Error
- Wind Gust: Threat Score and Heidke Skill
- POPO/POP6/POP12: Brier Score
- Precip Type: Heidke Skill
- Total Sky Cover: Heidke Skill
- Ceiling Height: Threat Score (Four different heights)
- Visibility: Threat Score (Three different visibilities)
- Obstruction to Vision: Heidke Skill
- Precip Characteristics:Heidke Skill
- Thunderstorms: Brier Score and Threat Score
Additionally, forecasts of IFR flight conditions were verified for Threat Score, Probability of Detection (POD) and False Alarm Ratio(FAR).
Q: What is IFR?
A: IFR (Instrument Flight Rules) is a set of aicraft regulations that occur during either one (or both) of the following flight conditions:
- 1 ≤ Visibility < 3 (statute miles)
- 500 ≤ Ceiling height < 1,000 (feet)
Q: Where can I find more information about the GFS-LAMP elements and categories?