GEFS_faq - EMC
BACKGROUND INFO ON NCEP GLOBAL ENSEMBLE PRODUCTS & DATA SOURCES
The colors on these figures measure the relative predictability associated with ensemble mean forecasts (contours) on a scale from 0 to 100. Relative predictability is measured by the number of ensemble members falling into the same, climatologically equally (10%) likely bin in which the ensemble mean falls. Quantitatively, dark blue (red) areas are associated with 10% of all cases exhibiting the lowest (highest) level of predictability at any lead time (Fig. 1), while intermediate colors mark areas with predictability values in between.
Verification studies indicate that the more ensemble members predict an event, the more likely that event verifies. Forecast probability values, marked by the same colors, are based on past verification statistics, reflecting the relative frequency at which the verifying analysis falls into a climate bin with a given number of ensemble members. The probability values reflect temporal/spatial variations in the degree of predictability arising due to initial value uncertainty in a chaotic system, and also account for the effect of using an imperfect model. While probability values, due to the overall loss of predictability, tend to approach the asymptotic value of 10% climate probability with increasing lead time, the range of relative predictability valules is kept unchanged by design.
Verification studies indicate that the more ensemble members predict an event, the more likely that event verifies. Forecast probability values, marked by the same colors, are based on past verification statistics, reflecting the relative frequency at which the verifying analysis falls into a climate bin with a given number of ensemble members. The probability values reflect temporal/spatial variations in the degree of predictability arising due to initial value uncertainty in a chaotic system, and also account for the effect of using an imperfect model. While probability values, due to the overall loss of predictability, tend to approach the asymptotic value of 10% climate probability with increasing lead time, the range of relative predictability valules is kept unchanged by design.
In these charts, the probability that 24-hour precipitation amounts over a 2.5x2.5 lat-lon grid box will exceed a certain threshold values is given. The forecast probability is estimated directly from the 17-member global ensemble. At each gridpoint, the number of ensemble members having a 24-hour precipitation amount greater than the limit considered is counted (M) and the probability is expressed as 100*(M/17). In addition to the color shading, the 5, 35, 65 and 95% probability isolines are also drawn. If the NCEP MRF model has a bias or the perturbed ensemble forecasts do not span the range of uncertainty perfectly, the raw probability values from the ensemble may be biased as well. So the products should be used and evaluated with this in mind. We plan to calibrate the forecast probabilities based on observed rainfall data after which the new, more reliable probability values will be displayed. In particular we expect that the calibrated probability forecasts will be less sharp, i. e., the probabilities close to 0 and 100 will be used less frequently. For the purpose of comparison, plots of the MRF precipitation forecasts are included for the US region.
The red and blue curves on Fig. 2 summarize verification statistics corresponding to the most and least predictable 10-15% of all cases, marked as red and blue areas respectively on the relative measure of predictability charts.
Note that at longer lead times the low- and medium predictability cases show similar probability values. The relative measure of predictability charts can help users take advantage of the ensemble's most important benefit, its ability to objectively identify cases when the forecasts are either below or above the average level of predictability (marked by a green curve on Fig. 2).
Note that at longer lead times the low- and medium predictability cases show similar probability values. The relative measure of predictability charts can help users take advantage of the ensemble's most important benefit, its ability to objectively identify cases when the forecasts are either below or above the average level of predictability (marked by a green curve on Fig. 2).
Statistical Downscaling Approach and its Application - PPT - 12 Apr 2007
Tropical-Extratropical Interactions Case Study - PPT - 23 Nov 2005
WSR-2005 - PPT - 23 Nov 2005
Advanced Warnings for High Impact Weather - PPT - 7 Nov 2005
Adaptive Use of AQUA Brightness Temperatures in Winter Storm Recon Case Studies - PPT - 11 Oct 2005
Probabilistic Forecasting - PDF - 28 Mar 2005
Data Assimilation and Ensemble Forecasting - PDF - 28 Mar 2005
Predictability Research with Indian Monsoon Examples - PDF - 28 Mar 2005
North American Ensemble Forecast System (NAEFS) - PDF - 28 Mar 2005
Ensemble Forecasting at NCEP - PDF - 28 Mar 2005
Weather Forecast Process and Uncertainty - PDF - 28 Mar 2005
WSR-2004 - PDF - 23 Nov 2005 (updated)
General Description of the Weather Forecast Process with Emphasis on Forecast Uncertainty - PDF - 22 Feb 2005
Present status and future directions for ensemble forecasting at NCEP - PDF - June 16, 2004
Review of North American Ensemble Forecast System - PDF - June 3, 2004
Probabilistic Forecasts and their Verification - PPT - 11 May 2004
Performance of Experimental Ensemble Forecasts with Multiple Versions of NCEP's GFS Model - PPT - January 2004
Results from the WSR 2002-2003 - PPT - January 2004
Overview of Predictability Related Work at NCEP - PDF - November 2003
THORPEX - PPT - 5 November 2003
The North American Ensemble Forecast System PPT - 30 September 2003
Recent Changes in the NCEP Global Ensemble Forecast System - PPT - 19 September 2003
Track Verification - PPT - 4 September 2003
Predicting Predictability - PPT - 19 February 2003
Methods to Evaluate Probabilistic and Ensemble Forecasts - PPT - 5 November 2002
Intercomparison of the Canadian, ECMWF, and NCEP Ensemble Forecast Systems - PPT
Ensemble Forecasting at NCEP - PPT
NOAA THORPEX Initiative - PPT
Validation of Probabilistic Forecasts - PPT - September 2002
Adaptive Observations at the NWS: The Winter Storm Reconnaissance Program - PPT - July 2002
Global Ensemble Forecasting at NCEP - PPT - November 2000 (Short Version)
Ensembles in Climate Forecasting - PPT - March 2002
On the Use and Value of Probabilistic Forecasts - web page
Bias-Corrected ProbabilistIC QPF Forecasts - PPT - GCIP/GAPP
ENSEMBLE FORECASTING AT NCEP - web page
Economic Value of Forecasts - web page
- Variations in Forecast Reliability
- A Weather Event with Exceptionally High Predictability.
- A "False Alarm" Case with Low Predictability
- Impact of Targeted Observations, 2000020900
- Can Regime Changes be Predicted?
- Regime Change Forecast at 15-day Lead Time
- The Snow Storm of 4-6 March 2001 -Can We Foresee Difficulties?
- A Highly Predictable Heavy Precipitation Event
- A Case With a Highly Predictable Small, and a Poorly Predictable Large Scale Feature
- Assessing High and Low Predictability in Real Time
After a little statistical postprocessing (that can be done operationally and that takes care of the bias in probabilities arising due to model/ensemble problems) probabilistic forecasts from the global ensemble are very reliable. If the ensemble suggests an event will happen with 80% (or 10%) probability, that type of event will verify with roughly 80% (or 10%) of such forecast cases! Please see examples of reliability diagrams for 12-24 hr and 120 hr forecasts.
We get similarly reliable probabilistic forecasts at all lead times out to 15 days. Note, however, that at longer lead times forecast probability values will generally be further and further restricted toward values near climatological probabilities, reflecting the loss of predictability at longer lead times.
On different days the loss of predictability, however, occurs at various lead times: for example, 10-15% OF THE TIME A 12-DAY FORECAST CAN BE AS GOOD, OR A 1-DAY FORECAST CAN BE AS POOR AS AN AVERAGE 4-DAY FORECAST!
And the ensemble can reliably identify these cases in advance. By using the full probability information from the ensemble (red line), as compared to using categorical forecasts from the control forecasts (green and blue lines), the information content of a 5-day forecast can be more than doubled; or a 7.5 day forecast with the full ensemble-based probablity distribution has as much information as a 5-day categorical forecast guidance based on a single deterministic run.
ENSEMBLE NORMALIZED SPREAD
Normalized spread is computed by averaging the spread values at every gridpoint separately for the most recent 30 days, for each lead time. Today's ensemble spread then is divided by the average spread (for each gridpoint and lead time separately). Normalized spread values below (above) 1 represent areas where today's spread is below (above) the typical spread over the most recent 30 day period, With this normalization the effect of different lead times and different geographical areas is eliminated. Consequently the forecasters can identify areas with below and above average uncertainty in the forecast at any lead time and at any geographical location.