National Hurricane Center Forecast Verification

Contents

1. Introduction
2. Forecast verification procedures
3. Annual NHC verification reports
4. Official five-year mean errors and distributions
5. Official error trends
6. Model error trends
7. NHC official forecast error database
8. Performance measures and goals
9. References

2. Forecast verification procedures

For all operationally designated tropical cyclones in the Atlantic and eastern North Pacific basins, the NHC issues official forecasts of the cyclone's center position and maximum 1-min surface wind speed. These forecasts are issued every 6 hours (at 0300, 0900, 1500, and 2100 UTC), and each contains projections valid 12, 24, 36, 48, 72, 96, and 120 h after the forecast's nominal initial time (0000, 0600, 1200, and 1800 UTC, respectively). These forecasts can be found in two of NHC's public products, the Tropical Cyclone Forecast/Advisory and the Tropical Cyclone Discussion. Over the years there have been numerous changes to the NHC forecast products and procedures; a partial chronology is given in Table 1. Periods of record in the digital databases used to generate the verifications in this report are given in Table 2 and Table 3.

At the conclusion of each season, official forecasts are evaluated by comparison with the cyclone's "best track" database. Best track data represent NHC's analysis of a tropical cyclone, and consist of 6-hourly representative estimates of the cyclone's center location and maximum sustained wind as well as other parameters, determined by a post-storm analysis of all available storm data. Track forecast error is defined as the great-circle distance between a cyclone's forecast position and the best track position at the forecast verification time. Forecast intensity error is defined as the absolute value of the difference between the forecast and best track intensity at the forecast verifying time. Current practice is to include a forecast in the verification only if the system was a tropical or subtropical cyclone at both the forecast and the verifying time; all other stages of development (e.g., extratropical, tropical wave, remnant low) are excluded.  When forecasts from both regular and special advisories are available for the same synoptic time, current practice is to verify the regular forecast.

To facilitate comparisons between official forecasts from different storms or different years, a simultaneous (homogeneous) verification of a skill-baseline model such as CLIPER (Neumann 1972, Aberson 1998) or SHIFOR (Jarvinen and Neumann 1979, Knaff et al. 2003) is usually done. The skill baseline models can be run off of operational parameters, or they can be run post-storm using best track data. It is generally preferable to use operational parameters to assess the skill of operational forecasts; however only in recent years are the required parameters available. Very small differences in sample size can arise depending on whether an operational or best track baseline is used. Operational skill baselines were used for the verifications in Sections 3, 4, 6, and 8 below; while best-track baselines were required for the longer period data presented in Section 5 and Section 7. It should also be noted that the 5-day CLIPER model was updated in 2005, and is now based on dependant data sets covering the periods 1931-2004 for the Atlantic basin and 1949-2004 for the Eastern North Pacific basin.  In 2006, an enhancement was made to the SHIFOR model that includes a weakening of the cyclone over land; this version of the model is known as "Decay-SHIFOR5".

In addition to forecasting the location and intensity of a tropical cyclone, NHC also forecasts the extent of 34 and 50 kt winds in each of four quadrants surrounding the cyclone out to 72 h from the initial time, and the extent of 64 kt winds out to 36 h from the initial time. These "wind radii" forecasts are available in the Tropical Cyclone Forecast/Advisory. Only since 2004 have wind radii been included in the post-storm best track database, but even so, these estimates are not considered reliable enough to warrant a formal verification of the wind radii forecasts. 

Next: Annual NHC verification reports