11.1.2 Overall Performance Assessment
The performance of the RWFS was analyzed via comparison of the forecast skill of
individual inputs (forecast modules) to the consensus forecast. The system was designed
to optimize forecast skill using recent skill and statistical techniques, to produce better
forecasts than the individual inputs, overall.
Seasonal assessments of RMSE and bias (forecast-observation) were performed for the
ten input forecast modules and the final consensus forecast for six meteorological state
variables (temperature, dewpoint, wind speed, cloud cover, wind-u, and wind-v). Results
were based on weighted average RMSE and bias values at every lead time for all
forecasts initiated at 18 UTC over the entire season (29 December 2003 thru 24 March
2004) for all sites across the state of Iowa. Note that the mesoscale models only covered
a 15 hour forecast period, whereas the NCEP model products covered a 48 -hour period.
For all variables except cloud cover, the RWFS forecast has a lower RMSE than its
components for a majority of the lead times (0-48 hours out; Fig. 11.1). For the air
temperature forecasts, it is interesting to note that the individual inputs are clustered
between 2.0 and 2.5 C. The MM5 based air temperature forecasts were slightly more
accurate than WRF and AVN MOS is slightly better than both. For dew point
predictions, the AVN MOS again had the best skill of the individual components, while
the mesoscale models had less skill. The RMSE for the RWFS at 12 hours was 2.2 C.
The increased skill of the RWFS air and dew point temperatures over the first
demonstration is good news as these fields were important to predictions of road
temperature and frost. The RMSE of the RWFS wind speed predictions was 1.5 meters
per second (~3 mph). For cloud-cover, the ETA-DMOS was better than the RWFS
beyond 6 hours. This may have been due in part to the module weights being static
instead of dynamic for this field and the empirical weights were fixed so that the MM5
and the WRF dominated the forecasts. This approach appears to have resulted in a less
than ideal weight distribution for this parameter. For temperature and wind speed, there
is a slight increase in RMSE during the heat of the day (24-30 hr lead times). In general,
the RMSE values increased with forecast lead-time, as one would expect.
94
Previous Page
