The object of verifying these or any forecasts is to determine if predictors are capable of
adding quality to the forecast service. The concept behind ensemble modeling is that a
properly combined group of forecasts can provide a better forecast than any single
predictor in the ensemble. However, for this advantage to be consistently realized, the
ensemble should consist of predictors (model forecasts) each of which is equally likely to
be the most accurate predictor for a given forecast event. Furthermore, the ensemble
members should be as different as possible, so that errors among the different models are
uncorrelated. In other words, if the models are all making the same kinds of errors for
the same reasons, the ensemble forecast will not be optimum. Thus, verification efforts
are also aimed at determining progress toward the goal of good dispersion among the
ensemble members.
12-h forecast verification statistics (RMS and bias) for the state variables temperature,
wind speed, and dewpoint are given in the following table:
TRmperature (K) Wind speed (m/s)
e
Dewpoint (K)
MS
bias
RMS
bias
RMS
bias
MM5
3.2
+0.2
2.4
+1.6
3.7
+1.5
WRF
3.0
+1.3
2.3
+1.3
3.7
+2.2
Eta
2.7
+0.5
2.7
-0.2
2.6
+1.7
Shelter-level (2m) temperature and dewpoint. All the models had warm-biased 12-h
temperature forecasts; the same is true for the dewpoint forecasts. Neither of the MDSS
supplemental models (MM5 and WRF) were as consistently accurate as the large-scale
Eta model, which has the lowest RMS error. The slight advantage for the Eta model is
because it has surface flux formulations that are closely tied and tuned to its Land Surface
Modules (LSM), which characterizes the earth surface by vegetation type, roughness,
albedo, moisture content, land use, etc. By contrast, the MDSS models use LSM
information from outside sources. In the report following the 2003 MDSS Demo,
improvement in this area was identified as an opportunity for advancement in quality
because of changes expected in the LSM data preparation and parameterizations. While
the temperature forecast errors are about the same as in last year's results, the dewpoint
forecast RMS errors are two full centigrade degrees smaller than last year's errors. This
is significant improvement in humidity forecast quality from last year.
Tower-level (10m) winds. The Eta model showed almost identical wind speed errors in
this year's results compared to last year's results (not shown). However, both of the
MDSS models showed a slight improvement in wind forecast errors and may have
actually surpassed the Eta model, although these margins of error magnitude are not large
enough to claim significant advancement in this area.
132
Previous Page
