Using the fractions skill score to assess the relationship between an ensemble QPF spread and skill

The numerical weather prediction model LM COSMO was employed to study the regional ensemble forecast of convective precipitation. The relationship between ensemble spread and ensemble skill and the possibility of estimating ensemble skill on the basis of ensemble spread were investigated. Five convective events that produced heavy local rainfall in the Czech Republic were studied. The LM COSMO was run with a horizontal resolution of 2.8 km and an ensemble of 13 forecasts was created by modifying the initial and boundary conditions. Forecasts were verified by gauge-adjusted radar-based rainfalls. Ensemble skill and ensemble spread were determined using the Fractions Skill Score (FSS), which depended on the scale of the elementary area and on a precipitation threshold. The spread represents the differences between the control forecast and the forecasts provided by each ensemble member, while the skill evaluates the difference between the precipitation forecast and radar-based rainfalls. In this study, the ensemble skill is estimated on the basis of the ensemble spread. The numerical experiments used the FSS-based skill and spread values related to four events to estimate the skill–spread relationship. The relationship was applied to a fifth event to estimate the QPF ensemble skill given the ensemble FSS-based spread. The evaluation was performed separately for 1, 3, and 6 h rainfalls using various threshold values and scales. The absolute frequencies of the differences between diagnostic and prognostic FSS-based skill show that all of the distributions have means and medians close to zero and that the interquartile ranges are between 0.10 and 0.30. The results indicate that 67% of all the fitted FSS-skill values were within 0.15 of the true values. One of five events showed a marked overestimation of the prognostic FSS-skill so that only 39% of skill values were fitted. At the other four events, the 75% of predicted FSS-skill values were in the range of 0.15 of the diagnosed FSS-skill. The results appear to be encouraging; however, tests with more extended data are needed to confirm the potential of the technique.