A suite of metrics for assessing the performance of solar power forecasting

• A suite of metrics was developed for assessing the performance of solar power forecasting.
• The metrics were evaluated with solar forecasts at multiple spatial and temporal scales.
• We analyzed the sensitivity of the metrics to different types of solar forecasting improvements.
• Nonparametric statistical testing was performed to discern differences between metrics.

Forecasting solar energy generation is a challenging task because of the variety of solar power systems and weather regimes encountered. Inaccurate forecasts can result in substantial economic losses and power system reliability issues. One of the key challenges is the unavailability of a consistent and robust set of metrics to measure the accuracy of a solar forecast. This paper presents a suite of generally applicable and value-based metrics for solar forecasting for a comprehensive set of scenarios (i.e., different time horizons, geographic locations, and applications) that were developed as part of the U.S. Department of Energy SunShot Initiative’s efforts to improve the accuracy of solar forecasting. In addition, a comprehensive framework is developed to analyze the sensitivity of the proposed metrics to three types of solar forecasting improvements using a design-of-experiments methodology in conjunction with response surface, sensitivity analysis, and nonparametric statistical testing methods. The three types of forecasting improvements are (i) uniform forecasting improvements when there is not a ramp, (ii) ramp forecasting magnitude improvements, and (iii) ramp forecasting threshold changes. Day-ahead and 1-hour-ahead forecasts for both simulated and actual solar power plants are analyzed. The results show that the proposed metrics can efficiently evaluate the quality of solar forecasts and assess the economic and reliability impacts of improved solar forecasting. Sensitivity analysis results show that (i) all proposed metrics are suitable to show the changes in the accuracy of solar forecasts with uniform forecasting improvements, and (ii) the metrics of skewness, kurtosis, and Rényi entropy are specifically suitable to show the changes in the accuracy of solar forecasts with ramp forecasting improvements and a ramp forecasting threshold.