Electric load forecasting with recency effect: A big data approach

Temperature plays a key role in driving the electricity demand. We adopt the “recency effect”, a term drawn from psychology, to represent the fact that the electricity demand is affected by the temperatures of the preceding hours. In the load forecasting literature, the temperature variables are often constructed in the form of lagged hourly temperatures and moving average temperatures. In the past, computing power has limited the amount of temperature variables that can be used in a load forecasting model. In this paper, we present a comprehensive study to model the recency effect using a big data approach. We take advantage of modern computing power to answer a fundamental question: how many lagged hourly temperatures and/or moving average temperatures are needed in a regression model in order to capture the recency effect fully without compromising the forecasting accuracy? Using a case study based on data from the load forecasting track of the Global Energy Forecasting Competition 2012, we first demonstrate that a model with the recency effect outperforms its counterpart (a.k.a. Tao's Vanilla Benchmark Model) by 18% to 21% for forecasting the load series at the top (aggregated) level. We then model the recency effect in order to customize load forecasting models at the bottom level of a geographic hierarchy, again showing a superiority over the benchmark model of 12% to 15% on average. Finally, we discuss four different implementations of the recency effect modeling by hour of a day. In addition, this paper also presents two interesting findings: 1) the naive models are not useful for benchmark purposes in load forecasting at aggregated level due to their lack of accuracy; and 2) slicing the data into 24 pieces to develop one model for each hour is not necessarily better than building one interaction regression model using all 24 hours together.