Quantifying forecast quality of IT business value

This article discusses how to quantify the forecasting quality of IT business value. We address a common economic indicator often used to determine the business value of project proposals, the Net Present Value (NPV). To quantify the forecasting quality of IT business value, we develop a generalized method that is able to account for asymptotic cases and negative valued entities. We assess the generalization with real-world data of four organizations together consisting of 1435 IT assets with a total investment cost of 1232+ million Euro for which 6328 forecasts were made. Using the generalized method, we determine the forecasting quality of the NPV, along with the benefits and cost using real-world data of another 102 IT assets with a total business value of 1812 million Euro. For the real-world case study, we will find that the quality of the forecasted NPVs is lower than the forecasted benefits, which is again lower than the forecasting quality of the cost. Also, we perform a sensitivity analysis to investigate the impact on the quality of an asset’s forecasted NPV when the forecasting quality of benefits or cost improves. Counterintuitively, it turned out in this case study that if the quality of cost forecasts would improve, the overall quality of its NPV predictions would degrade. This underlines the importance of both accurate cost and benefit predictions. Finally, we show how to use the quantified forecast information to enhance decision information using two simulation examples.

► We propose a method to quantify the forecasting quality of IT business value.
► This method allows for asymptotic cases and negative values.
► We determine the forecasting quality of the NPV of 102 real-world IT assets.
► Their quality turns out to be lower than the forecast quality of benefits and cost.
► Counterintuitively, improving cost forecasts can degrade the NPV forecast quality.