A new R2-based metric to shed greater insight on variable importance in artificial neural networks

Artificial neural networks (ANNs) represent a powerful analytical tool designed for predictive modeling. However the shortage of straightforward and reliable approaches for calculating variable importance and characterizing predictor-response relationships has likely hindered the broader use of ANNs in ecology. Two such metrics - product-of-connection-weights (PCW) and product-of-standardized-weights (PSW) have received much attention in the published literature. A recent paper (Fischer, in press, Ecological Modelling) found that PSW was comparable to PCW for retrieving variable importance values in linear models - seemingly overturning the conclusions of Olden et al. (2004, Ecological Modelling) - and that PSW was superior to PCW in nonlinear models. In this paper we call into question the findings of Fischer (in press) and more importantly we explain why neither PCW nor PSW are universally good measures of variable. Next, we advance the field by proposing a new permutational R2-based variable importance metric and show that it accurately estimates the proportion of the total variance in the response variable that is uniquely associated with each predictor variable in both linear and non-linear data contexts. By enabling ecologists to measure relative strengths of predictor variables in a transparent and straightforward way, this metric has the potential to help widen the use of ANNs in ecology.