Prediction and structural uncertainty analyses of artificial neural networks using hierarchical Bayesian model averaging

- Hierarchical Bayesian model averaging improves the generalization ability of ANNs.
- HBMA uses parsimony principle to combine ANNs and outperforms committee machines.
- HBMA enables to quantify uncertainties of ANN model parameters and structure.
- HBMA assesses the contribution of each source of uncertainty to prediction variance.

SummaryThis study adopts a hierarchical Bayesian model averaging (HBMA) method to analyze prediction uncertainty resulted from uncertain components in artificial neural networks (ANNs). The HBMA is an ensemble method for prediction and is used to segregate the sources of model structure uncertainty in ANNs and investigate their variance contributions to total prediction variance. Specific sources of uncertainty considered in ANNs include the uncertainty in neural network weights and biases (model parameters), uncertainty of selecting an activation function for the hidden layer, and uncertainty of selecting a number of hidden layer nodes (model structure). Prediction uncertainties due to uncertain inputs and ANN model parameters are represented by within-model variance. Prediction uncertainties due to uncertain activation function and uncertain number of nodes for the hidden layer are represented by between-model variance. The method is demonstrated through a study that employs ANNs to predict fluoride concentration in the aquifers of the Maku area, Azarbaijan, Iran. The results show that uncertain inputs and ANN model parameters produces the most prediction variance, followed by prediction variances from uncertain number of hidden layer nodes and uncertain activation function.