Design of ensemble neural network using entropy theory

Ensemble neural networks (ENNs) are commonly used neural networks in many engineering applications due to their better generalization properties. An ENN usually includes several component networks in its structure, and each component network commonly uses a single feed-forward network trained with the back-propagation learning rule. As the neural network architecture has a significant influence on its generalization ability, it is crucial to develop a proper algorithm to determine the ENN architecture. In this paper, an ENN, which combines the component networks using the entropy theory, is proposed. The entropy-based ENN searches the best structure of each component network first, and employs entropy as an automating design tool to determine the best combining weights. Two analytical functions – the peak function and the Friedman function are used to assess the accuracy of the proposed ensemble approach. Then, the entropy-based ENN is applied to the modeling of peak particle velocity (PPV) damage criterion for rock mass. These computational experiments have verified that the proposed entropy-based ENN outperforms the simple averaging ENN and the single NN.

► We apply the entropy to combine the component neural networks.
► The entropy reduces the over-fitting of the component networks.
► The entropy-based weights improve the overall performance of the ENN.
► The Newton’s method is used for the optimization problem.
► The ENN accuracy is verified by analytical and practical examples.