Calibrating artificial neural networks by global optimization

Artificial neural networks (ANNs) are used extensively to model unknown or unspecified functional relationships between the input and output of a “black box” system. In order to apply the generic ANN concept to actual system model fitting problems, a key requirement is the training of the chosen (postulated) ANN structure. Such training serves to select the ANN parameters in order to minimize the discrepancy between modeled system output and the training set of observations. We consider the parameterization of ANNs as a potentially multi-modal optimization problem, and then introduce a corresponding global optimization (GO) framework. The practical viability of the GO based ANN training approach is illustrated by finding close numerical approximations of one-dimensional, yet visibly challenging functions. For this purpose, we have implemented a flexible ANN framework and an easily expandable set of test functions in the technical computing system Mathematica. The MathOptimizer Professional global–local optimization software has been used to solve the induced (multi-dimensional) ANN calibration problems.

► Artificial neural networks (ANNs) are used extensively to model unknown or unspecified functional relationships between the input and output of a ‘‘black box’’ system. ANNs replace the unknown functional relationships by adaptively constructed approximating functions.
► In order to apply the generic ANN concept to actual model fitting problems, an essential requirement is to find the “best possible” ANN parameterization with respect to a given training data set. Such calibration of ANNs in many cases requires global optimization methodology and software.
► Our study presents the Lipschitz Global Optimizer (LGO) solver suite, in its MathOptimizer Professional implementation linked to Mathematica, to solve non-trivial ANN model calibration examples.