Convergent design of piecewise linear neural networks

Piecewise linear networks (PLNs) are attractive because they can be trained quickly and provide good performance in many nonlinear approximation problems. Most existing design algorithms for piecewise linear networks are not convergent, non-optimal, or are not designed to handle noisy data. In this paper, four algorithms are presented which attack this problem. They are: (1) a convergent design algorithm which builds the PLN one module at a time using a branch and bound technique; (2) two pruning algorithms which eliminate less useful modules from the network; and (3) a sifting algorithm which picks the best networks out of the many designed. The performance of the PLN is compared with that of the multilayer perceptron (MLP) using several benchmark data sets. Numerical results demonstrate that piecewise linear networks are adequate for many approximation problems.