Efficient construction of sparse radial basis function neural networks using L1-regularization

This paper investigates the construction of sparse radial basis function neural networks (RBFNNs) for classification problems. An efficient two-phase construction algorithm (which is abbreviated as TPCLR1 for simplicity) is proposed by using L1 regularization. In the first phase, an improved maximum data coverage (IMDC) algorithm is presented for the initialization of RBF centers and widths. Then a specialized Orthant-Wise Limited-memory Quasi-Newton (sOWL-QN) method is employed to perform simultaneous network pruning and parameter optimization in the second phase. The advantages of TPCLR1 lie in that better generalization performance is guaranteed with higher model sparsity, and the required storage space and testing time are much reduced. Besides these, only the regularization parameter and the maximum number of function evaluations are required to be prescribed, then the entire construction procedure becomes automatic. The learning algorithm is verified by several classification benchmarks with different levels of complexity. The experimental results show that an appropriate value of the regularization parameter is easy to find without using costly cross validation, and the proposed TPCLR1 offers an efficient procedure to construct sparse RBFNN classifiers with good generalization performance.