Topology and simulations of a Hierarchical Markovian Radial Basis Function Neural Network classifier

This paper presents a Hierarchical Markovian Radial Basis Function Neural Network (HiMarkovRBFNN) model that enables recursive operations. The hierarchical structure of this network is composed of recursively nested RBF Neural Networks with arbitrary levels of hierarchy. All hidden neurons in the hierarchy levels are composed of truly RBF Neural Networks with two weight matrices, for the RBF centers and the linear output weights, in contrast to the simple summation neurons with only linear weighted combinations which are usually encountered in ensemble models and cascading networks. Thus the neural network operation in every node is exactly the same at all levels of the hierarchical integration. The hidden RBF response units are recursive. The training methods also remain the same for all levels as in a typical single RBF Neural Network. The simplicity in the neural network construction process is demonstrated by means of three textbook algorithms, namely the well known k-means clustering, the classical tree-based recursion function and the standard regularized least squares solver. Determining centers can be performed top-down and calculation of linear output weights is performed bottom-up. The framework is rather general and optimization algorithms can also be applied. Experimental simulations on various benchmark datasets show that the recursive operation for the hidden RBF response units is promising. Comparisons with the two standard model meta-learning architectures, namely Committee Machines and Cascaded Machines, reveal that the proposed method produces similar results and compares well as a combiner that can merge many HiMarkovRBFNN child nodes into one higher level parent HiMarkovRBFNN node of the same functionality.