A new pruning method for extreme learning machines via genetic algorithms

• We propose a pruning method for ELM using genetic algorithms (GA).
• Our proposal estimates the leave-one-out (LOO) error using the PRESS statistic.
• Our proposal, called GAP-ELM, was tested on 7 real world datasets.
• GAP-ELM was compared with MLP and RBF neural networks and showed competitive results.

Extreme learning machine (ELM) is a recently proposed learning algorithm for single hidden layer feedfoward neural networks (SLFN) that achieved remarkable performances in various applications. In ELM, the hidden neurons are randomly assigned and the output layer weights are learned in a single step using the Moore-Penrose generalized inverse. This approach results in a fast learning neural network algorithm with a single hyperparameter (the number of hidden neurons). Despite the aforementioned advantages, using ELM can result in models with a large number of hidden neurons and this can lead to poor generalization. To overcome this drawback, we propose a novel method to prune hidden layer neurons based on genetic algorithms (GA). The proposed approach, referred as GAP-ELM, selects subset of the hidden neurons to optimize a multiobjective fitness function that defines a compromise between accuracy and the number of pruned neurons. The performance of GAP-ELM is assessed on several real world datasets and compared to other SLFN and a well known pruning method called Optimally Pruned ELM (OP-ELM). On the basis of our experiments, we can state that GAP-ELM is a valid alternative for classification tasks.

Figure optionsDownload as PowerPoint slide