Evolutionary multi-objective generation of recurrent neural network ensembles for time series prediction

Ensembles have been shown to provide better generalization performance than single models. However, the creation, selection and combination of individual predictors is critical to the success of an ensemble, as each individual model needs to be both accurate and diverse. In this paper we present a hybrid multi-objective evolutionary algorithm that trains and optimizes the structure of recurrent neural networks for time series prediction. We then present methods of selecting individual prediction models from the Pareto set of solutions. The first method selects all individuals below a threshold in the Pareto front and the second one is based on the training error. Individuals near the knee point of the Pareto front are also selected and the final method selects individuals based on the diversity of the individual predictors. Results on two time series data sets, Mackey-Glass and Sunspot, show that the training algorithm is competitive with other algorithms and that the final two selection methods are better than selecting all individuals below a given threshold or based on the training error.