An adaptive ensemble of on-line Extreme Learning Machines with variable forgetting factor for dynamic system prediction

A demand for predictive models for on-line estimation of variables is increasing in industry. As industrial processes are time-varying, on-line learning algorithms should be adaptive to capture process changes. On-line ensemble methods have been shown to provide better generalization performance than single models in changing environments. However, most on-line ensembles do not include and exclude models during on-line operation. As a result, the ensembles have limited adaptation capability. Moreover, a higher performance can be obtained by combining a selected set of most relevant models of the ensemble for the current situation, rather than combining all the models. This paper proposes a new on-line learning ensemble of regressor models using an ordered aggregation (OA) technique which is able to provide on-line predictions of variables in changing environments. OA dynamically selects an optimal size and composition of a subset of models based on the minimization of the ensemble error on the newest sample. The proposed strategy overcomes the problem of defining the optimal ensemble size, and in most cases it obtains better performance than aggregating all the models. Models are added or removed for assuring adaptation of the ensemble in changing environments. Furthermore, this paper proposes and integrates a new on-line Extreme Learning Machine (ELM) neural network model with variable forgetting factor (FF) using the directional FF method which shows superior performance in industrial applications when compared to the well-known On-line Sequential ELM (OS-ELM) algorithm. Experiments are reported to demonstrate the performance and effectiveness of the proposed methods.