Locally linear ensemble for regression

Considerable research effort has been dedicated to the development of prediction models for yielding greater prediction accuracy in regression problems. Although non-linear models have achieved superior prediction accuracy by addressing the non-linearity of complex data, linear models are still favored because of their high prediction speed. In this study, a locally linear ensemble regression (LLER) is proposed in order to effectively address non-linearity while maintaining the advantage of linear models. The LLER predicts new instances based on multiple linear models that are trained on the regions that identify the local linearity of data. To achieve this, data are decomposed into several locally linear regions based on an expectation-maximization procedure, and linear models are built as local experts for each region to constitute an ensemble. We demonstrate the effectiveness of the LLER through experimental validation with benchmark datasets.