Robust regularized extreme learning machine for regression using iteratively reweighted least squares

Extreme learning machine (ELM) for regression has been used in many fields because of its easy-implementation, fast training speed and good generalization performance. However, basic ELM with ℓ2-norm loss function is sensitive to outliers. Recently, ℓ1-norm loss function and Huber loss function have been used in ELM to enhance the robustness. However, the ℓ1-norm loss function and the Huber loss function can also be effected by outliers because of their linear correlation with the errors. Moreover, existing robust ELM methods only use ℓ2-norm regularization or have no regularization term. In this study, we propose a unified model for robust regularized ELM regression using iteratively reweighted least squares (IRLS), and call it RELM-IRLS. We perform a comprehensive study on the robust loss function and regularization term for robust ELM regression. Four loss functions (i.e., ℓ1-norm, Huber, Bisquare and Welsch) are used to enhance the robustness, and two types of regularization (ℓ2-norm and ℓ1-norm) are used to avoid overfitting. Experiments show that our proposed RELM-IRLS with ℓ2-norm and ℓ1-norm regularization is stable and accurate for data with 0∼40% outlier levels, and that RELM-IRLS with ℓ1-norm regularization can obtain a compact network because of the highly sparse output weights of the network.