Robust semi-supervised extreme learning machine

The existence of outliers among labeled data is a major challenge for semi-supervised learning. An effective method to handle this problem is to employ the non-convex loss functions, which give constant penalties to outliers to avoid their negative influences. Along this line, in this paper, by adopting the non-convex squared loss function, we propose a novel robust semi-supervised learning algorithm to overcome the limitation of the classical semi-supervised extreme learning machine (SS-ELM) that it is sensitivity to outliers, termed as robust SS-ELM, or RSS-ELM for short. After expressing the non-convex squared loss function by a difference of two convex ones, RSS-ELM is effectively solved with the help of the concave-convex procedure (CCCP) approach. For the specific implementation, RSS-ELM iteratively builds the output function by solving a sequence of linear systems at each iteration. Moreover, we analyze the computational complexity of RSS-ELM, and prove its convergence and robustness from a theoretical point of view. The proposed RSS-ELM includes the conventional ELM and SS-ELM as its special cases. Extensive experiments conducted across multiple image datasets and benchmark datasets validate that RSS-ELM not only inherits the advantages of semi-supervised learning, but also enjoys the merit of robustness.