Robust activation function and its application: Semi-supervised kernel extreme learning method

Semi-supervised learning is a hot topic in the field of pattern recognition, this paper analyzes an effective classification algorithm – Extreme Learning Machine (ELM). ELM has been widely used in the applications of pattern recognition and data mining for its extremely fast training speed and highly recognition rate. But in most of real-world applications, there are irregular distributions and outlier problems which lower the classification rate of ELM (kernel ELM). This is mainly because: (1) Overfitting caused by outliers and unreasonable selections of activation function and kernel function and (2) the labeled sample size is small and we do not making full use of the information of unlabeled data either. Against problem one, this paper proposes a robust activation function (RAF) based on analyzing several different activation functions in-depth. RAF keeps the output of activation function away from zero as much as possible and minimizes the impacts of outliers to the algorithm. Thus, it improves the performance of ELM (kernel ELM); simultaneously, RAF can be applied to other kernel methods and a neural network learning algorithm. Against problem two, we propose a semi-supervised kernel ELM (SK-ELM). Experimental results on synthetic and real-world datasets demonstrate that RAF and SK-ELM outperform the ELM which use other activation functions and semi-supervised (kernel) ELM methods.