An incremental extreme learning machine for online sequential learning problems

A fast and outstanding incremental learning algorithm is required to meet the demand of online applications where data comes one by one or chunk by chunk to avoid retraining and save precious time. Although many interesting research results have been achieved, there are still a lot of difficulties in real applications because of their unsatisfying generalization performance or intensive computation cost. This paper presents an Incremental Extreme Learning Machine (IELM) which is developed based on Extreme Learning Machine (ELM), a unified framework of LS-SVM and PSVM presented by Hang et al. (2011) in [15]. Under different application demand and different computational cost and efficiency, three different alternative solutions of IELM are achieved. Detailed comparisons of the IELM algorithm with other incremental algorithms are achieved by simulation on benchmark problems and real critical dimension (CD) prediction problem in lithography of actual semiconductor production line. The results show that kernel based IELM solution performs best while least square IELM solution is the fastest of the three alterative solutions when the number of training data is huge. All the results show that the presented IELM algorithms have better performance than other incremental algorithms such as online sequential ELM (OS-ELM) presented by Liang et al. (2006) [8] and fixed size LSSVM presented by Espinoza et al. (2006) [11].