Optimization method based extreme learning machine for classification

Extreme learning machine (ELM) as an emergent technology has shown its good performance in regression applications as well as in large dataset (and/or multi-label) classification applications. The ELM theory shows that the hidden nodes of the “generalized” single-hidden layer feedforward networks (SLFNs), which need not be neuron alike, can be randomly generated and the universal approximation capability of such SLFNs can be guaranteed. This paper further studies ELM for classification in the aspect of the standard optimization method and extends ELM to a specific type of “generalized” SLFNs—support vector network. This paper shows that: (1) under the ELM learning framework, SVM's maximal margin property and the minimal norm of weights theory of feedforward neural networks are actually consistent; (2) from the standard optimization method point of view ELM for classification and SVM are equivalent but ELM has less optimization constraints due to its special separability feature; (3) as analyzed in theory and further verified by the simulation results, ELM for classification tends to achieve better generalization performance than traditional SVM. ELM for classification is less sensitive to user specified parameters and can be implemented easily.