Sparse Bayesian mixed-effects extreme learning machine, an approach for unobserved clustered heterogeneity

The well-known Extreme Learning Machine (ELM) method is widely used to analyse independent random distributed measurements. However, in some applications, the problem of modeling clustered data which contains several correlated groups is of interest. The research presented in this paper utilizes the concept of random effects in the ELM framework to model inter-cluster heterogeneity, provided the inherent correlation among the samples of a particular cluster is taken into account, as well. The proposed random effect model includes an additional variance component to accommodate correlated data and to allow for differences among clusters. Inference techniques based on Bayesian evidence procedure are derived for the estimation of model weights, random effect and residual variance parameters as well as hyperparameters. The proposed model is applied to both synthesis and real world clustered datasets. Experimental results show that our proposed method can achieve better performance in terms of accuracy and model size, compared with the previous ELM-based models (ELM, BELM, and SBELM) with the assumption of independency, in cases where the data actually have within cluster correlation. The generalization performance and sparsity of the proposed model are also superior to those of the ME-LSSVM method.