Root-quatric mixture of experts for complex classification problems

• We design a new ensemble system based on mixture of experts.
• The anti-correlation measure is augmented to error function of mixture of experts.
• The gating network assigns the weights to all of the output neurons of the experts.
• The effect of anti-correlation measure is investigated.
• Increasing anti-correlation measure will increase the diversity among the experts.

Mixture of experts (ME) as an ensemble method consists of several experts and a gating network to decompose the input space into some subspaces regarding to the experts specialties. To increase the diversity between experts in ME, this paper incorporates a correlation penalty function into the error function of ME. The significant of this modification is providing an occasion to encourage experts to specialize on different parts of the input space and to create decorrelated experts. The experimental results of this approach reveals that the impacts of this penalty function is extremely improved the diversity of experts and the tradeoff between the accuracy and the diversity in ME. Moreover in the implementation of this method, the experts are trained simultaneously and they can communicate by the aid of the correlation penalty function. The performance of the proposed method on ten classification benchmark datasets shows that the average of accuracy of this method improves 1.94%, 3.7%, and 3.74% compared with the mixture of negatively correlated experts, ME and the negative correlation learning, respectively. Thus the proposed method can be considered as a better classifier for healthy and medical problems and also when the great non-stationary data should be classified.