Weighted classifier ensemble based on quadratic form

•A new weighted classifier ensemble method is proposed.•An approximation form of the ensemble error is introduced.•An optimal weight vector is sought based on minimizing the approximation form.•It is converted into maximizing quadratic forms by invoking a known weight vector.•The larger the value of the quadratic form is, the lower the ensemble error is.

Diversity and accuracy are the two key factors that decide the ensemble generalization error. Constructing a good ensemble method by balancing these two factors is difficult, because increasing diversity is at the cost of reducing accuracy normally. In order to improve the performance of an ensemble while avoiding the difficulty derived of balancing diversity and accuracy, we propose a novel method that weights each classifier in the ensemble by maximizing three different quadratic forms. In this paper, the optimal weight of individual classifiers is obtained by minimizing the ensemble error, rather than analyzing diversity and accuracy. Since it is difficult to minimize the general form of the ensemble error directly, we approximate the error in an objective function subject to two constraints (∑wi=1∑wi=1 and −1