MSSBoost: A new multiclass boosting to semi-supervised learning

In this article, we focus on the multiclass classification problem to semi-supervised learning. Semi-supervised learning is a learning task from both labeled and unlabeled data points. We formulate the multiclass semi-supervised classification problem as an optimization problem. In this formulation, we combine the classifier predictions, based on the labeled data, and the pairwise similarity. The goal here is to minimize the inconsistency between classifier predictions and the pairwise similarity. A boosting algorithm is proposed to solve the multiclass classification problem directly. The proposed multiclass approach uses a new multiclass formulation to loss function, which includes two terms. The first term is the multiclass margin cost of the labeled data and the second term is a regularization term on unlabeled data. The regularization term is used to minimize the inconsistency between the pairwise similarity and the classifier predictions. It in fact assigns the soft labels weighted with the similarity between unlabeled and labeled examples. First, the gradient descent approach is used to solve the resulting optimization problem and derive a boosting algorithm, named MSSBoost. The derived algorithm also uses a learning optimal similarity function for a given data. The second approach to solve the optimization problem is to apply the coordinate gradient descent. The resulting algorithm is called CD-MSSB. We also use a variation of CD-MSSB in the experiments. The results of our experiments on a number of UCI and real-world text classification benchmark datasets show that MSSBoost and CD-MSSB outperform the state-of-the-art boosting methods to multiclass semi-supervised learning. Another observation is that the proposed methods exploit the informative unlabeled data.