Unconstrained large margin distribution machines

Large margin distribution machines (LDMs) maximize the margin mean and minimize the margin variance, and show good generalization performance compared to support vector machines (SVMs). But because two additional hyperparameters are necessary, model selection needs more time. In this paper we propose unconstrained large margin distribution machines (ULDMs). In the ULDM, the objective function is the sum of the margin mean (a linear term), the margin variance (a quadratic term), and the weighted regularization term (a quadratic term), and constraints are not included. Therefore, the solution is expressed by a set of linear equations with one hyperparameter for the regularization term. Theoretical analysis proves that the decision boundary between two classes passes through the mean of all mapped training data if the numbers of training data of both classes are the same. The case where the numbers are different is analyzed for a one-dimensional input and how the decision boundary is determined is clarified. Using benchmark data sets, we show that the generalization performance of ULDMs is comparable to or better than that of SVMs.