Semi-supervised linear discriminant analysis through moment-constraint parameter estimation

• Semi-supervised learning can lead to worse performance with more unlabeled data.
• A novel take on semi-supervised learning through intrinsic constraints is introduced.
• An affine invariant, constrained, semi-supervised approach to LDA is proposed.
• Extensive experiments show good performance and clear benefits of constrained LDA.

A semi-supervised version of classical linear discriminant analysis is presented. As opposed to most current approaches to semi-supervised learning, no additional extrinsic assumptions are made to tie information coming from labeled and unlabeled data together. Our approach exploits the fact that the parameters that are to be estimated fulfill particular relations, intrinsic to the classifier, that link label-dependent with label-independent quantities. In this way, the latter type of parameters, which can be estimated based on unlabeled data, impose constraints on the former and lead to a reduction in variability of the label dependent estimates. As a result, the performance of our semi-supervised linear discriminant is typically expected to improve over that of its regular supervised match. Possibly more important, our semi-supervised linear discriminant analysis does not show the severe deteriorations other approaches frequently display with increasing numbers of unlabeled data. This work recapitulates, corrects, extends, and revises our previous work that has been published as part of the First IAPR TC3 Workshop on Partially Supervised Learning. The main novelty it provides over our earlier work is an affine invariant approach to semi-supervised learning befitting linear discriminant analysis. Besides, more elaborate and convincing experimental evidence of the potential of our general approach is provided. We essentially believe that the general principle of intrinsic constraints is of interest as such and may inspire other novel semi-supervised methods.