Infinite max-margin factor analysis via data augmentation

• Jointly learning FA and SVM, MMFA is proposed to get a discriminative subspace.
• Clustering the dataset in the subspace by DPM, MMFA is extended to iMMFA.
• Thanks to the jointly learning framework, they gain good prediction performance.
• Having the data description ability, the proposed models can reject outlier samples.
• In Bayesian framework, parameters can be inferred efficiently by the Gibbs sampler.

This paper addresses the Bayesian estimation of the discriminative probabilistic latent models, especially the mixture models. We develop the max-margin factor analysis (MMFA) model, which utilizes the latent variable support vector machine (LVSVM) as the classification criterion in the latent space to learn a discriminative subspace with max-margin constraint. Furthermore, to deal with multimodally distributed data, we further extend MMFA to infinite Gaussian mixture model and develop the infinite max-margin factor analysis (iMMFA) model, via the consideration of Dirichlet process mixtures (DPM). It jointly learns clustering, max-margin classifiers and the discriminative latent space in a united framework to improve the prediction performance. Moreover, both of MMFA and iMMFA are natural to handle outlier rejection problem, since the observations are described by a single or a mixture of Gaussian distributions. Additionally, thanks to the conjugate property, the parameters in the two models can be inferred efficiently via the simple Gibbs sampler. Finally, we implement our models on synthesized and real-world data, including multimodally distributed datasets and measured radar echo data, to validate the classification and rejection performance of the proposed models.