On estimating simple probabilistic discriminative models with subclasses

Discriminative subclass models can provide good estimates of complex ‘continuous to discrete’ conditional probabilities for hybrid Bayesian network models. However, the conventional approach of specifying deterministic ‘hard’ subclasses via unsupervised clustering can lead to inaccurate models. The multimodal softmax (MMS) model is presented as a new probabilistic discriminative subclass model that overcomes this unreliability. By invoking fully probabilistic latent ‘soft’ subclasses, MMS permits learning via standard statistical methods without requiring explicit clustering/relabeling of data. MMS is also shown to be closely related to the mixture of experts model and the generative Gaussian mixture classifier. Synthetic and benchmark classification results demonstrate the MMS model’s correctness and usefulness for hybrid probabilistic modeling.

► We propose multimodal softmax (MMS) models for hybrid Bayesian networks. ► Classifiers based on subclasses are probabilistically generalized by MMS. ► MMS more compact than related Gaussian mixture and mixture of expert models. ► MMS can be learned with statistical ML/MAP estimators and selection metrics. ► Accuracy and usefulness of MMS shown on difficult classification problems.