Online variational learning of generalized Dirichlet mixture models with feature selection

Three frequently recurring themes in machine learning, data mining and related disciplines are clustering, feature selection and online learning. Motivated by the importance of these themes which are generally interrelated, we propose a statistical framework for simultaneous online clustering and feature selection using finite generalized Dirichlet mixture model. The proposed framework allows to control overfitting by, dynamically and simultaneously, adjusting the mixture model's parameters, number of components and the features weights. We describe a principled variational approach for learning the parameters of the proposed statistical model. Results on both synthetic data and real applications involving online documents and images clustering show the merits of the proposed approach.