On Bayesian principal component analysis

A complete Bayesian framework for principal component analysis (PCA) is proposed. Previous model-based approaches to PCA were often based upon a factor analysis model with isotropic Gaussian noise. In contrast to PCA, these approaches do not impose orthogonality constraints. A new model with orthogonality restrictions is proposed. Its approximate Bayesian solution using the variational approximation and results from directional statistics is developed. The Bayesian solution provides two notable results in relation to PCA. The first is uncertainty bounds on principal components (PCs), and the second is an explicit distribution on the number of relevant PCs. The posterior distribution of the PCs is found to be of the von-Mises–Fisher type. This distribution and its associated hypergeometric function, F10, are studied. Numerical reductions are revealed, leading to a stable and efficient orthogonal variational PCA (OVPCA) algorithm. OVPCA provides the required inferences. Its performance is illustrated in simulation, and for a sequence of medical scintigraphic images.