Stepwise estimation of common principal components

The standard common principal components (CPCs) may not always be useful for simultaneous dimensionality reduction in kk groups. Moreover, the original FG   algorithm finds the CPCs in arbitrary order, which does not reflect their importance with respect to the explained variance. A possible alternative is to find an approximate common subspace for all kk groups. A new stepwise estimation procedure for obtaining CPCs is proposed, which imitates standard PCA. The stepwise CPCs facilitate simultaneous dimensionality reduction, as their variances are decreasing at least approximately in all kk groups. Thus, they can be a better alternative for dimensionality reduction than the standard CPCs. The stepwise CPCs are found sequentially by a very simple algorithm, based on the well-known power method for a single covariance/correlation matrix. Numerical illustrations on well-known data are considered.