A fast and effective principal singular subspace tracking algorithm

In this paper, we propose a fast and effective neural network algorithm to perform singular value decomposition (SVD) of a cross-covariance matrix between two high-dimensional data streams. Firstly, we derive a dynamical system from a newly proposed information criterion. This system exhibits a single stable stationary point if and only if the weight matrices of the left and right neural networks span the left and right principal singular subspace of a cross-covariance matrix, respectively, and the other stationary points are (unstable) saddle points. Then, a principal singular subspace (PSS) tracking algorithm is obtained from the dynamical system. Moreover, convergence analysis shows that the proposed algorithm converges to a stationary point that relates to the principal singular values. Thus, compared with traditional algorithms who can only track the PSS, the proposed algorithm can not only track the PSS but also estimate all of the corresponding principal singular values based on the extracted subspace. Finally, numerical simulations and practical application are carried to further demonstrate the efficiency of the proposed algorithm.