Fast optimization algorithm on Riemannian manifolds and its application in low-rank learning

The paper proposes a first-order fast optimization algorithm on Riemannian manifolds (FOA) to address the problem of speeding up optimization algorithms for a class of composite functions on Riemannian manifolds. The theoretical analysis for FOA shows that the algorithm achieves the optimal rate of convergence for function values sequence. The experiments on the matrix completion task show that FOA has better performance than other existing first-order optimization methods on Riemannian manifolds. A subspace pursuit method (SP-RPRG(ALM)) based on FOA is also proposed to solve the low-rank representation model with the augmented Lagrange method (ALM) on the low-rank matrix variety. Experimental results on synthetic data and public databases are presented to demonstrate that both FOA and SP-RPRG (ALM) can achieve superior performance in terms of faster convergence and higher accuracy. We have made the experimental code public at https://github.com/Haoran2014.