SVD revisited: A new variational principle, compatible feature maps and nonlinear extensions

In this letter a new variational principle to the matrix singular value decomposition (SVD) is proposed. It is formulated as a constrained optimization problem where two sets of constraints are expressed in terms of compatible feature maps, which are evaluated on data vectors that relate to the rows and columns of the given matrix. Provided that a compatibility condition holds the solution can be related to Lanczos' decomposition theorem. The method is further extended to nonlinear SVD, which is illustrated also on image examples.