Musings on multilinear fitting

We show that the problems of approximating tensors and multivariate functions as a sums of (tensor) products of vectors/functions can be considered in a unified framework, thus exposing their common multilinear structure. We study the alternating least squares algorithm within this framework from the orthogonal projection and gradient perspectives. We then use these perspectives to study its convergence behavior with and without regularization. Finally, we formulate the infinite dimensional version of this problem and an algorithm to compute in that context.