A note on sparse least-squares regression

• The goal in this article is to compute a sparse solution to a least-squares problem ‖Ax−b‖‖Ax−b‖.
• Algorithm 1 is the main algorithm to compute such a sparse solution x to ‖Ax−b‖‖Ax−b‖.
• Algorithm 1 uses Algorithm 2, a known algorithm in the literature.
• Theorem 1 provides an approximation bound for Algorithm 1.
• Theorem 1 uses Lemma 3 (known result) and Lemma 4 (the main contribution of this work).

We compute a sparse   solution to the classical least-squares problem minx‖Ax−b‖2, where A is an arbitrary matrix. We describe a novel algorithm for this sparse least-squares problem. The algorithm operates as follows: first, it selects columns from A, and then solves a least-squares problem only with the selected columns. The column selection algorithm that we use is known to perform well for the well studied column subset selection problem. The contribution of this article is to show that it gives favorable results for sparse least-squares as well. Specifically, we prove that the solution vector obtained by our algorithm is close to the solution vector obtained via what is known as the “SVD-truncated regularization approach”.