A bilinear formulation for vector sparsity optimization

Sparsity plays an important role in many fields of engineering. The cardinality penalty function, often used as a measure of sparsity, is neither continuous nor differentiable and therefore smooth optimization algorithms cannot be applied directly. In this paper we present a continuous yet non-differentiable sparsity function which constitutes a tight lower bound on the cardinality function. The novelty of this approach is that we cast the problem of minimizing the new sparsity function as a problem with a bilinear objective function. We present a numerical comparison to other sparsity encouraging penalty functions for several applications. Additionally, we apply the techniques developed to minimize an objective function with a truncated hinge loss function. We present highly competitive results for all of the applications.