An ℓ2/ℓ1 regularization framework for diverse learning tasks

• We present a new ℓ2/ℓ1ℓ2/ℓ1 regularization framework for learning diverse tasks.
• We derive an efficient first-order algorithm for solving the problem.
• Applications of the new regularization framework are provided.

Regularization plays an important role in learning tasks, to incorporate prior knowledge about a problem and thus improve learning performance. Well known regularization methods, including ℓ2 and ℓ1 regularization, have shown great success in a variety of conventional learning tasks, and new types of regularization have also been developed to deal with modern problems, such as multi-task learning. In this paper, we introduce the ℓ2/ℓ1ℓ2/ℓ1 regularization for diverse learning tasks. The ℓ2/ℓ1ℓ2/ℓ1 regularization is a mixed norm defined over the parameters of the diverse learning tasks. It adaptively encourages the diversity of features among diverse learning tasks, i.e., when a feature is responsible for some tasks it is unlikely to be responsible for the rest of the tasks. We consider two applications of the ℓ2/ℓ1ℓ2/ℓ1 regularization framework, i.e., learning sparse self-representation of a dataset for clustering and learning one-vs.-rest binary classifiers for multi-class classification, both of which confirm the effectiveness of the new regularization framework over benchmark datasets.