Label transfer via sparse representation

•We present a novel sparse representation based scene parsing algorithm.•We use boosting concept to refine estimated likelihoods.•This is the first paper to utilize sparse representation for superpixel based scene parsing.•Our approach reduces data capacity compared with nonparametric approaches.

In this paper, we present a simple and effective approach to the image parsing (or labeling image regions) problem. Inspired by sparse representation techniques for super-resolution, we convert the image parsing problem into a superpixel-wise sparse representation problem with coupled dictionaries related to features and likelihoods. This algorithm works by image-level classification with global image descriptors, followed by sparse representation based likelihood estimation with local features. Finally, Markov random field (MRF) optimization is applied to incorporate neighborhood context. Experimental results on the SIFTflow dataset support the use of our approach for solving the task of image parsing. The advantage of the proposed algorithm is that it can estimate likelihoods from a small set of bases (dictionary) whereas recent nonparametric scene parsing algorithms need features and labels of whole datasets to compute likelihoods. To our knowledge, this is the first approach that utilizes sparse representation to superpixel-based image parsing.