Hierarchical sparse representation with deep dictionary for multi-modal classification

Sparse representation based classification (SRC) methods have achieved many successes in pattern recognition and machine learning. In such methods, the training samples of all categories are mixed and compose a dictionary to represent the test sample via sparsity constraint. Then, the class with the minimum representation error wins for labeling the test sample. In general, SRC is more flexible and effective than many supervised learning methods. However, in some cases it is unlikely to represent the test sample accurately, which tends to undermine the classification accuracy. To alleviate this issue, we propose a hierarchical sparse representation based classification method by augmenting the single-layer sparse representation into the hierarchical representation with a deep dictionary. Specifically, the features from all training samples are first divided into several groups according to their labels. Then we employ hierarchical clustering in each group and combine them to form a deep dictionary such that the root layer includes only a certain amount of the most representative exemplars while the subsequent layers focus on characterizing the remaining individual information across different groups. Furthermore, we use the layer-after-layer residuals to encode the variation patterns across individuals in different scales. Given the deep dictionary, a hierarchical sparse representation based classification method is presented to determine the label for each test sample by iteratively representing its primary part with the exemplars in different groups but the remaining parts by the variation patterns encoded in different layers. To further improve the classification accuracy and robustness, we extend our method by taking advantage of the complementary information in multi-view features. Experiments on Multiple Features Data Set show promising results compared with the state-of-the-art classification methods.