Multi-modal kernel ridge regression for social image classification

There is growing interest in social image classification because of its importance in web-based image application. Though there are many approaches on image classification, it is still a great problem to integrate multi-modal contents of social images simultaneously for classification, since the textual content and visual content are represented in two heterogeneous feature spaces. In this study, a multi-modal learning algorithm is proposed to fuse the multiple features through their correlation seamlessly. Specifically, two classification modules based on the kernel ridge regression (KRR) are learned for the two types of features, and they are integrated via a joint model. With the joint model, the classification based on visual features can be reinforced by the classification based on textual features, and vice verse. Then, an efficient optimization method is proposed to resolving the object function. The query image can be classified based on both of the textual features and visual features by combing the results of the two classifiers. Two methods are proposed to combine the classification results to obtain the final result. To evaluate the approach, extensive experiments are conducted on the real-world datasets, and the result demonstrates the superiority of our approach.