LSH-based semantic dictionary learning for large scale image understanding

• A semantic dictionary based on the LSH is proposed to solve the VPCP problem.
• Online dictionary learning is proposed to deal with lots of training samples.
• A combination of SPM and semantic dictionary is introduced for image description.

Large scale image understanding is a challenging but significant task to comprehend image contents on the internet. The de-facto standard methods based on machine learning or computer vision still suffer from a phenomenon of visual polysemia and concept polymorphism (VPCP). To resolve the VPCP, semantic dictionary has been proposed to characterize the membership distribution between visual appearances and semantic concepts. In this paper, we propose an online semantic dictionary learning algorithm on the base of both locality sensitive hashing (LSH) and stochastic approximations, which can scale up to large scale datasets with millions of training samples and speedup the efficiency of follow-up processing. With the help of the LSH-based semantic dictionary, we develop an extension of the spatial pyramid matching (SPM) kernel method by generalizing the dictionary as a basic semantic description. The efficiency of our approach is validated in the experiments of web-scale semantic image search and image classification on the ImageNet dataset and Caltech-256 dataset.