Multiple feature kernel hashing for large-scale visual search

•We propose a generic multiple feature hashing framework using multiple kernels.•Visual features are implicitly mapped and concatenated to reduce complexity.•We formulate both supervised and unsupervised hashing problems in the framework.•Alternating optimization ways efficiently learn hashing functions and the kernel space.•Experiments validate the superior performances and efficiency of the proposed approach.

Recently hashing has become attractive in large-scale visual search, owing to its theoretical guarantee and practical success. However, most of the state-of-the-art hashing methods can only employ a single feature type to learn hashing functions. Related research on image search, clustering, and other domains has proved the advantages of fusing multiple features. In this paper we propose a novel multiple feature kernel hashing framework, where hashing functions are learned to preserve certain similarities with linearly combined multiple kernels corresponding to different features. The framework is not only compatible with general types of data and diverse types of similarities indicated by different visual features, but also general for both supervised and unsupervised scenarios. We present efficient alternating optimization algorithms to learn both the hashing functions and the optimal kernel combination. Experimental results on three large-scale benchmarks CIFAR-10, NUS-WIDE and a-TRECVID show that the proposed approach can achieve superior accuracy and efficiency over state-of-the-art methods.