Semantic consistency hashing for cross-modal retrieval

The task of cross-modal retrieval is to query similar objects in dataset of multi-modality, such as using text to query images and vice versa. However, most of existing works suffer from high computational complexity and storage cost in large-scale applications. Recently, hashing method mapping the high-dimensional data to compact binary codes has attracted a lot of concerns due to its efficiency and low storage cost over large-scale dataset. In this paper, we propose a Semantic Consistency Hashing (SCH) method for cross-modal retrieval. SCH learns a shared semantic space simultaneously taking both inter-modal and intra-modal semantic correlations into account. In order to preserve the inter-modal semantic consistency, an identical representation is learned using non-negative matrix factorization for the samples with different modalities. Meanwhile, neighbor preserving algorithm is adopted to preserve the semantic consistency in each modality. In addition, an effective optimal algorithm is proposed to reduce the time complexity from traditional O(N2) or higher to O(N). Extensive experiments on two public datasets demonstrate that the proposed approach significantly outperforms the existing schemes.