Zero-shot learning with regularized cross-modality ranking

Zero-Shot Learning tries to predict the novel class samples that do not have any labeled instances in the training stage. This is typically achieved by exploring intermediate side information to transfer knowledge from seen classes to unseen testing ones. Different approaches vary in the usage of the side information and embedding methods. However, most methods only concern the relationships among different modalities but ignore to preserve the consistency among different samples in the same modality. In this paper, we propose an approach called Regularized Cross-Modality Ranking (ReCMR) to capture the semantic information from heterogeneous sources by taking both intra-modal and inter-modal semantics into consideration. Specifically, we employ the hinge ranking loss to exploit the structures among different modalities and devise efficient regularizers to constrain the variation of the samples in the identical modality. Experimental results on the popular AwA and CUB datasets show that ReCMR significantly outperforms the state-of-the-art methods.