Modeling continuous visual features for semantic image annotation and retrieval

Automatic image annotation has become an important and challenging problem due to the existence of semantic gap. In this paper, we firstly extend probabilistic latent semantic analysis (PLSA) to model continuous quantity. In addition, corresponding Expectation–Maximization (EM) algorithm is derived to determine the model parameters. Furthermore, in order to deal with the data of different modalities in terms of their characteristics, we present a semantic annotation model which employs continuous PLSA and standard PLSA to model visual features and textual words respectively. The model learns the correlation between these two modalities by an asymmetric learning approach and then it can predict semantic annotation precisely for unseen images. Finally, we compare our approach with several state-of-the-art approaches on the Corel5k and Corel30k datasets. The experiment results show that our approach performs more effectively and accurately.

Research highlights► We propose continuous PLSA (probabilistic latent semantic analysis), which extend PLSA to model continuous quantity. In addition, corresponding EM (Expectation–Maximization) algorithm is derived to determine the model parameters. ► In order to deal with the data of different modalities in terms of their characteristics, we present a semantic annotation model which employs continuous PLSA and standard PLSA to model visual features and textual words respectively. ► The semantic annotation model learns the correlation between visual and textual modalities by an asymmetric learning algorithm. So it can predict semantic annotation precisely for unseen images. ► We compare our approach with several state-of-the-art approaches on the Corel5k and Corel30k datasets. The experiment results show that our approach performs more effectively and accurately.