A multi-expert based framework for automatic image annotation

•A multi-expert framework for AIA is presented that uses the idea of prototypes.•A solution to the problem of choosing effective subset of feature spaces is presented.•How to fuse labels suggested by different experts is another contribution of paper.•An implementation of proposed method containing many experimentation is performed.

Automatic image annotation (AIA) for a wide-range collection of image data is a difficult challenging topic and has attracted the interest of many researchers in the last decade. To achieve the goal of AIA, a multi-expert based framework is presented in this paper which is based on the combination of results obtained from feature space and concept space. Considering a real-world image dataset, a large storage is required; therefore, the idea of generating prototypes in both feature and concept spaces is used. The prototypes are generated in learning phase using a clustering technique. The input unlabeled images are assigned to the nearest prototypes in both feature and concept spaces, and primary labels are obtained from the nearest prototypes. Eventually, these labels are fused and final labels for a target image are chosen. Since all feature types do not describe a concept label equally, some prototypes are more effective to represent a concept and bridge the semantic gap, so a metaheuristic algorithm is employed to search for the best subset of feature types and best criterion of fusion. To evaluate the performance of the proposed framework, an example of its implementation is presented. A comparative experimental study with several state-of-the-art methods is reported on two standard databases of about 20k images. The obtained results confirm the effectiveness of the proposed framework in the field of automatic image annotation.