An effective use of adaptive combination of visual features to retrieve image semantics from a hierarchical image database

•Branch selection and pruning algorithms are proposed to assign semantics to images.•Algorithms use <1/3<1/3 search space to retrieve general as well as specific semantics.•Algorithms report ≈0.7 precision on ImageNet using only ≈10% of the search space.•Using 1/4 space of WANG, attained precision is 0.94 compared to Li and Wang (0.64).•The study explores the existing correlation among semantic categories in ImageNet.

Correlating semantic and visual similarity of an image is a challenging task. Unlimited possibilities of objects classification in real world are challenges for learning based techniques. Semantics based categorization of images gives a semantically categorized hierarchical image database. This work utilizes the strength of such database and proposes a system for automatic semantics assignment to images using an adaptive combination of multiple visual features. ‘Branch Selection Algorithm’ selects only a few subtrees to search from this image database. Pruning Algorithms further reduce this search space. Correlation of semantic and visual similarities is also explored to understand overlapping of semantics in visual space. The efficacy of the proposed algorithms analyzed on hierarchical and non-hierarchical databases shows that the system is capable of assigning accurate general and specific semantics to images automatically.