Query difficulty estimation for image retrieval

Query difficulty estimation predicts the performance of the search result of the given query. It is a powerful tool for multimedia retrieval and receives increasing attention. It can guide the pseudo relevance feedback to rerank the image search results and re-write the query by suggesting “easy” alternatives to obtain better search results. Many techniques to estimate the query difficulty have been proposed in the textual information retrieval, but directly employing them for image search will result in poor performance. That is because image query is more complex with spatial or structural information, and the well-known semantic gap induces extra burdens for accurate estimations. In this paper, we propose a query difficulty estimation approach by analyzing the top ranked images obtained by ad hoc retrieval models. Specifically, we seamlessly integrate the language model based clarity score, the spatial consistency of local descriptors and the appearance consistency of global features. Experimental results demonstrate that the query difficulty estimated by the proposed algorithm correlates well with the actual retrieval performance. Two applications of query difficulty estimation, namely guided pseudo relevance feedback (GPRF) and selective query refinement (SQR), are also proposed from both system and user perspectives. Experimental results show that both strategies further boost the retrieval performance.