High performance query expansion using adaptive co-training

The quality of feedback documents is crucial to the effectiveness of query expansion (QE) in ad hoc retrieval. Recently, machine learning methods have been adopted to tackle this issue by training classifiers from feedback documents. However, the lack of proper training data has prevented these methods from selecting good feedback documents. In this paper, we propose a new method, called AdapCOT, which applies co-training in an adaptive manner to select feedback documents for boosting QE’s effectiveness. Co-training is an effective technique for classification over limited training data, which is particularly suitable for selecting feedback documents. The proposed AdapCOT method makes use of a small set of training documents, and labels the feedback documents according to their quality through an iterative process. Two exclusive sets of term-based features are selected to train the classifiers. Finally, QE is performed on the labeled positive documents. Our extensive experiments show that the proposed method improves QE’s effectiveness, and outperforms strong baselines on various standard TREC collections.

► We apply the co-training method for query expansion to select feedback documents of good quality.
► Extensive experiments on five large standard TREC collections have been done.
► Our proposed AdapCOT method markedly improves the effectiveness and robustness of traditional QE.
► The proposed AdapCOT method takes the quality of learned classifiers into account that makes the query expansion process more adaptive.