An entropy-based query expansion approach for learning researchers’ dynamic information needs

•Proposing an entropy-based query expansion with a reweighting (E_QE) approach.•The E_QE used to learn the researchers’ evolving information needs at different levels of topic change.•Adopting a simulation pseudo-relevance feedback process to evaluate the proposed approach.•The results show that the proposed E_QE approach can achieve better search results than the TFIDF.

Literature survey is one of the most important steps in the process of academic research, allowing researchers to explore and understand topics. However, novice researchers without sufficient prior knowledge lack the skills to determine proper keywords for searching topics of choice. To tackle this problem, we propose an entropy-based query expansion with a reweighting (E_QE) approach to revise queries during the iterative retrieval process. We designed a series of experiments that consider the researcher’s changing information needs during task execution. Three topic change situations are considered in this work: minor, moderate and dramatic topic changes. The simulation-based pseudo-relevance feedback technique is applied during the search process to evaluate the effectiveness of the proposed approach without the intervention of human effort. We measured the effectiveness of the TFIDF and E_QE approaches for different types of topic change situations. The results show that the proposed E_QE approach achieves better search results than the TFIDF, helping researchers to revise queries. The results also confirm that the E_QE approach is effective when considering the relevant and irrelevant pages during the relevance feedback process at different levels of topic change.