Using author-specified keywords in building an initial reading list of research papers in scientific paper retrieval and recommender systems

- The requirements for the task of building an initial reading list in literature review are re-conceptualized and a novel retrieval technique centered on author-specified keywords of papers is proposed for this task.
- The HITS variant of the proposed technique best satisfies the requirements of the task in an offline evaluation experiment.
- The proposed technique is evaluated by 132 researchers using 14 evaluation measures in a user evaluation study.
- Relevance, Recency and Usefulness were identified as the measures with high agreement percentages from participants.
- Students group were more satisfied with the results than staff group.
- Three predictors for user satisfaction were identified through the evaluation study.

An initial reading list is prepared by researchers at the start of literature review for getting an overview of the research performed in a particular area. Prior studies have taken the approach of merely recommending seminal or popular papers to aid researchers in such a task. In this paper, we present an alternative technique called the AKR (Author-specified Keywords based Retrieval) technique for providing popular, recent, survey and a diverse set of papers as a part of the initial reading list. The AKR technique is based on a novel coverage value that has its calculation centered on author-specified keywords. We performed an offline evaluation experiment with four variants of the AKR technique along with three state-of-the-art approaches involving collaborative filtering and graph ranking algorithms. Findings show that the Hyperlink-Induced Topic Search (HITS) enhanced variant of the AKR technique performs better than other techniques, satisfying most requirements for a reading list. A user evaluation study was conducted with 132 researchers to gauge user interest on the proposed technique using 14 evaluation measures. Results show that (i) students group are more satisfied with the recommended papers than staff group, (ii) popularity measure is strongly correlated with the output quality measures and (iii) the measures familiarity, usefulness and 'agreeability on a good list' were found to be strong predictors for user satisfaction. The AKR technique provides scope for extension in future information retrieval (IR) and content-based recommender systems (RS) studies.