The influence of the document ranking in expert search

The retrieval effectiveness of the underlying document search component of an expert search engine can have an important impact on the effectiveness of the generated expert search results. In this large-scale study, we perform novel experiments in the context of the document search and expert search tasks of the TREC Enterprise track, to measure the influence that the performance of the document ranking has on the ranking of candidate experts. In particular, our experiments show that while the expert search system performance is related to the relevance of the retrieved documents, surprisingly, it is not always the case that increasing document search effectiveness causes an increase in expert search performance. Moreover, we simulate document rankings designed with expert search performance in mind and, through a failure analysis, show why even a perfect document ranking may not result in a perfect ranking of candidate experts.

Research highlights
► Expert search approaches often build upon a ranking of documents for the query.
► Document ranking effectiveness and expert ranking effectiveness are correlated.
► Expert search approaches vary in their preferred document ranking properties.
► Simulated perfect document rankings do not lead to high expert search performance.
► The causes preventing perfect expert rankings were obtained using a failure analysis.