An overview of Web search evaluation methods

Web search evaluation is the process of measuring the effectiveness of a Web search system. Such an evaluation helps in identifying the most effective one and helps the users to find the required information with less effort. Web search systems have been evaluated in many different ways in the last 15 years. In this paper, we review some of the efforts made for the evaluation of Web search systems. We discuss these evaluation studies by classifying them into eight different categories. As the size and content of Web is changing rapidly, and hence, the Web search techniques, we mention the necessity of an automatic evaluation methodology. But, at the same time, we emphasize that the significance of user based evaluation can not be neglected. Finally, we conclude that an automatic evaluation method that models users’ feedback based evaluation is required for the effective and realistic evaluation of Web search systems.