A subjective measure of web search quality

We measure the “satisfaction” a user gets when presented with the search results. We watch the actions of the user on the search results presented before him in response to his query, and infer the feedback of the user therefrom. The implicit ranking thus provided by the user is compared with the original ranking given by the search engine. The correlation coefficient thus obtained is averaged for a set of queries. We show our results pertaining to 7 public search engines and 15 ad hoc queries. Our emphasis is more to demonstrate the procedure of quality measurement than to carry out the actual performance measurement of these search engines.