Applying reinforcement learning for web pages ranking algorithms

Ranking web pages for presenting the most relevant web pages to user's queries is one of the main issues in any search engine. In this paper, two new ranking algorithms are offered, using Reinforcement Learning (RL) concepts. RL is a powerful technique of modern artificial intelligence that tunes agent's parameters, interactively. In the first step, with formulation of ranking as an RL problem, a new connectivity-based ranking algorithm, called RL_Rank, is proposed. In RL_Rank, agent is considered as a surfer who travels between web pages by clicking randomly on a link in the current page. Each web page is considered as a state and value function of state is used to determine the score of that state (page). Reward is corresponded to number of out links from the current page. Rank scores in RL_Rank are computed in a recursive way. Convergence of these scores is proved. In the next step, we introduce a new hybrid approach using combination of BM25 as a content-based algorithm and RL_Rank. Both proposed algorithms are evaluated by well known benchmark datasets and analyzed according to concerning criteria. Experimental results show using RL concepts leads significant improvements in raking algorithms.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slideHighlights► We proposed a novel ranking connectivity-based algorithm for ranking web pages. ► We used the Reinforcement Learning (RL) concepts, so our algorithm is called RL_Rank. ► We also introduced a hybrid algorithm using RL_Rank and BM25. ► The convergence of RL_Rank was proved. ► RL_Rank outperforms PageRank on the considered data sets, especially in dense web graphs.