MarlRank: Multi-agent Reinforced Learning to Rank

by   Shihao Zou, et al.

When estimating the relevancy between a query and a document, ranking models largely neglect the mutual information among documents. A common wisdom is that if two documents are similar in terms of the same query, they are more likely to have similar relevance score. To mitigate this problem, in this paper, we propose a multi-agent reinforced ranking model, named MarlRank. In particular, by considering each document as an agent, we formulate the ranking process as a multi-agent Markov Decision Process (MDP), where the mutual interactions among documents are incorporated in the ranking process. To compute the ranking list, each document predicts its relevance to a query considering not only its own query-document features but also its similar documents features and actions. By defining reward as a function of NDCG, we can optimize our model directly on the ranking performance measure. Our experimental results on two LETOR benchmark datasets show that our model has significant performance gains over the state-of-art baselines. We also find that the NDCG shows an overall increasing trend along with the step of interactions, which demonstrates that the mutual information among documents helps improve the ranking performance.


page 1

page 2

page 3

page 4


A Passage-Based Approach to Learning to Rank Documents

According to common relevance-judgments regimes, such as TREC's, a docum...

Listwise Learning to Rank with Deep Q-Networks

Learning to Rank is the problem involved with ranking a sequence of docu...

SetRank: Learning a Permutation-Invariant Ranking Model for Information Retrieval

In learning-to-rank for information retrieval, a ranking model is automa...

Deep Relevance Ranking Using Enhanced Document-Query Interactions

We explore several new models for document relevance ranking, building u...

Unsupervised Common Question Generation from Multiple Documents using Reinforced Contrastive Coordinator

Web search engines today return a ranked list of document links in respo...

Integrate Document Ranking Information into Confidence Measure Calculation for Spoken Term Detection

This paper proposes an algorithm to improve the calculation of confidenc...

Non-Linear Multiple Field Interactions Neural Document Ranking

Ranking tasks are usually based on the text of the main body of the page...

Please sign up or login with your details

Forgot password? Click here to reset