RLIRank: Learning to Rank with Reinforcement Learning for Dynamic Search

by   Jianghong Zhou, et al.

To support complex search tasks, where the initial information requirements are complex or may change during the search, a search engine must adapt the information delivery as the user's information requirements evolve. To support this dynamic ranking paradigm effectively, search result ranking must incorporate both the user feedback received, and the information displayed so far. To address this problem, we introduce a novel reinforcement learning-based approach, RLIrank. We first build an adapted reinforcement learning framework to integrate the key components of the dynamic search. Then, we implement a new Learning to Rank (LTR) model for each iteration of the dynamic search, using a recurrent Long Short Term Memory neural network (LSTM), which estimates the gain for each next result, learning from each previously ranked document. To incorporate the user's feedback, we develop a word-embedding variation of the classic Rocchio Algorithm, to help guide the ranking towards the high-value documents. Those innovations enable RLIrank to outperform the previously reported methods from the TREC Dynamic Domain Tracks 2017 and exceed all the methods in 2016 TREC Dynamic Domain after multiple search iterations, advancing the state of the art for dynamic search.


page 1

page 2

page 3

page 4


A Flexible Approach to Automated RNN Architecture Generation

The process of designing neural architectures requires expert knowledge ...

Maximizing Marginal Fairness for Dynamic Learning to Rank

Rankings, especially those in search and recommendation systems, often d...

De-Biased Modelling of Search Click Behavior with Reinforcement Learning

Users' clicks on Web search results are one of the key signals for evalu...

The Search for Stability: Learning Dynamics of Strategic Publishers with Initial Documents

We study a game-theoretic model of information retrieval, in which strat...

Interactive Video Corpus Moment Retrieval using Reinforcement Learning

Known-item video search is effective with human-in-the-loop to interacti...

Dynamic Search – Optimizing the Game of Information Seeking

This article presents the emerging topic of dynamic search (DS). To posi...

Analysis of Evolutionary Behavior in Self-Learning Media Search Engines

The diversity of intrinsic qualities of multimedia entities tends to imp...

Please sign up or login with your details

Forgot password? Click here to reset