Parameterized Neural Network Language Models for Information Retrieval

by   Benjamin Piwowarski, et al.

Information Retrieval (IR) models need to deal with two difficult issues, vocabulary mismatch and term dependencies. Vocabulary mismatch corresponds to the difficulty of retrieving relevant documents that do not contain exact query terms but semantically related terms. Term dependencies refers to the need of considering the relationship between the words of the query when estimating the relevance of a document. A multitude of solutions has been proposed to solve each of these two problems, but no principled model solve both. In parallel, in the last few years, language models based on neural networks have been used to cope with complex natural language processing tasks like emotion and paraphrase detection. Although they present good abilities to cope with both term dependencies and vocabulary mismatch problems, thanks to the distributed representation of words they are based upon, such models could not be used readily in IR, where the estimation of one language model per document (or query) is required. This is both computationally unfeasible and prone to over-fitting. Based on a recent work that proposed to learn a generic language model that can be modified through a set of document-specific parameters, we explore use of new neural network models that are adapted to ad-hoc IR tasks. Within the language model IR framework, we propose and study the use of a generic language model as well as a document-specific language model. Both can be used as a smoothing component, but the latter is more adapted to the document at hand and has the potential of being used as a full document language model. We experiment with such models and analyze their results on TREC-1 to 8 datasets.


page 1

page 2

page 3

page 4


Unsupervised Document Expansion for Information Retrieval with Stochastic Text Generation

One of the challenges in information retrieval (IR) is the vocabulary mi...

Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval

Neural networks with deep architectures have demonstrated significant pe...

Modelling Word Burstiness in Natural Language: A Generalised Polya Process for Document Language Models in Information Retrieval

We introduce a generalised multivariate Polya process for document langu...

Rhetorical relations for information retrieval

Typically, every part in most coherent text has some plausible reason fo...

PACRR: A Position-Aware Neural IR Model for Relevance Matching

In order to adopt deep learning for information retrieval, models are ne...

Learning Term Discrimination

Document indexing is a key component for efficient information retrieval...

ABNIRML: Analyzing the Behavior of Neural IR Models

Numerous studies have demonstrated the effectiveness of pretrained conte...

Please sign up or login with your details

Forgot password? Click here to reset