Large Language Models for Information Retrieval: A Survey

by   Yutao Zhu, et al.

As a primary means of information acquisition, information retrieval (IR) systems, such as search engines, have integrated themselves into our daily lives. These systems also serve as components of dialogue, question-answering, and recommender systems. The trajectory of IR has evolved dynamically from its origins in term-based methods to its integration with advanced neural models. While the neural models excel at capturing complex contextual signals and semantic nuances, thereby reshaping the IR landscape, they still face challenges such as data scarcity, interpretability, and the generation of contextually plausible yet potentially inaccurate responses. This evolution requires a combination of both traditional methods (such as term-based sparse retrieval methods with rapid response) and modern neural architectures (such as language models with powerful language understanding capacity). Meanwhile, the emergence of large language models (LLMs), typified by ChatGPT and GPT-4, has revolutionized natural language processing due to their remarkable language understanding, generation, generalization, and reasoning abilities. Consequently, recent research has sought to leverage LLMs to improve IR systems. Given the rapid evolution of this research trajectory, it is necessary to consolidate existing methodologies and provide nuanced insights through a comprehensive overview. In this survey, we delve into the confluence of LLMs and IR systems, including crucial aspects such as query rewriters, retrievers, rerankers, and readers. Additionally, we explore promising directions within this expanding field.


Recommender Systems in the Era of Large Language Models (LLMs)

With the prosperity of e-commerce and web applications, Recommender Syst...

Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community

The research field of Information Retrieval (IR) has evolved significant...

Beyond CO2 Emissions: The Overlooked Impact of Water Consumption of Information Retrieval Models

As in other fields of artificial intelligence, the information retrieval...

Knowledge Refinement via Interaction Between Search Engines and Large Language Models

Information retrieval (IR) plays a crucial role in locating relevant res...

A Survey of Quantum Theory Inspired Approaches to Information Retrieval

Since 2004, researchers have been using the mathematical framework of Qu...

Neural Retriever and Go Beyond: A Thesis Proposal

Information Retriever (IR) aims to find the relevant documents (e.g. sni...

Tutorial: Modern Theoretical Tools for Understanding and Designing Next-generation Information Retrieval System

In the relatively short history of machine learning, the subtle balance ...

Please sign up or login with your details

Forgot password? Click here to reset