Active Retrieval Augmented Generation

by   Zhengbao Jiang, et al.

Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrieving information from external knowledge resources is one promising solution. Most existing retrieval-augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. This is limiting, however, in more general scenarios involving generation of long texts, where continually gathering information throughout the generation process is essential. There have been some past efforts to retrieve information multiple times while generating outputs, which mostly retrieve documents at fixed intervals using the previous context as queries. In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and what to retrieve across the course of the generation. We propose Forward-Looking Active REtrieval augmented generation (FLARE), a generic retrieval-augmented generation method which iteratively uses a prediction of the upcoming sentence to anticipate future content, which is then utilized as a query to retrieve relevant documents to regenerate the sentence if it contains low-confidence tokens. We test FLARE along with baselines comprehensively over 4 long-form knowledge-intensive generation tasks/datasets. FLARE achieves superior or competitive performance on all tasks, demonstrating the effectiveness of our method. Code and datasets are available at


page 17

page 18

page 19

page 20

page 21

page 22

page 23

page 24


Recitation-Augmented Language Models

We propose a new paradigm to help Large Language Models (LLMs) generate ...

Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy

Large language models are powerful text processors and reasoners, but ar...

RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit

Although Large Language Models (LLMs) have demonstrated extraordinary ca...

Long-range Language Modeling with Self-retrieval

Retrieval-augmented language models (LMs) have received much attention r...

Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP

Retrieval-augmented in-context learning has emerged as a powerful approa...

Query Rewriting for Retrieval-Augmented Large Language Models

Large Language Models (LLMs) play a powerful Reader of the Retrieve-then...

Autoregressive Search Engines: Generating Substrings as Document Identifiers

Knowledge-intensive language tasks require NLP systems to both provide t...

Please sign up or login with your details

Forgot password? Click here to reset