RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text

by   Wangchunshu Zhou, et al.

The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is built upon a large language model (LLM) such as ChatGPT and uses natural language to simulate the Long Short-Term Memory mechanism in an LSTM. At each timestep, RecurrentGPT generates a paragraph of text and updates its language-based long-short term memory stored on the hard drive and the prompt, respectively. This recurrence mechanism enables RecurrentGPT to generate texts of arbitrary length without forgetting. Since human users can easily observe and edit the natural language memories, RecurrentGPT is interpretable and enables interactive generation of long text. RecurrentGPT is an initial step towards next-generation computer-assisted writing systems beyond local editing suggestions. In addition to producing AI-generated content (AIGC), we also demonstrate the possibility of using RecurrentGPT as an interactive fiction that directly interacts with consumers. We call this usage of generative models by “AI As Contents” (AIAC), which we believe is the next form of conventional AIGC. We further demonstrate the possibility of using RecurrentGPT to create personalized interactive fiction that directly interacts with readers instead of interacting with writers. More broadly, RecurrentGPT demonstrates the utility of borrowing ideas from popular model designs in cognitive science and deep learning for prompting LLMs. Our code is available at and an online demo is available at


page 2

page 7

page 13


Real-time interactive sequence generation and control with Recurrent Neural Network ensembles

Recurrent Neural Networks (RNN), particularly Long Short Term Memory (LS...

Machine Translation from Natural Language to Code using Long-Short Term Memory

Making computer programming language more understandable and easy for th...

AmadeusGPT: a natural language interface for interactive animal behavioral analysis

The process of quantifying and analyzing animal behavior involves transl...

Semi-supervised Sequence Learning

We present two approaches that use unlabeled data to improve sequence le...

GenNI: Human-AI Collaboration for Data-Backed Text Generation

Table2Text systems generate textual output based on structured data util...

CRYPTEXT: Database and Interactive Toolkit of Human-Written Text Perturbations in the Wild

User-generated textual contents on the Internet are often noisy, erroneo...

Exploring the Long Short-Term Dependencies to Infer Shot Influence in Badminton Matches

Identifying significant shots in a rally is important for evaluating pla...

Please sign up or login with your details

Forgot password? Click here to reset