Neural Language Generation: Formulation, Methods, and Evaluation

by   Cristina Garbacea, et al.

Recent advances in neural network-based generative modeling have reignited the hopes in having computer systems capable of seamlessly conversing with humans and able to understand natural language. Neural architectures have been employed to generate text excerpts to various degrees of success, in a multitude of contexts and tasks that fulfil various user needs. Notably, high capacity deep learning models trained on large scale datasets demonstrate unparalleled abilities to learn patterns in the data even in the lack of explicit supervision signals, opening up a plethora of new possibilities regarding producing realistic and coherent texts. While the field of natural language generation is evolving rapidly, there are still many open challenges to address. In this survey we formally define and categorize the problem of natural language generation. We review particular application tasks that are instantiations of these general formulations, in which generating natural language is of practical importance. Next we include a comprehensive outline of methods and neural architectures employed for generating diverse texts. Nevertheless, there is no standard way to assess the quality of text produced by these generative models, which constitutes a serious bottleneck towards the progress of the field. To this end, we also review current approaches to evaluating natural language generation systems. We hope this survey will provide an informative overview of formulations, methods, and assessments of neural natural language generation.


page 1

page 2

page 3

page 4


Why is constrained neural language generation particularly challenging?

Recent advances in deep neural language models combined with the capacit...

Human Motion Generation: A Survey

Human motion generation aims to generate natural human pose sequences an...

An Overview on Controllable Text Generation via Variational Auto-Encoders

Recent advances in neural-based generative modeling have reignited the h...

Persuasive Natural Language Generation – A Literature Review

This literature review focuses on the use of Natural Language Generation...

Deep Learning for Text Attribute Transfer: A Survey

Driven by the increasingly larger deep learning models, neural language ...

Visualizing Natural Language Descriptions: A Survey

A natural language interface exploits the conceptual simplicity and natu...

Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models

Recent years have seen a growing number of publications that analyse Nat...

Please sign up or login with your details

Forgot password? Click here to reset