Generative Adversarial Networks for Annotated Data Augmentation in Data Sparse NLU

by   Olga Golovneva, et al.

Data sparsity is one of the key challenges associated with model development in Natural Language Understanding (NLU) for conversational agents. The challenge is made more complex by the demand for high quality annotated utterances commonly required for supervised learning, usually resulting in weeks of manual labor and high cost. In this paper, we present our results on boosting NLU model performance through training data augmentation using a sequential generative adversarial network (GAN). We explore data generation in the context of two tasks, the bootstrapping of a new language and the handling of low resource features. For both tasks we explore three sequential GAN architectures, one with a token-level reward function, another with our own implementation of a token-level Monte Carlo rollout reward, and a third with sentence-level reward. We evaluate the performance of these feedback models across several sampling methodologies and compare our results to upsampling the original data to the same scale. We further improve the GAN model performance through the transfer learning of the pretrained embeddings. Our experiments reveal synthetic data generated using the sequential generative adversarial network provides significant performance boosts across multiple metrics and can be a major benefit to the NLU tasks.


page 1

page 2

page 3

page 4


Data augmentation for low resource sentiment analysis using generative adversarial networks

Sentiment analysis is a task that may suffer from a lack of data in cert...

Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review

In agricultural image analysis, optimal model performance is keenly purs...

Attention-stacked Generative Adversarial Network (AS-GAN)-empowered Sensor Data Augmentation for Online Monitoring of Manufacturing System

Machine learning (ML) has been extensively adopted for the online sensin...

A Bayesian Data Augmentation Approach for Learning Deep Models

Data augmentation is an essential part of the training process applied t...

Adversarial Conversational Shaping for Intelligent Agents

The recent emergence of deep learning methods has enabled the research c...

Mitigating Greenhouse Gas Emissions Through Generative Adversarial Networks Based Wildfire Prediction

Over the past decade, the number of wildfire has increased significantly...

Please sign up or login with your details

Forgot password? Click here to reset