Adversarial Bootstrapping for Dialogue Model Training

by   Oluwatobi Olabiyi, et al.

Open domain neural dialogue models, despite their successes, are known to produce responses that lack relevance, diversity, and in many cases coherence. These shortcomings stem from the limited ability of common training objectives to directly express these properties as well as their interplay with training datasets and model architectures. Toward addressing these problems, this paper proposes bootstrapping a dialogue response generator with an adversarially trained discriminator as an effective solution. The proposed method involves training a neural generator in both auto-regressive and traditional teacher-forcing modes, with the maximum likelihood loss of the auto-regressive outputs weighted by the score from a metric-based discriminator model. The discriminator input is a mixture of ground truth labels, the teacher-forcing outputs of the generator, and distractors sampled from the dataset, thereby allowing for richer feedback on the autoregressive outputs of the generator. To improve the calibration of the discriminator output, we also bootstrap the discriminator with the matching of the intermediate features of the ground truth and the generator's autoregressive output. We explore different sampling and adversarial policy optimization strategies during training in order to understand how to encourage response diversity without sacrificing relevance. Our experiments shows that adversarial bootstrapping is effective at addressing exposure bias, leading to improvement in response relevance and coherence. The improvement is demonstrated with the state-of-the-art results on Movie and Ubuntu dialogue datasets with respect to human evaluations and BLUE, ROGUE, and DISTINCT scores.


page 1

page 2

page 3

page 4


Multi-turn Dialogue Response Generation with Autoregressive Transformer Models

Neural dialogue models, despite their successes, still suffer from lack ...

Multi-turn Dialogue Response Generation in an Adversarial Learning Framework

We propose an adversarial learning approach to the generation of multi-t...

Adaptive Bridge between Training and Inference for Dialogue

Although exposure bias has been widely studied in some NLP tasks, it fac...

Meta-Context Transformers for Domain-Specific Response Generation

Despite the tremendous success of neural dialogue models in recent years...

EnsembleGAN: Adversarial Learning for Retrieval-Generation Ensemble Model on Short-Text Conversation

Generating qualitative responses has always been a challenge for human-c...

Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity

We present three enhancements to existing encoder-decoder models for ope...

Discriminative Adversarial Search for Abstractive Summarization

We introduce a novel approach for sequence decoding, Discriminative Adve...

Please sign up or login with your details

Forgot password? Click here to reset