Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses

by   Liyan Tang, et al.

A human decision-maker benefits the most from an AI assistant that corrects for their biases. For problems such as generating interpretation of a radiology report given findings, a system predicting only highly likely outcomes may be less useful, where such outcomes are already obvious to the user. To alleviate biases in human decision-making, it is worth considering a broad differential diagnosis, going beyond the most likely options. We introduce a new task, "less likely brainstorming," that asks a model to generate outputs that humans think are relevant but less likely to happen. We explore the task in two settings: a brain MRI interpretation generation setting and an everyday commonsense reasoning setting. We found that a baseline approach of training with less likely hypotheses as targets generates outputs that humans evaluate as either likely or irrelevant nearly half of the time; standard MLE training is not effective. To tackle this problem, we propose a controlled text generation method that uses a novel contrastive learning strategy to encourage models to differentiate between generating likely and less likely outputs according to humans. We compare our method with several state-of-the-art controlled text generation models via automatic and human evaluations and show that our models' capability of generating less likely outputs is improved.


page 1

page 2

page 3

page 4


Controllable Text Generation with Focused Variation

This work introduces Focused-Variation Network (FVN), a novel model to c...

CommonGen: A Constrained Text Generation Dataset Towards Generative Commonsense Reasoning

Rational humans can generate sentences that cover a certain set of conce...

Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts

Generative commonsense reasoning (GCR) in natural language is to reason ...

From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?

Noun compound interpretation is the task of expressing a noun compound (...

Surfacing Biases in Large Language Models using Contrastive Input Decoding

Ensuring that large language models (LMs) are fair, robust and useful re...

Can Language Models perform Abductive Commonsense Reasoning?

Abductive Reasoning is a task of inferring the most plausible hypothesis...

On-the-Fly Attention Modularization for Neural Generation

Despite considerable advancements with deep neural language models (LMs)...

Please sign up or login with your details

Forgot password? Click here to reset