research
∙
06/27/2023
"Is a picture of a bird a bird": Policy recommendations for dealing with ambiguity in machine vision models
Many questions that we ask about the world do not have a single clear an...
share
research
∙
06/15/2023
Inverse Scaling: When Bigger Isn't Better
Work on scaling laws has found that large language models (LMs) show pre...
research
∙
05/23/2023
Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs
Large language models (LLMs) have achieved widespread success on a varie...
research
∙
05/22/2023
Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
The generative AI revolution in recent years has been spurred by an expa...
research
∙
10/19/2022
Two-Turn Debate Doesn't Help Humans Answer Hard Reading Comprehension Questions
The use of language-model-based question-answering systems to aid humans...
research
∙
08/26/2022
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
We present the results of the NLP Community Metasurvey. Run from May to ...
research
∙
04/11/2022
Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions
Current QA systems can generate reasonable-sounding yet false answers wi...
research
∙
12/16/2021
QuALITY: Question Answering with Long Input Texts, Yes!
To enable building and testing models on long-document comprehension, we...
research
∙
10/15/2021
BBQ: A Hand-Built Bias Benchmark for Question Answering
It is well documented that NLP models learn social biases present in the...
research
∙
09/14/2021
NOPE: A Corpus of Naturally-Occurring Presuppositions in English
Understanding language requires grasping not only the overtly stated con...
research
∙
04/15/2021
Does Putting a Linguist in the Loop Improve NLU Data Collection?
Many crowdsourced NLP datasets contain systematic gaps and biases that a...
research
∙
12/02/2019
BLiMP: A Benchmark of Linguistic Minimal Pairs for English
We introduce The Benchmark of Linguistic Minimal Pairs (shortened to BLi...
research
∙
09/05/2019