research
∙
08/31/2023
The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 Languages
Gender biases in language generation systems are challenging to mitigate...
share
research
∙
08/11/2023
DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity
The unprecedented photorealistic results achieved by recent text-to-imag...
research
∙
07/18/2023
Llama 2: Open Foundation and Fine-Tuned Chat Models
In this work, we develop and release Llama 2, a collection of pretrained...
research
∙
07/11/2023
Weisfeiler and Lehman Go Measurement Modeling: Probing the Validity of the WL Test
The expressive power of graph neural networks is usually measured by com...
research
∙
01/27/2023
Call for Papers – The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus
We present the call for papers for the BabyLM Challenge: Sample-efficien...
research
∙
12/18/2022
Language model acceptability judgements are not always robust to context
Targeted syntactic evaluations of language models ask whether models sho...
research
∙
10/23/2022
The Curious Case of Absolute Position Embeddings
Transformer language models encode the notion of word order using positi...
research
∙
08/17/2022
Learning Transductions to Test Systematic Compositionality
Recombining known primitive concepts into larger novel combinations is a...
research
∙
07/20/2022
DataPerf: Benchmarks for Data-Centric AI Development
Machine learning (ML) research has generally focused on models, while th...
research
∙
05/25/2022
Perturbation Augmentation for Fairer NLP
Unwanted and often harmful social biases are becoming ever more salient ...
research
∙
05/18/2022
"I'm sorry to hear that": finding bias in language models with a holistic descriptor dataset
As language models grow in popularity, their biases across all possible ...
research
∙
04/07/2022
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality
We present a novel task and dataset for evaluating the ability of vision...
research
∙
04/05/2022
Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks
We introduce Dynatask: an open source system for setting up custom NLP t...
research
∙
01/20/2022
A Latent-Variable Model for Intrinsic Probing
The success of pre-trained contextualized representations has prompted r...
research
∙
11/07/2021
A Word on Machine Ethics: A Response to Jiang et al. (2021)
Ethics is one of the longest standing intellectual endeavors of humanity...
research
∙
10/16/2021
Analyzing Dynamic Adversarial Training Data in the Limit
To create models that are robust across a wide range of test inputs, tra...
research
∙
09/07/2021
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models
All AI models are susceptible to learning biases in data that they are t...
research
∙
04/16/2021
Investigating Failures of Automatic Translation in the Case of Unambiguous Gender
Transformer based models are the modern work horses for neural machine t...
research
∙
04/15/2021
Sometimes We Want Translationese
Rapid progress in Neural Machine Translation (NMT) systems over the last...
research
∙
04/14/2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
A possible explanation for the impressive performance of masked language...
research
∙
12/30/2020
Unnatural Language Inference
Natural Language Understanding has witnessed a watershed moment with the...
research
∙
12/24/2020
To what extent do human explanations of model behavior align with actual model behavior?
Given the increasingly prominent role NLP models (will) play in our live...
research
∙
10/24/2020
ANLIzing the Adversarial Natural Language Inference Dataset
We perform an in-depth error analysis of Adversarial NLI (ANLI), a recen...
research
∙
10/06/2020
Intrinsic Probing through Dimension Selection
Most modern NLP systems make use of pre-trained contextual representatio...
research
∙
10/05/2020
Pareto Probing: Trading Off Accuracy for Complexity
The question of how to probe contextual word representations in a way th...
research
∙
06/20/2020
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
A broad goal in natural language processing (NLP) is to develop a system...
research
∙
05/04/2020
A Tale of a Probe and a Parser
Measuring what linguistic information is encoded in neural models of lan...
research
∙
05/03/2020
On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs
We use large-scale corpora in six different gendered languages, along wi...
research
∙
05/01/2020
Predicting Declension Class from Form and Meaning
The noun lexica of many natural languages are divided into several decle...
research
∙
05/01/2020
Multi-Dimensional Gender Bias Classification
Machine learning models are trained to find patterns in data. NLP models...
research
∙
04/07/2020
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition
Natural language inference (NLI) is an increasingly important task for n...
research
∙
04/07/2020
Information-Theoretic Probing for Linguistic Structure
The success of neural networks on a diverse set of NLP tasks has led res...
research
∙
11/10/2019
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation
Models often easily learn biases present in the training data, and their...
research
∙
10/31/2019
Adversarial NLI: A New Benchmark for Natural Language Understanding
We introduce a new large-scale NLI benchmark dataset, collected via an i...
research
∙
10/29/2019
Quantifying the Semantic Core of Gender Systems
Many of the world's languages employ grammatical gender on the lexeme. F...
research
∙
02/26/2019
On the Idiosyncrasies of the Mandarin Chinese Classifier System
While idiosyncrasies of the Chinese classifier system have been a richly...
research
∙
11/27/2018
Verb Argument Structure Alternations in Word and Sentence Embeddings
Verbs occur in different syntactic environments, or frames. We investiga...
research
∙
09/13/2018
XNLI: Evaluating Cross-lingual Sentence Representations
State-of-the-art natural language processing systems rely on supervision...
research
∙
09/04/2017
Learning to parse from a semantic objective: It works. Is it syntax?
Recent work on reinforcement learning and other gradient estimators for ...
research
∙
07/25/2017
The RepEval 2017 Shared Task: Multi-Genre Natural Language Inference with Sentence Representations
This paper presents the results of the RepEval 2017 Shared Task, which e...
research
∙
04/18/2017