research
∙
08/29/2023
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
How do language models learn to make predictions during pre-training? To...
share
research
∙
05/24/2023
Emergent inabilities? Inverse scaling over the course of pretraining
Does inverse scaling only occur as a function of model parameter size, o...
research
∙
03/20/2023
Language Model Behavior: A Comprehensive Survey
Transformer language models have received widespread public attention, y...
research
∙
01/20/2023
Can Peanuts Fall in Love with Distributional Semantics?
The context in which a sentence appears can drastically alter our expect...
research
∙
12/16/2022
'Rarely' a problem? Language models exhibit inverse scaling in their predictions following 'few'-type quantifiers
Language Models appear to perform poorly on quantification. We ask how b...
research
∙
11/09/2022
Collateral facilitation in humans and language models
Are the predictions of humans and language models affected by similar th...
research
∙
08/30/2022
Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?
Some languages allow arguments to be omitted in certain contexts. Yet hu...
research
∙
05/22/2022
The Geometry of Multilingual Language Model Representations
We assess how multilingual language models maintain a shared multilingua...
research
∙
10/05/2021
Word Acquisition in Neural Language Models
We investigate how neural language models acquire individual words durin...
research
∙
09/02/2021
So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgements
More predictable words are easier to process - they are read faster and ...
research
∙
07/20/2021
Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?
Despite being designed for performance rather than cognitive plausibilit...
research
∙
10/09/2020
How well does surprisal explain N400 amplitude under different experimental conditions?
We investigate the extent to which word surprisal can be used to predict...
research
∙
07/06/2020