research
∙
07/19/2023
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation
Rising computational demands of modern natural language processing (NLP)...
share
research
∙
06/29/2023
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Many recent improvements in NLP stem from the development and use of lar...
research
∙
06/16/2023
Reproducibility in NLP: What Have We Learned from the Checklist?
Scientific progress in NLP rests on the reproducibility of researchers' ...
research
∙
06/09/2023
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Generative AI systems across modalities, ranging from text, image, audio...
research
∙
06/03/2023
Stubborn Lexical Bias in Data and Models
In NLP, recent work has seen increased focus on spurious correlations be...
research
∙
02/14/2023
AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models
Pretrained language models (PLMs) are trained on massive corpora, but of...
research
∙
12/19/2022
Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications
Scholarly text is often laden with jargon, or specialized language that ...
research
∙
06/13/2022
Modeling the Machine Learning Multiverse
Amid mounting concern about the reliability and credibility of machine l...
research
∙
06/10/2022
Measuring the Carbon Intensity of AI in Cloud Instances
By providing unprecedented access to computational resources, cloud comp...
research
∙
05/04/2022
Data Governance in the Age of Large-Scale Data-Driven Language Technology
The recent emergence and adoption of Machine Learning technology, and sp...
research
∙
03/11/2022
Staged Training for Transformer Language Models
The current standard approach to scaling transformer language models tra...
research
∙
12/16/2021
Efficient Hierarchical Domain Adaptation for Pretrained Language Models
Generative language models are trained on diverse, general domain corpor...
research
∙
10/01/2021
Expected Validation Performance and Estimation of a Random Variable's Maximum
Research in NLP is often supported by experimental results, and improved...
research
∙
04/18/2021
Documenting the English Colossal Clean Crawled Corpus
As language models are trained on ever more text, researchers are turnin...
research
∙
04/17/2021
Competency Problems: On Finding and Removing Artifacts in Language Data
Much recent work in NLP has documented dataset artifacts, bias, and spur...
research
∙
04/16/2020
The Right Tool for the Job: Matching Model and Instance Complexities
As NLP models become larger, executing a trained model requires signific...
research
∙
02/15/2020
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
Fine-tuning pretrained contextual word embedding models to supervised do...
research
∙
09/06/2019
RNN Architecture Learning with Sparse Regularization
Neural models for NLP typically use large numbers of parameters to reach...
research
∙
09/06/2019
Show Your Work: Improved Reporting of Experimental Results
Research in natural language processing proceeds, in part, by demonstrat...
research
∙
07/22/2019
Green AI
The computations required for deep learning research have been doubling ...
research
∙
06/06/2017
Random Search for Hyperparameters using Determinantal Point Processes
We propose the use of k-determinantal point processes in hyperparameter ...
research
∙
06/09/2016
Key-Value Memory Networks for Directly Reading Documents
Directly reading documents and being able to answer questions from them ...
research
∙
11/21/2015