research
∙
03/29/2022
Training Compute-Optimal Large Language Models
We investigate the optimal model size and number of tokens for training ...
share
research
∙
02/02/2022
Unified Scaling Laws for Routed Language Models
The performance of a language model has been shown to be effectively mod...
research
∙
12/08/2021
Improving language models by retrieving from trillions of tokens
We enhance auto-regressive language models by conditioning on document c...
research
∙
11/28/2017
