research
∙
06/21/2023
OBELISC: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Large multimodal models trained on natural documents, which interleave i...
share
research
∙
03/07/2023
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
As language models grow ever larger, the need for large-scale high-quali...
research
∙
10/27/2022
What Language Model to Train if You Have One Million GPU Hours?
The crystallization of modeling methods around the Transformer architect...
research
∙
07/07/2022
Training Transformers Together
The infrastructure necessary for training state-of-the-art models is bec...
research
∙
06/18/2021