research
∙
03/01/2023
Diffusion Probabilistic Fields
Diffusion probabilistic models have quickly become a major approach for ...
share
research
∙
07/27/2022
GAUDI: A Neural Architect for Immersive 3D Scene Generation
We introduce GAUDI, a generative model capable of capturing the distribu...
research
∙
07/21/2022
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
There have been a lot of interest in the scaling properties of Transform...
research
∙
10/05/2021
Exploring the Limits of Large Scale Pre-training
Recent developments in large-scale machine learning suggest that by scal...
research
∙
09/22/2021
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
There remain many open questions pertaining to the scaling behaviour of ...
research
∙
06/10/2021
Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absent
We focus on the problem of domain adaptation when the goal is shifting t...
research
∙
11/08/2020
Long Range Arena: A Benchmark for Efficient Transformers
Transformers do not scale very well to long sequence lengths largely bec...
research
∙
05/31/2020
Transferring Inductive Biases through Knowledge Distillation
Having the right inductive biases can be crucial in many tasks or scenar...
research
∙
05/02/2020
Quantifying Attention Flow in Transformers
In the Transformer model, "self-attention" combines information from att...
research
∙
12/15/2019
A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings
The lack of annotated data in many languages is a well-known challenge w...
research
∙
06/04/2019
Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains
In this paper, we define and apply representational stability analysis (...
research
∙
04/04/2019
Robust Evaluation of Language-Brain Encoding Experiments
Language-brain encoding experiments evaluate the ability of language mod...
research
∙
01/15/2019
Incremental Reading for Question Answering
Any system which performs goal-directed continual learning must not only...
research
∙
11/25/2017