research
∙
02/20/2023
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation
Skip connections and normalisation layers form two standard architectura...
share
research
∙
04/29/2022
Flamingo: a Visual Language Model for Few-Shot Learning
Building models that can be rapidly adapted to numerous tasks using only...
research
∙
11/23/2021
Towards Learning Universal Audio Representations
The ability to learn universal audio representations that can solve dive...
research
∙
07/30/2021
Perceiver IO: A General Architecture for Structured Inputs Outputs
The recently-proposed Perceiver model obtains good results on several do...
research
∙
05/27/2021
Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error
In computer vision, it is standard practice to draw a single sample from...
research
∙
04/02/2021
Skillful Precipitation Nowcasting using Deep Generative Models of Radar
Precipitation nowcasting, the high-resolution forecasting of precipitati...
research
∙
03/04/2021
Perceiver: General Perception with Iterative Attention
Biological systems understand the world by simultaneously processing hig...
research
∙
02/11/2021
High-Performance Large-Scale Image Recognition Without Normalization
Batch normalization is a key component of most image classification mode...
research
∙
01/21/2021
Characterizing signal propagation to close the performance gap in unnormalized ResNets
Batch Normalization is a key component in almost all state-of-the-art im...
research
∙
10/28/2020
Training Generative Adversarial Networks by Solving Ordinary Differential Equations
The instability of Generative Adversarial Network (GAN) training has fre...
research
∙
10/20/2020
BYOL works even without batch statistics
Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach ...
research
∙
04/06/2020
Evolving Normalization-Activation Layers
Normalization layers and activation functions are critical components in...
research
∙
09/28/2018
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Despite recent progress in generative image modeling, successfully gener...
research
∙
06/15/2017
FreezeOut: Accelerate Training by Progressively Freezing Layers
The early layers of a deep neural net have the fewest parameters, but ta...
research
∙
09/22/2016
Neural Photo Editing with Introspective Adversarial Networks
The increasingly photorealistic sample quality of generative image model...
research
∙
08/15/2016