research
∙
03/25/2022
MKQ-BERT: Quantized BERT with 4-bits Weights and Activations
Recently, pre-trained Transformer based language models, such as BERT, h...
share
research
∙
08/04/2021
ErrorCompensatedX: error compensation for variance reduced algorithms
Communication cost is one major bottleneck for the scalability for distr...
research
∙
05/30/2021
On the geometry of generalization and memorization in deep neural networks
Understanding how large neural networks avoid memorizing training data i...
research
∙
04/15/2021
Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models
While vector-based language representations from pretrained language mod...
research
∙
04/13/2021
1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed
To train large models (like BERT and GPT-3) with hundreds or even thousa...
research
∙
02/04/2021
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
Scalable training of large models (like BERT and GPT-3) requires careful...
research
∙
08/26/2020
APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm
Adam is the important optimization algorithm to guarantee efficiency and...
research
∙
07/14/2020
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
As modern neural networks have grown to billions of parameters, meeting ...
research
∙
06/01/2020
Emergence of Separable Manifolds in Deep Language Representations
Deep neural networks (DNNs) have shown much empirical success in solving...
research
∙
03/03/2020
Untangling in Invariant Speech Recognition
Encouraged by the success of deep neural networks on a variety of visual...
research
∙
01/16/2020
Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural Networks
Training with larger number of parameters while keeping fast iterations ...
research
∙
11/19/2019
Mimic The Raw Domain: Accelerating Action Recognition in the Compressed Domain
Video understanding usually requires expensive computation that prohibit...
research
∙
11/06/2019
MLPerf Inference Benchmark
Machine-learning (ML) hardware and software system demand is burgeoning....
research
∙
10/11/2019
Central Server Free Federated Learning over Single-sided Trust Social Networks
Federated learning has become increasingly important for modern machine ...
research
∙
10/02/2019
Using Image Priors to Improve Scene Understanding
Semantic segmentation algorithms that can robustly segment objects acros...
research
∙
10/02/2019
MLPerf Training Benchmark
Machine learning is experiencing an explosion of software and hardware s...
research
∙
09/19/2019
Triplet-Aware Scene Graph Embeddings
Scene graphs have become an important form of structured knowledge for t...
research
∙
07/17/2019
DeepSqueeze: Decentralization Meets Error-Compensated Compression
Communication is a key bottleneck in distributed training. Recently, an ...
research
∙
07/17/2019
DeepSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
Communication is a key bottleneck in distributed training. Recently, an ...
research
∙
06/26/2019
Generalization to Novel Objects using Prior Relational Knowledge
To solve tasks in new environments involving objects unseen during train...
research
∙
06/13/2019
Amur Tiger Re-identification in the Wild
Monitoring the population and movements of endangered species is an impo...
research
∙
05/15/2019
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
A standard approach in large scale machine learning is distributed stoch...
research
∙
04/19/2019
Compact Scene Graphs for Layout Composition and Patch Retrieval
Structured representations such as scene graphs serve as an efficient an...
research
∙
03/28/2019
SpaceNet MVOI: a Multi-View Overhead Imagery Dataset
Detection and segmentation of objects in overheard imagery is a challeng...
research
∙
01/11/2019
Using Scene Graph Context to Improve Image Generation
Generating realistic images from scene graphs asks neural networks to be...
research
∙
10/17/2018
Distributed Learning over Unreliable Networks
Most of today's distributed machine learning systems assume reliable ne...
research
∙
03/19/2018
D^2: Decentralized Training over Decentralized Data
While training a machine learning model using multiple workers, each of ...
research
∙
03/17/2018
Decentralization Meets Quantization
Optimizing distributed learning systems is an art of balancing between c...
research
∙
06/07/2017