research
∙
04/04/2023
TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings
In response to innovations in machine learning (ML) models, production w...
share
research
∙
11/29/2022
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts
We present MegaBlocks, a system for efficient Mixture-of-Experts (MoE) t...
research
∙
11/07/2020
Exploring the limits of Concurrency in ML Training on Google TPUs
Recent results in language understanding using neural networks have requ...
research
∙
06/18/2020
Sparse GPU Kernels for Deep Learning
Scientific workloads have traditionally exploited high levels of sparsit...
research
∙
04/11/2020
Bit-Parallel Vector Composability for Neural Acceleration
Conventional neural accelerators rely on isolated self-sufficient functi...
research
∙
10/02/2019
MLPerf Training Benchmark
Machine learning is experiencing an explosion of software and hardware s...
research
∙
11/05/2018
Mesh-TensorFlow: Deep Learning for Supercomputers
Batch-splitting (data-parallelism) is the dominant distributed Deep Neur...
research
∙
04/16/2017
In-Datacenter Performance Analysis of a Tensor Processing Unit
Many architects believe that major improvements in cost-energy-performan...
research
∙
09/26/2016