research
∙
04/21/2023
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
It is widely acknowledged that large models have the potential to delive...
share
research
∙
07/24/2018
Supporting Very Large Models using Automatic Dataflow Graph Partitioning
There is a trend towards using very large deep neural networks (DNN) to ...
research
∙
05/10/2018
Unifying Data, Model and Hybrid Parallelism in Deep Learning via Tensor Tiling
Deep learning systems have become vital tools across many fields, but th...
research
∙
12/09/2015