research
∙
02/06/2023
Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models
In recent years, large-scale models have demonstrated state-of-the-art p...
share
research
∙
12/10/2022
Elixir: Train a Large Language Model on a Small GPU Cluster
In recent years, the number of parameters of one deep learning (DL) mode...
research
∙
09/06/2022
EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models
Large transformer models display promising performance on a wide range o...
research
∙
08/08/2022
A Frequency-aware Software Cache for Large Recommendation System Embeddings
Deep learning recommendation models (DLRMs) have been widely applied in ...
research
∙
08/12/2021
PatrickStar: Parallel Training of Pre-trained Models via a Chunk-based Memory Management
The pre-trained model (PTM) is revolutionizing Artificial intelligence (...
research
∙
08/08/2021
Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters
Efficient GPU resource scheduling is essential to maximize resource util...
research
∙
04/12/2021