Improving scalability and reliability of MPI-agnostic transparent checkpointing for production workloads at NERSC

03/15/2021
by   Prashant Singh Chouhan, et al.
0

Checkpoint/restart (C/R) provides fault-tolerant computing capability, enables long running applications, and provides scheduling flexibility for computing centers to support diverse workloads with different priority. It is therefore vital to get transparent C/R capability working at NERSC. MANA, by Garg et. al., is a transparent checkpointing tool that has been selected due to its MPI-agnostic and network-agnostic approach. However, originally written as a proof-of-concept code, MANA was not ready to use with NERSC's diverse production workloads, which are dominated by MPI and hybrid MPI+OpenMP applications. In this talk, we present ongoing work at NERSC to enable MANA for NERSC's production workloads, including fixing bugs that were exposed by the top applications at NERSC, adding new features to address system changes, evaluating C/R overhead at scale, etc. The lessons learned from making MANA production-ready for HPC applications will be useful for C/R tool developers, supercomputing centers and HPC end-users alike.

READ FULL TEXT
research
12/10/2021

MANA-2.0: A Future-Proof Design for Transparent Checkpointing of MPI at Scale

MANA-2.0 is a scalable, future-proof design for transparent checkpointin...
research
04/20/2019

MANA for MPI: MPI-Agnostic Network-Agnostic Transparent Checkpointing

Transparently checkpointing MPI for fault tolerance and load balancing i...
research
03/04/2021

Checkpointing SPAdes for Metagenome Assembly: Transparency versus Performance in Production

The SPAdes assembler for metagenome assembly is a long-running applicati...
research
10/05/2022

Spot-on: A Checkpointing Framework for Fault-Tolerant Long-running Workloads on Cloud Spot Instances

Spot instances offer a cost-effective solution for applications running ...
research
12/12/2022

Collective Vector Clocks: Low-Overhead Transparent Checkpointing for MPI

MPI is the de facto standard for parallel computation on a cluster of co...
research
03/08/2021

Transparent Checkpointing for OpenGL Applications on GPUs

This work presents transparent checkpointing of OpenGL applications, ref...

Please sign up or login with your details

Forgot password? Click here to reset