ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills

by   Jiayuan Gu, et al.

Generalizable manipulation skills, which can be composed to tackle long-horizon and complex daily chores, are one of the cornerstones of Embodied AI. However, existing benchmarks, mostly composed of a suite of simulatable environments, are insufficient to push cutting-edge research works because they lack object-level topological and geometric variations, are not based on fully dynamic simulation, or are short of native support for multiple types of manipulation tasks. To this end, we present ManiSkill2, the next generation of the SAPIEN ManiSkill benchmark, to address critical pain points often encountered by researchers when using benchmarks for generalizable manipulation skills. ManiSkill2 includes 20 manipulation task families with 2000+ object models and 4M+ demonstration frames, which cover stationary/mobile-base, single/dual-arm, and rigid/soft-body manipulation tasks with 2D/3D-input data simulated by fully dynamic engines. It defines a unified interface and evaluation protocol to support a wide range of algorithms (e.g., classic sense-plan-act, RL, IL), visual observations (point cloud, RGBD), and controllers (e.g., action type and parameterization). Moreover, it empowers fast visual input learning algorithms so that a CNN-based policy can collect samples at about 2000 FPS with 1 GPU and 16 processes on a regular workstation. It implements a render server infrastructure to allow sharing rendering resources across all environments, thereby significantly reducing memory usage. We open-source all codes of our benchmark (simulator, environments, and baselines) and host an online challenge open to interdisciplinary researchers.


page 1

page 10

page 20

page 24

page 25

page 27


M-EMBER: Tackling Long-Horizon Mobile Manipulation via Factorized Domain Transfer

In this paper, we propose a method to create visuomotor mobile manipulat...

Multi-skill Mobile Manipulation for Object Rearrangement

We study a modular approach to tackle long-horizon mobile manipulation t...

N^2M^2: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments

Despite its importance in both industrial and service robotics, mobile m...

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

We introduce Habitat 2.0 (H2.0), a simulation platform for training virt...

Let's Handle It: Generalizable Manipulation of Articulated Objects

In this project we present a framework for building generalizable manipu...

Benchmarking Robot Manipulation with the Rubik's Cube

Benchmarks for robot manipulation are crucial to measuring progress in t...

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Despite recent successes of reinforcement learning (RL), it remains a ch...

Please sign up or login with your details

Forgot password? Click here to reset