The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AI

by   Chuang Gan, et al.

We introduce a visually-guided and physics-driven task-and-motion planning benchmark, which we call the ThreeDWorld Transport Challenge. In this challenge, an embodied agent equipped with two 9-DOF articulated arms is spawned randomly in a simulated physical home environment. The agent is required to find a small set of objects scattered around the house, pick them up, and transport them to a desired final location. We also position containers around the house that can be used as tools to assist with transporting objects efficiently. To complete the task, an embodied agent must plan a sequence of actions to change the state of a large number of objects in the face of realistic physical constraints. We build this benchmark challenge using the ThreeDWorld simulation: a virtual 3D environment where all objects respond to physics, and where can be controlled using fully physics-driven navigation and interaction API. We evaluate several existing agents on this benchmark. Experimental results suggest that: 1) a pure RL model struggles on this challenge; 2) hierarchical planning-based agents can transport some objects but still far from solving this task. We anticipate that this benchmark will empower researchers to develop more intelligent physics-driven robots for the physical world.


page 1

page 4

page 6

page 7

page 8


Learning to Ground Objects for Robot Task and Motion Planning

Task and motion planning (TAMP) algorithms have been developed to help r...

OPEn: An Open-ended Physics Environment for Learning Without a Task

Humans have mental models that allow them to plan, experiment, and reaso...

RFUniverse: A Physics-based Action-centric Interactive Environment for Everyday Household Tasks

Household environments are important testbeds for embodied AI research. ...

Physics-Based Task Generation Through Causal Sequence of Physical Interactions

Performing tasks in a physical environment is a crucial yet challenging ...

Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport

We address key challenges in long-horizon embodied exploration and navig...

PHYRE: A New Benchmark for Physical Reasoning

Understanding and reasoning about physics is an important ability of int...

Feature Guided Search for Creative Problem Solving Through Tool Construction

Robots in the real world should be able to adapt to unforeseen circumsta...

Please sign up or login with your details

Forgot password? Click here to reset