ASC me to Do Anything: Multi-task Training for Embodied AI

by   Jiasen Lu, et al.

Embodied AI has seen steady progress across a diverse set of independent tasks. While these varied tasks have different end goals, the basic skills required to complete them successfully overlap significantly. In this paper, our goal is to leverage these shared skills to learn to perform multiple tasks jointly. We propose Atomic Skill Completion (ASC), an approach for multi-task training for Embodied AI, where a set of atomic skills shared across multiple tasks are composed together to perform the tasks. The key to the success of this approach is a pre-training scheme that decouples learning of the skills from the high-level tasks making joint training effective. We use ASC to train agents within the AI2-THOR environment to perform four interactive tasks jointly and find it to be remarkably effective. In a multi-task setting, ASC improves success rates by a factor of 2x on Seen scenes and 4x on Unseen scenes compared to no pre-training. Importantly, ASC enables us to train a multi-task agent that has a 52 task agents. Finally, our hierarchical agents are more interpretable than traditional black-box architectures.


page 15

page 16

page 17

page 18

page 19

page 20

page 21

page 22


Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning

Learning policies for complex tasks that require multiple different skil...

PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale

The predictive information, the mutual information between the past and ...

12-in-1: Multi-Task Vision and Language Representation Learning

Much of vision-and-language research focuses on a small but diverse set ...

Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets

Robot learning holds the promise of learning policies that generalize br...

When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP

Multi-task learning (MTL) aims at achieving a better model by leveraging...

Towards Exploiting Geometry and Time for Fast Off-Distribution Adaptation in Multi-Task Robot Learning

We explore possible methods for multi-task transfer learning which seek ...

Designing an AI Health Coach and Studying its Utility in Promoting Regular Aerobic Exercise

Our research aims to develop interactive, social agents that can coach p...

Please sign up or login with your details

Forgot password? Click here to reset