LISA: Learning Interpretable Skill Abstractions from Language

02/28/2022
by   Divyansh Garg, et al.
1

Learning policies that effectually utilize language instructions in complex, multi-task environments is an important problem in imitation learning. While it is possible to condition on the entire language instruction directly, such an approach could suffer from generalization issues. To encode complex instructions into skills that can generalize to unseen instructions, we propose Learning Interpretable Skill Abstractions (LISA), a hierarchical imitation learning framework that can learn diverse, interpretable skills from language-conditioned demonstrations. LISA uses vector quantization to learn discrete skill codes that are highly correlated with language instructions and the behavior of the learned policy. In navigation and robotic manipulation environments, LISA is able to outperform a strong non-hierarchical baseline in the low data regime and compose learned skills to solve tasks containing unseen long-range instructions. Our method demonstrates a more natural way to condition on language in sequential decision-making problems and achieve interpretable and controllable behavior with the learned skills.

READ FULL TEXT

page 7

page 12

page 13

page 14

page 15

page 16

page 20

page 22

research
04/15/2022

Divide Conquer Imitation Learning

When cast into the Deep Reinforcement Learning framework, many robotics ...
research
05/06/2022

SKILL-IL: Disentangling Skill and Knowledge in Multitask Imitation Learning

In this work, we introduce a new perspective for learning transferable c...
research
11/19/2018

Guiding Policies with Language via Meta-Learning

Behavioral skills or policies for autonomous agents are conventionally l...
research
03/21/2023

Text2Motion: From Natural Language Instructions to Feasible Plans

We propose Text2Motion, a language-based planning framework enabling rob...
research
07/20/2020

Complex Skill Acquisition through Simple Skill Adversarial Imitation Learning

Humans are able to think of complex tasks as combinations of simpler sub...
research
07/19/2021

Hierarchical Few-Shot Imitation with Skill Transition Models

A desirable property of autonomous agents is the ability to both solve l...
research
08/24/2023

BridgeData V2: A Dataset for Robot Learning at Scale

We introduce BridgeData V2, a large and diverse dataset of robotic manip...

Please sign up or login with your details

Forgot password? Click here to reset