A Simple Approach to Continual Learning by Transferring Skill Parameters

by   K. R. Zentner, et al.

In order to be effective general purpose machines in real world environments, robots not only will need to adapt their existing manipulation skills to new circumstances, they will need to acquire entirely new skills on-the-fly. A great promise of continual learning is to endow robots with this ability, by using their accumulated knowledge and experience from prior skills. We take a fresh look at this problem, by considering a setting in which the robot is limited to storing that knowledge and experience only in the form of learned skill policies. We show that storing skill policies, careful pre-training, and appropriately choosing when to transfer those skill policies is sufficient to build a continual learner in the context of robotic manipulation. We analyze which conditions are needed to transfer skills in the challenging Meta-World simulation benchmark. Using this analysis, we introduce a pair-wise metric relating skills that allows us to predict the effectiveness of skill transfer between tasks, and use it to reduce the problem of continual learning to curriculum selection. Given an appropriate curriculum, we show how to continually acquire robotic manipulation skills without forgetting, and using far fewer samples than needed to train them from scratch.


page 5

page 7


A Scalable Approach to Multi-Context Continual Learning via Lifelong Skill Encoding

Continual or lifelong learning (CL) is one of the most challenging probl...

Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation

Lifelong-learning robots need to be able to acquire new skills and plan ...

Continual Learning of Control Primitives: Skill Discovery via Reset-Games

Reinforcement learning has the potential to automate the acquisition of ...

Voyager: An Open-Ended Embodied Agent with Large Language Models

We introduce Voyager, the first LLM-powered embodied lifelong learning a...

Continual Learning from Demonstration of Robotic Skills

Methods for teaching motion skills to robots focus on training for a sin...

Building a Subspace of Policies for Scalable Continual Learning

The ability to continuously acquire new knowledge and skills is crucial ...

Active Task Randomization: Learning Visuomotor Skills for Sequential Manipulation by Proposing Feasible and Novel Tasks

Solving real-world sequential manipulation tasks requires robots to have...

Please sign up or login with your details

Forgot password? Click here to reset