ACuTE: Automatic Curriculum Transfer from Simple to Complex Environments

by   Yash Shukla, et al.

Despite recent advances in Reinforcement Learning (RL), many problems, especially real-world tasks, remain prohibitively expensive to learn. To address this issue, several lines of research have explored how tasks, or data samples themselves, can be sequenced into a curriculum to learn a problem that may otherwise be too difficult to learn from scratch. However, generating and optimizing a curriculum in a realistic scenario still requires extensive interactions with the environment. To address this challenge, we formulate the curriculum transfer problem, in which the schema of a curriculum optimized in a simpler, easy-to-solve environment (e.g., a grid world) is transferred to a complex, realistic scenario (e.g., a physics-based robotics simulation or the real world). We present "ACuTE", Automatic Curriculum Transfer from Simple to Complex Environments, a novel framework to solve this problem, and evaluate our proposed method by comparing it to other baseline approaches (e.g., domain adaptation) designed to speed up learning. We observe that our approach produces improved jumpstart and time-to-threshold performance even when adding task elements that further increase the difficulty of the realistic scenario. Finally, we demonstrate that our approach is independent of the learning algorithm used for curriculum generation, and is Sim2Real transferable to a real world scenario using a physical robot.


Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey

Reinforcement learning (RL) is a popular paradigm for addressing sequent...

Effects of Reward Shaping on Curriculum Learning in Goal Conditioned Tasks

Real-time control for robotics is a popular research area in the reinfor...

Automaton-Guided Curriculum Generation for Reinforcement Learning Agents

Despite advances in Reinforcement Learning, many sequential decision mak...

ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills

Humans are highly adept at walking in environments with foot placement c...

Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments

We present NavACL, a method of automatic curriculum learning tailored to...

Robots Learn Increasingly Complex Tasks with Intrinsic Motivation and Automatic Curriculum Learning

Multi-task learning by robots poses the challenge of the domain knowledg...

Self-Paced Absolute Learning Progress as a Regularized Approach to Curriculum Learning

The usability of Reinforcement Learning is restricted by the large compu...

Please sign up or login with your details

Forgot password? Click here to reset