Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning

by   Richard Liaw, et al.
berkeley college

Rather than learning new control policies for each new task, it is possible, when tasks share some structure, to compose a "meta-policy" from previously learned policies. This paper reports results from experiments using Deep Reinforcement Learning on a continuous-state, discrete-action autonomous driving simulator. We explore how Deep Neural Networks can represent meta-policies that switch among a set of previously learned policies, specifically in settings where the dynamics of a new scenario are composed of a mixture of previously learned dynamics and where the state observation is possibly corrupted by sensing noise. We also report the results of experiments varying dynamics mixes, distractor policies, magnitudes/distributions of sensing noise, and obstacles. In a fully observed experiment, the meta-policy learning algorithm achieves 2.6x the reward achieved by the next best policy composition technique with 80 experiment, the meta-policy learning algorithm converges after 50 iterations while a direct application of RL fails to converge even after 200 iterations.


Learning to Drive in a Day

We demonstrate the first application of deep reinforcement learning to a...

Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling

Reinforcement learning algorithms can acquire policies for complex tasks...

A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning

To achieve general intelligence, agents must learn how to interact with ...

Deep Reinforcement Learning for Autonomous Driving

Reinforcement learning has steadily improved and outperform human in lot...

Local Explanations for Reinforcement Learning

Many works in explainable AI have focused on explaining black-box classi...

Bottom-Up Meta-Policy Search

Despite of the recent progress in agents that learn through interaction,...

Composable Learning with Sparse Kernel Representations

We present a reinforcement learning algorithm for learning sparse non-pa...

Please sign up or login with your details

Forgot password? Click here to reset