Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning

by   Kaleb Ben Naveed, et al.

Planning safe trajectories under uncertain and dynamic conditions makes the autonomous driving problem significantly complex. Current sampling-based methods such as Rapidly Exploring Random Trees (RRTs) are not ideal for this problem because of the high computational cost. Supervised learning methods such as Imitation Learning lack generalization and safety guarantees. To address these problems and in order to ensure a robust framework, we propose a Hierarchical Reinforcement Learning (HRL) structure combined with a Proportional-Integral-Derivative (PID) controller for trajectory planning. HRL helps divide the task of autonomous vehicle driving into sub-goals and supports the network to learn policies for both high-level options and low-level trajectory planner choices. The introduction of sub-goals decreases convergence time and enables the policies learned to be reused for other scenarios. In addition, the proposed planner is made robust by guaranteeing smooth trajectories and by handling the noisy perception system of the ego-car. The PID controller is used for tracking the waypoints, which ensures smooth trajectories and reduces jerk. The problem of incomplete observations is handled by using a Long-Short-Term-Memory (LSTM) layer in the network. Results from the high-fidelity CARLA simulator indicate that the proposed method reduces convergence time, generates smoother trajectories, and is able to handle dynamic surroundings and noisy observations.


A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning

Autonomous vehicles need to handle various traffic conditions and make s...

Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning

Reinforcement Learning (RL) has made promising progress in planning and ...

A Hierarchical Approach for Strategic Motion Planning in Autonomous Racing

We present an approach for safe trajectory planning, where a strategic t...

Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Well-established optimization-based methods can guarantee an optimal tra...

Spatially and Seamlessly Hierarchical Reinforcement Learning for State Space and Policy space in Autonomous Driving

Despite advances in hierarchical reinforcement learning, its application...

A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning

For safe and efficient planning and control in autonomous driving, we ne...

RoCUS: Robot Controller Understanding via Sampling

As robots are deployed in complex situations, engineers and end users mu...

Please sign up or login with your details

Forgot password? Click here to reset