Continuous Control with Deep Reinforcement Learning for Autonomous Vessels

by   Nader Zare, et al.

Maritime autonomous transportation has played a crucial role in the globalization of the world economy. Deep Reinforcement Learning (DRL) has been applied to automatic path planning to simulate vessel collision avoidance situations in open seas. End-to-end approaches that learn complex mappings directly from the input have poor generalization to reach the targets in different environments. In this work, we present a new strategy called state-action rotation to improve agent's performance in unseen situations by rotating the obtained experience (state-action-state) and preserving them in the replay buffer. We designed our model based on Deep Deterministic Policy Gradient, local view maker, and planner. Our agent uses two deep Convolutional Neural Networks to estimate the policy and action-value functions. The proposed model was exhaustively trained and tested in maritime scenarios with real maps from cities such as Montreal and Halifax. Experimental results show that the state-action rotation on top of the CVN consistently improves the rate of arrival to a destination (RATD) by up 11.96 Navigator with Planner and Local View (VNPLV), as well as it achieves superior performance in unseen mappings by up 30.82 advantages in terms of robustness when tested in a new environment, supporting the idea that generalization can be achieved by using state-action rotation.


page 3

page 5

page 6

page 8


Autonomous UAV Navigation: A DDPG-based Deep Reinforcement Learning Approach

In this paper, we propose an autonomous UAV path planning framework usin...

Graph augmented Deep Reinforcement Learning in the GameRLand3D environment

We address planning and navigation in challenging 3D video games featuri...

Learning to Navigate in Indoor Environments: from Memorizing to Reasoning

Autonomous navigation is an essential capability of smart mobility for m...

When to Replan? An Adaptive Replanning Strategy for Autonomous Navigation using Deep Reinforcement Learning

The hierarchy of global and local planners is one of the most commonly u...

On Reward Shaping for Mobile Robot Navigation: A Reinforcement Learning and SLAM Based Approach

We present a map-less path planning algorithm based on Deep Reinforcemen...

Rotation, Translation, and Cropping for Zero-Shot Generalization

Deep Reinforcement Learning (DRL) has shown impressive performance on do...

Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving

Automated driving in urban settings is challenging chiefly due to the in...

Please sign up or login with your details

Forgot password? Click here to reset