Multi-UAV Path Planning for Wireless Data Harvesting with Deep Reinforcement Learning

by   Harald Bayerlein, et al.

Harvesting data from distributed Internet of Things (IoT) devices with multiple autonomous unmanned aerial vehicles (UAVs) is a challenging problem requiring flexible path planning methods. We propose a multi-agent reinforcement learning (MARL) approach that, in contrast to previous work, can adapt to profound changes in the scenario parameters defining the data harvesting mission, such as the number of deployed UAVs, number and position of IoT devices, or the maximum flying time, without the need to perform expensive recomputations or relearn control policies. We formulate the path planning problem for a cooperative, non-communicating, and homogeneous team of UAVs tasked with maximizing collected data from distributed IoT sensor nodes subject to flying time and collision avoidance constraints. The path planning problem is translated into a decentralized partially observable Markov decision process (Dec-POMDP), which we solve by training a double deep Q-network (DDQN) to approximate the optimal UAV control policy. By exploiting global-local maps of the environment that are fed into convolutional layers of the agents, we show that our proposed network architecture enables the agents to cooperate effectively by carefully dividing the data collection task among themselves, adapt to large state spaces, and make movement decisions that balance data collection goals, flight-time efficiency, and navigation constraints.


page 1

page 3

page 6

page 10

page 11


UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach

Autonomous deployment of unmanned aerial vehicles (UAVs) supporting next...

UAV Path Planning using Global and Local Map Information with Deep Reinforcement Learning

Path planning methods for autonomous unmanned aerial vehicles (UAVs) are...

Cooperative Data Collection with Multiple UAVs for Information Freshness in the Internet of Things

Maintaining the freshness of information in the Internet of Things (IoT)...

Efficient UAV Trajectory-Planning using Economic Reinforcement Learning

Advances in unmanned aerial vehicle (UAV) design have opened up applicat...

Distributed Deep Reinforcement Learning for Intelligent Traffic Monitoring with a Team of Aerial Robots

This paper studies the traffic monitoring problem in a road network usin...

Model-aided Federated Reinforcement Learning for Multi-UAV Trajectory Planning in IoT Networks

Deploying teams of cooperative unmanned aerial vehicles (UAVs) to harves...

Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks

Unmanned aerial vehicles (UAVs) are seen as a promising technology to pe...

Please sign up or login with your details

Forgot password? Click here to reset