Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks

by   Mouhamed Naby Ndiaye, et al.

Unmanned aerial vehicles (UAVs) are seen as a promising technology to perform a wide range of tasks in wireless communication networks. In this work, we consider the deployment of a group of UAVs to collect the data generated by IoT devices. Specifically, we focus on the case where the collected data is time-sensitive, and it is critical to maintain its timeliness. Our objective is to optimally design the UAVs' trajectories and the subsets of visited IoT devices such as the global Age-of-Updates (AoU) is minimized. To this end, we formulate the studied problem as a mixed-integer nonlinear programming (MINLP) under time and quality of service constraints. To efficiently solve the resulting optimization problem, we investigate the cooperative Multi-Agent Reinforcement Learning (MARL) framework and propose an RL approach based on the popular on-policy Reinforcement Learning (RL) algorithm: Policy Proximal Optimization (PPO). Our approach leverages the centralized training decentralized execution (CTDE) framework where the UAVs learn their optimal policies while training a centralized value function. Our simulation results show that the proposed MAPPO approach reduces the global AoU by at least a factor of 1/2 compared to conventional off-policy reinforcement learning approaches.


Multi-Agent Graph Reinforcement Learning based On-Demand Wireless Energy Transfer in Multi-UAV-aided IoT Network

This paper proposes a new on-demand wireless energy transfer (WET) schem...

Learning based Age of Information Minimization in UAV-relayed IoT Networks

Unmanned Aerial Vehicles (UAVs) are used as aerial base-stations to rela...

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV based Random Access IoT Networks with NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) techni...

Multi-UAV Path Planning for Wireless Data Harvesting with Deep Reinforcement Learning

Harvesting data from distributed Internet of Things (IoT) devices with m...

A Hybrid Framework of Reinforcement Learning and Convex Optimization for UAV-Based Autonomous Metaverse Data Collection

Unmanned aerial vehicles (UAVs) are promising for providing communicatio...

Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization

The nuclear fuel loading pattern optimization problem has been studied s...

Please sign up or login with your details

Forgot password? Click here to reset