Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet

by   Kaidi Xu, et al.

In heterogeneous networks (HetNets), the overlap of small cells and the macro cell causes severe cross-tier interference. Although there exist some approaches to address this problem, they usually require global channel state information, which is hard to obtain in practice, and get the sub-optimal power allocation policy with high computational complexity. To overcome these limitations, we propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet, where each access point makes power control decisions independently based on local information. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. By introducing regularization terms in the loss function, each agent tends to choose an experienced action with high reward when revisiting a state, and thus the policy updating speed slows down. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process. We then implement the proposed PQL in the considered HetNet and compare it with other distributed-training-and-execution (DTE) algorithms. Simulation results show that our proposed PQL can learn the desired power control policy from a dynamic environment where the locations of users change episodically and outperform existing DTE MADRL algorithms.


page 1

page 2

page 3

page 4


PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control

This paper develops an efficient multi-agent deep reinforcement learning...

Deep Reinforcement Learning for Multi-Agent Non-Cooperative Power Control in Heterogeneous Networks

We consider a typical heterogeneous network (HetNet), in which multiple ...

Reinforcement Learning for Self-Organization and Power Control of Two-Tier Heterogeneous Networks

Self-organizing networks (SONs) can help manage the severe interference ...

Learning Power Control from a Fixed Batch of Data

We address how to exploit power control data, gathered from a monitored ...

Active collaboration in relative observation for Multi-agent visual SLAM based on Deep Q Network

This paper proposes a unique active relative localization mechanism for ...

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

This paper proposes a data-driven distributed voltage control approach b...

Faded-Experience Trust Region Policy Optimization for Model-Free Power Allocation in Interference Channel

Policy gradient reinforcement learning techniques enable an agent to dir...

Please sign up or login with your details

Forgot password? Click here to reset