A Comparative Analysis of Deep Reinforcement Learning-based xApps in O-RAN

by   Maria Tsampazi, et al.

The highly heterogeneous ecosystem of Next Generation (NextG) wireless communication systems calls for novel networking paradigms where functionalities and operations can be dynamically and optimally reconfigured in real time to adapt to changing traffic conditions and satisfy stringent and diverse Quality of Service (QoS) demands. Open Radio Access Network (RAN) technologies, and specifically those being standardized by the O-RAN Alliance, make it possible to integrate network intelligence into the once monolithic RAN via intelligent applications, namely, xApps and rApps. These applications enable flexible control of the network resources and functionalities, network management, and orchestration through data-driven control loops. Despite recent work demonstrating the effectiveness of Deep Reinforcement Learning (DRL) in controlling O-RAN systems, how to design these solutions in a way that does not create conflicts and unfair resource allocation policies is still an open challenge. In this paper, we perform a comparative analysis where we dissect the impact of different DRL-based xApp designs on network performance. Specifically, we benchmark 12 different xApps that embed DRL agents trained using different reward functions, with different action spaces and with the ability to hierarchically control different network parameters. We prototype and evaluate these xApps on Colosseum, the world's largest O-RAN-compliant wireless network emulator with hardware-in-the-loop. We share the lessons learned and discuss our experimental results, which demonstrate how certain design choices deliver the highest performance while others might result in a competitive behavior between different classes of traffic with similar objectives.


Deep Reinforcement Learning for Wireless Scheduling with Multiclass Services

In this paper, we investigate the problem of scheduling and resource all...

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

Artificial intelligence (AI) and Machine Learning (ML) are considered as...

Deep reinforcement learning for RAN optimization and control

Due to the high variability of the traffic in the radio access network (...

Evolutionary Deep Reinforcement Learning for Dynamic Slice Management in O-RAN

The next-generation wireless networks are required to satisfy a variety ...

MR-iNet Gym: Framework for Edge Deployment of Deep Reinforcement Learning on Embedded Software Defined Radio

Dynamic resource allocation plays a critical role in the next generation...

TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources

Recent advances in deep reinforcement learning (DRL) have largely promot...

Safe and Accelerated Deep Reinforcement Learning-based O-RAN Slicing: A Hybrid Transfer Learning Approach

The open radio access network (O-RAN) architecture supports intelligent ...

Please sign up or login with your details

Forgot password? Click here to reset