Management and Orchestration of Virtual Network Functions via Deep Reinforcement Learning

by   Joan S. Pujol Roig, et al.

Management and orchestration (MANO) of resources by virtual network functions (VNFs) represents one of the key challenges towards a fully virtualized network architecture as envisaged by 5G standards. Current threshold-based policies inefficiently over-provision network resources and under-utilize available hardware, incurring high cost for network operators, and consequently, the users. In this work, we present a MANO algorithm for VNFs allowing a central unit (CU) to learn to autonomously re-configure resources (processing power and storage), deploy new VNF instances, or offload them to the cloud, depending on the network conditions, available pool of resources, and the VNF requirements, with the goal of minimizing a cost function that takes into account the economical cost as well as latency and the quality-of-service (QoS) experienced by the users. First, we formulate the stochastic resource optimization problem as a parameterized action Markov decision process (PAMDP). Then, we propose a solution based on deep reinforcement learning (DRL). More precisely, we present a novel RL approach called, parameterized action twin (PAT) deterministic policy gradient, which leverages an actor-critic architecture to learn to provision resources to the VNFs in an online manner. Finally, we present numerical performance results, and map them to 5G key performance indicators (KPIs). To the best of our knowledge, this is the first work that considers DRL for MANO of VNFs' physical resources.


Learning based E2E Energy Efficient in Joint Radio and NFV Resource Allocation for 5G and Beyond Networks

In this paper, we propose a joint radio and core resource allocation fra...

Continuous Multi-objective Zero-touch Network Slicing via Twin Delayed DDPG and OpenAI Gym

Artificial intelligence (AI)-driven zero-touch network slicing (NS) is a...

Zero-touch Continuous Network Slicing Control via Scalable Actor-Critic Learning

Artificial intelligence (AI)-driven zero-touch network slicing is envisa...

Actor-Critic-Based Learning for Zero-touch Joint Resource and Energy Control in Network Slicing

To harness the full potential of beyond 5G (B5G) communication systems, ...

Using Distributed Reinforcement Learning for Resource Orchestration in a Network Slicing Scenario

The Network Slicing (NS) paradigm enables the partition of physical and ...

Joint QoS-Aware Scheduling and Precoding for Massive MIMO Systems via Deep Reinforcement Learning

The rapid development of mobile networks proliferates the demands of hig...

Visualizing Dynamics: from t-SNE to SEMI-MDPs

Deep Reinforcement Learning (DRL) is a trending field of research, showi...

Please sign up or login with your details

Forgot password? Click here to reset