Accelerating Deep Reinforcement Learning With the Aid of a Partial Model: Power-Efficient Predictive Video Streaming

by   Dong Liu, et al.

Predictive power allocation is conceived for power-efficient video streaming over mobile networks using deep reinforcement learning. The goal is to minimize the accumulated energy consumption over a complete video streaming session for a mobile user under the quality of service constraint that avoids video playback interruptions. To handle the continuous state and action spaces, we resort to deep deterministic policy gradient (DDPG) algorithm for solving the formulated problem. In contrast to previous predictive resource policies that first predict future information with historical data and then optimize the policy based on the predicted information, the proposed policy operates in an on-line and end-to-end manner. By judiciously designing the action and state that only depend on slowly-varying average channel gains, the signaling overhead between the edge server and the base stations can be reduced, and the dynamics of the system can be learned effortlessly. To improve the robustness of streaming and accelerate learning, we further exploit the partially known dynamics of the system by integrating the concepts of safer layer, post-decision state, and virtual experience into the basic DDPG algorithm. Our simulation results show that the proposed policies converge to the optimal policy derived based on perfect prediction of the future large-scale channel gains and outperforms the first-predict-then-optimize policy in the presence of prediction errors. By harnessing the partially known model of the system dynamics, the convergence speed can be dramatically improved.


page 7

page 8

page 10

page 11

page 15

page 16

page 25

page 26


Graph Reinforcement Learning for Predictive Power Allocation to Mobile Users

Allocating resources with future channels can save resource to ensure qu...

Deep Reinforcement Learning with Symmetric Prior for Predictive Power Allocation to Mobile Users

Deep reinforcement learning has been applied for a variety of wireless t...

Power Control for Wireless VBR Video Streaming: From Optimization to Reinforcement Learning

In this paper, we investigate the problem of power control for streaming...

Deep Reinforcement Learning for Joint Spectrum and Power Allocation in Cellular Networks

A wireless network operator typically divides the radio spectrum it poss...

Quality-Aware Deep Reinforcement Learning for Streaming in Infrastructure-Assisted Connected Vehicles

This paper proposes a deep reinforcement learning-based video streaming ...

Improving UE Energy Efficiency through Network-aware Video Streaming over 5G

Adaptive Bitrate (ABR) Streaming over the cellular networks has been wel...

Adaptive Streaming Perception using Deep Reinforcement Learning

Executing computer vision models on streaming visual data, or streaming ...

Please sign up or login with your details

Forgot password? Click here to reset