An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic

01/09/2022
by   Tao Niu, et al.
0

Recently, the applications of deep neural network (DNN) have been very prominent in many fields such as computer vision (CV) and natural language processing (NLP) due to its superior feature extraction performance. However, the high-dimension parameter model and large-scale mathematical calculation restrict the execution efficiency, especially for Internet of Things (IoT) devices. Different from the previous cloud/edge-only pattern that brings huge pressure for uplink communication and device-only fashion that undertakes unaffordable calculation strength, we highlight the collaborative computation between the device and edge for DNN models, which can achieve a good balance between the communication load and execution accuracy. Specifically, a systematic on-demand co-inference framework is proposed to exploit the multi-branch structure, in which the pre-trained Alexnet is right-sized through early-exit and partitioned at an intermediate DNN layer. The integer quantization is enforced to further compress transmission bits. As a result, we establish a new Deep Reinforcement Learning (DRL) optimizer-Soft Actor Critic for discrete (SAC-d), which generates the exit point, partition point, and compressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an optimizer can well adapt to the complex environment like dynamic wireless channel and arbitrary CPU processing, and is capable of supporting the 5G URLLC. Real-world experiment on Raspberry Pi 4 and PC shows the outperformance of the proposed solution.

READ FULL TEXT
research
12/20/2021

Variational Quantum Soft Actor-Critic

Quantum computing has a superior advantage in tackling specific problems...
research
03/01/2023

The Point to Which Soft Actor-Critic Converges

Soft actor-critic is a successful successor over soft Q-learning. While ...
research
12/07/2021

Attention-Based Model and Deep Reinforcement Learning for Distribution of Event Processing Tasks

Event processing is the cornerstone of the dynamic and responsive Intern...
research
05/05/2021

ScissionLite: Accelerating Distributed Deep Neural Networks Using Transfer Layer

Industrial Internet of Things (IIoT) applications can benefit from lever...
research
11/06/2022

Collaborative Video Analytics on Distributed Edges with Multiagent Deep Reinforcement Learning

Deep Neural Network (DNN) based video analytics empowers many computer v...
research
10/24/2021

A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments

Fog/Edge computing is a novel computing paradigm supporting resource-con...
research
05/04/2020

MARS: Multi-Scalable Actor-Critic Reinforcement Learning Scheduler

In this paper, we introduce a new scheduling algorithm MARS based on a c...

Please sign up or login with your details

Forgot password? Click here to reset