Shifting Capsule Networks from the Cloud to the Deep Edge

by   Miguel Costa, et al.

Capsule networks (CapsNets) are an emerging trend in image processing. In contrast to a convolutional neural network, CapsNets are not vulnerable to object deformation, as the relative spatial information of the objects is preserved across the network. However, their complexity is mainly related with the capsule structure and the dynamic routing mechanism, which makes it almost unreasonable to deploy a CapsNet, in its original form, in a resource-constrained device powered by a small microcontroller (MCU). In an era where intelligence is rapidly shifting from the cloud to the edge, this high complexity imposes serious challenges to the adoption of CapsNets at the very edge. To tackle this issue, we present an API for the execution of quantized CapsNets in Cortex-M and RISC-V MCUs. Our software kernels extend the Arm CMSIS-NN and RISC-V PULP-NN, to support capsule operations with 8-bit integers as operands. Along with it, we propose a framework to perform post training quantization of a CapsNet. Results show a reduction in memory footprint of almost 75 software kernels for the Arm Cortex-M are, at least, 5.70x faster than a pre-quantized CapsNet running on an NVIDIA GTX 980 Ti graphics card. For RISC-V, the throughout gain increases to 26.28x and 56.91x for a single- and octa-core configuration, respectively.


page 1

page 2

page 3

page 4


Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks

Capsule Networks (CapsNets), recently proposed by the Google Brain team,...

CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs

Deep Neural Networks are becoming increasingly popular in always-on IoT ...

Towards Efficient Capsule Networks

From the moment Neural Networks dominated the scene for image processing...

Enabling Mixed-Precision Quantized Neural Networks in Extreme-Edge Devices

The deployment of Quantized Neural Networks (QNN) on advanced microcontr...

Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations

Complex Deep Neural Networks such as Capsule Networks (CapsNets) exhibit...

Handwritten Indic Character Recognition using Capsule Networks

Convolutional neural networks(CNNs) has become one of the primary algori...

Enabling Highly Efficient Capsule Networks Processing Through A PIM-Based Architecture Design

In recent years, the CNNs have achieved great successes in the image pro...

Please sign up or login with your details

Forgot password? Click here to reset