ReD-CaNe: A Systematic Methodology for Resilience Analysis and Design of Capsule Networks under Approximations

by   Alberto Marchisio, et al.

Recent advances in Capsule Networks (CapsNets) have shown their superior learning capability, compared to the traditional Convolutional Neural Networks (CNNs). However, the extremely high complexity of CapsNets limits their fast deployment in real-world applications. Moreover, while the resilience of CNNs have been extensively investigated to enable their energy-efficient implementations, the analysis of CapsNets' resilience is a largely unexplored area, that can provide a strong foundation to investigate techniques to overcome the CapsNets' complexity challenge. Following the trend of Approximate Computing to enable energy-efficient designs, we perform an extensive resilience analysis of the CapsNets inference subjected to the approximation errors. Our methodology models the errors arising from the approximate components (like multipliers), and analyze their impact on the classification accuracy of CapsNets. This enables the selection of approximate components based on the resilience of each operation of the CapsNet inference. We modify the TensorFlow framework to simulate the injection of approximation noise (based on the models of the approximate components) at different computational operations of the CapsNet inference. Our results show that the CapsNets are more resilient to the errors injected in the computations that occur during the dynamic routing (the softmax and the update of the coefficients), rather than other stages like convolutions and activation functions. Our analysis is extremely useful towards designing efficient CapsNet hardware accelerators with approximate components. To the best of our knowledge, this is the first proof-of-concept for employing approximations on the specialized CapsNet hardware.


Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations

Complex Deep Neural Networks such as Capsule Networks (CapsNets) exhibit...

Effectiveness of the Recent Advances in Capsule Networks

Convolutional neural networks (CNNs) have revolutionized the field of de...

Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks

Capsule Networks (CapsNets), recently proposed by the Google Brain team,...

Making Convolutions Resilient via Algorithm-Based Error Detection Techniques

The ability of Convolutional Neural Networks (CNNs) to accurately proces...

HarDNN: Feature Map Vulnerability Evaluation in CNNs

As Convolutional Neural Networks (CNNs) are increasingly being employed ...

AX-DBN: An Approximate Computing Framework for the Design of Low-Power Discriminative Deep Belief Networks

The power budget for embedded hardware implementations of Deep Learning ...

Evaluating complexity and resilience trade-offs in emerging memory inference machines

Neuromorphic-style inference only works well if limited hardware resourc...

Please sign up or login with your details

Forgot password? Click here to reset