Real Spike: Learning Real-valued Spikes for Spiking Neural Networks

by   Yufei Guo, et al.

Brain-inspired spiking neural networks (SNNs) have recently drawn more and more attention due to their event-driven and energy-efficient characteristics. The integration of storage and computation paradigm on neuromorphic hardwares makes SNNs much different from Deep Neural Networks (DNNs). In this paper, we argue that SNNs may not benefit from the weight-sharing mechanism, which can effectively reduce parameters and improve inference efficiency in DNNs, in some hardwares, and assume that an SNN with unshared convolution kernels could perform better. Motivated by this assumption, a training-inference decoupling method for SNNs named as Real Spike is proposed, which not only enjoys both unshared convolution kernels and binary spikes in inference-time but also maintains both shared convolution kernels and Real-valued Spikes during training. This decoupling mechanism of SNN is realized by a re-parameterization technique. Furthermore, based on the training-inference-decoupled idea, a series of different forms for implementing Real Spike on different levels are presented, which also enjoy shared convolutions in the inference and are friendly to both neuromorphic and non-neuromorphic hardware platforms. A theoretical proof is given to clarify that the Real Spike-based SNN network is superior to its vanilla counterpart. Experimental results show that all different Real Spike versions can consistently improve the SNN performance. Moreover, the proposed method outperforms the state-of-the-art models on both non-spiking static and neuromorphic datasets.


page 1

page 2

page 3

page 4


Deep Learning in Spiking Phasor Neural Networks

Spiking Neural Networks (SNNs) have attracted the attention of the deep ...

EqSpike: Spike-driven Equilibrium Propagation for Neuromorphic Implementations

Neuromorphic systems achieve high energy efficiency by computing with sp...

MAP-SNN: Mapping Spike Activities with Multiplicity, Adaptability, and Plasticity into Bio-Plausible Spiking Neural Networks

Spiking Neural Network (SNN) is considered more biologically realistic a...

Training Full Spike Neural Networks via Auxiliary Accumulation Pathway

Due to the binary spike signals making converting the traditional high-p...

Efficient spike encoding algorithms for neuromorphic speech recognition

Spiking Neural Networks (SNN) are known to be very effective for neuromo...

A New Neuromorphic Computing Approach for Epileptic Seizure Prediction

Several high specificity and sensitivity seizure prediction methods with...

Spiking Graph Convolutional Networks

Graph Convolutional Networks (GCNs) achieve an impressive performance du...

Please sign up or login with your details

Forgot password? Click here to reset