A Spatial-channel-temporal-fused Attention for Spiking Neural Networks

by   Wuque Cai, et al.

Spiking neural networks (SNNs) mimic brain computational strategies, and exhibit substantial capabilities in spatiotemporal information processing. As an essential factor for human perception, visual attention refers to the dynamic selection process of salient regions in biological vision systems. Although mechanisms of visual attention have achieved great success in computer vision, they are rarely introduced into SNNs. Inspired by experimental observations on predictive attentional remapping, we here propose a new spatial-channel-temporal-fused attention (SCTFA) module that can guide SNNs to efficiently capture underlying target regions by utilizing historically accumulated spatial-channel information. Through a systematic evaluation on three event stream datasets (DVS Gesture, SL-Animals-DVS and MNIST-DVS), we demonstrate that the SNN with the SCTFA module (SCTFA-SNN) not only significantly outperforms the baseline SNN (BL-SNN) and other two SNN models with degenerated attention modules, but also achieves competitive accuracy with existing state-of-the-art methods. Additionally, our detailed analysis shows that the proposed SCTFA-SNN model has strong robustness to noise and outstanding stability to incomplete data, while maintaining acceptable complexity and efficiency. Overall, these findings indicate that appropriately incorporating cognitive mechanisms of the brain may provide a promising approach to elevate the capability of SNNs.


page 1

page 3

page 4

page 6

page 7

page 9


TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks

Spiking Neural Networks (SNNs) is a practical approach toward more data-...

Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework

Recently, brain-inspired computing models have shown great potential to ...

Training Robust Spiking Neural Networks with ViewPoint Transform and SpatioTemporal Stretching

Neuromorphic vision sensors (event cameras) simulate biological visual p...

STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks

Spiking Neural Networks (SNNs), as one of the algorithmic models in neur...

Attention Spiking Neural Networks

Benefiting from the event-driven and sparse spiking characteristics of t...

Temporal-wise Attention Spiking Neural Networks for Event Streams Classification

How to effectively and efficiently deal with spatio-temporal event strea...

Deep recurrent spiking neural networks capture both static and dynamic representations of the visual cortex under movie stimuli

In the real world, visual stimuli received by the biological visual syst...

Please sign up or login with your details

Forgot password? Click here to reset