Unleashing the Potential of Spiking Neural Networks for Sequential Modeling with Contextual Embedding

08/29/2023
by   Xinyi Chen, et al.
The Hong Kong Polytechnic University
0

The human brain exhibits remarkable abilities in integrating temporally distant sensory inputs for decision-making. However, existing brain-inspired spiking neural networks (SNNs) have struggled to match their biological counterpart in modeling long-term temporal relationships. To address this problem, this paper presents a novel Contextual Embedding Leaky Integrate-and-Fire (CE-LIF) spiking neuron model. Specifically, the CE-LIF model incorporates a meticulously designed contextual embedding component into the adaptive neuronal firing threshold, thereby enhancing the memory storage of spiking neurons and facilitating effective sequential modeling. Additionally, theoretical analysis is provided to elucidate how the CE-LIF model enables long-term temporal credit assignment. Remarkably, when compared to state-of-the-art recurrent SNNs, feedforward SNNs comprising the proposed CE-LIF neurons demonstrate superior performance across extensive sequential modeling tasks in terms of classification accuracy, network convergence speed, and memory capacity.

READ FULL TEXT

page 3

page 7

08/25/2023

TC-LIF: A Two-Compartment Spiking Neuron Model for Long-term Sequential Modelling

The identification of sensory cues associated with potential opportuniti...
07/14/2023

Long Short-term Memory with Two-Compartment Spiking Neuron

The identification of sensory cues associated with potential opportuniti...
09/18/2023

Spiking-LEAF: A Learnable Auditory front-end for Spiking Neural Networks

Brain-inspired spiking neural networks (SNNs) have demonstrated great po...
07/24/2022

Modeling Associative Plasticity between Synapses to Enhance Learning of Spiking Neural Networks

Spiking Neural Networks (SNNs) are the third generation of artificial ne...
07/11/2022

A Macrocolumn Architecture Implemented with Temporal (Spiking) Neurons

With the long-term goal of reverse-architecting the computational brain ...
08/20/2023

HoSNN: Adversarially-Robust Homeostatic Spiking Neural Networks with Adaptive Firing Thresholds

Spiking neural networks (SNNs) offer promise for efficient and powerful ...

Please sign up or login with your details

Forgot password? Click here to reset