Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning

by   Duzhen Zhang, et al.

With the Deep Neural Networks (DNNs) as a powerful function approximator, Deep Reinforcement Learning (DRL) has been excellently demonstrated on robotic control tasks. Compared to DNNs with vanilla artificial neurons, the biologically plausible Spiking Neural Network (SNN) contains a diverse population of spiking neurons, making it naturally powerful on state representation with spatial and temporal information. Based on a hybrid learning framework, where a spike actor-network infers actions from states and a deep critic network evaluates the actor, we propose a Population-coding and Dynamic-neurons improved Spiking Actor Network (PDSAN) for efficient state representation from two different scales: input coding and neuronal coding. For input coding, we apply population coding with dynamically receptive fields to directly encode each input state component. For neuronal coding, we propose different types of dynamic-neurons (containing 1st-order and 2nd-order neuronal dynamics) to describe much more complex neuronal dynamics. Finally, the PDSAN is trained in conjunction with deep critic networks using the Twin Delayed Deep Deterministic policy gradient algorithm (TD3-PDSAN). Extensive experimental results show that our TD3-PDSAN model achieves better performance than state-of-the-art models on four OpenAI gym benchmark tasks. It is an important attempt to improve RL with SNN towards the effective computation satisfying biological plausibility.


Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Control

The energy-efficient control of mobile robots is crucial as the complexi...

Online computation of sparse representations of time varying stimuli using a biologically motivated neural network

Natural stimuli are highly redundant, possessing significant spatial and...

Mixture of personality improved Spiking actor network for efficient multi-agent cooperation

Adaptive human-agent and agent-agent cooperation are becoming more and m...

Neuro-Planner: A 3D Visual Navigation Method for MAV with Depth Camera based on Neuromorphic Reinforcement Learning

Traditional visual navigation methods of micro aerial vehicle (MAV) usua...

Efficient Computation in Adaptive Artificial Spiking Neural Networks

Artificial Neural Networks (ANNs) are bio-inspired models of neural comp...

A Deep 2-Dimensional Dynamical Spiking Neuronal Network for Temporal Encoding trained with STDP

The brain is known to be a highly complex, asynchronous dynamical system...

A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks

The biological neural systems evolved to adapt to ecological environment...

Please sign up or login with your details

Forgot password? Click here to reset