Abstract
Spiking neural networks (SNNs) are widely applied in various fields due totheir energy-efficient and fast-inference capabilities. Applying SNNs toreinforcement learning (RL) can significantly reduce the computational resourcerequirements for agents and improve the algorithm's performance underresource-constrained conditions. However, in current spiking reinforcementlearning (SRL) algorithms, the simulation results of multiple time steps canonly correspond to a single-step decision in RL. This is quite different fromthe real temporal dynamics in the brain and also fails to fully exploit thecapacity of SNNs to process temporal data. In order to address this temporalmismatch issue and further take advantage of the inherent temporal dynamics ofspiking neurons, we propose a novel temporal alignment paradigm (TAP) thatleverages the single-step update of spiking neurons to accumulate historicalstate information in RL and introduces gated units to enhance the memorycapacity of spiking neurons. Experimental results show that our method cansolve partially observable Markov decision processes (POMDPs) and multi-agentcooperation problems with similar performance as recurrent neural networks(RNNs) but with about 50% power consumption.