Abstract
In recent years, Recurrent Spiking Neural Networks (RSNNs) have shownpromising potential in long-term temporal modeling. Many studies focus onimproving neuron models and also integrate recurrent structures, leveragingtheir synergistic effects to improve the long-term temporal modelingcapabilities of Spiking Neural Networks (SNNs). However, these studies oftenplace an excessive emphasis on the role of neurons, overlooking the importanceof analyzing neurons and recurrent structures as an integrated framework. Inthis work, we consider neurons and recurrent structures as an integrated systemand conduct a systematic analysis of gradient propagation along the temporaldimension, revealing a challenging gradient vanishing problem. To address thisissue, we propose the Skip Recurrent Connection (SRC) as a replacement for thevanilla recurrent structure, effectively mitigating the gradient vanishingproblem and enhancing long-term temporal modeling performance. Additionally, wepropose the Adaptive Skip Recurrent Connection (ASRC), a method that can learnthe skip span of skip recurrent connection in each layer of the network.Experiments show that replacing the vanilla recurrent structure in RSNN withSRC significantly improves the model's performance on temporal benchmarkdatasets. Moreover, ASRC-SNN outperforms SRC-SNN in terms of temporal modelingcapabilities and robustness.