Abstract
In the field of image recognition, spiking neural networks (SNNs) haveachieved performance comparable to conventional artificial neural networks(ANNs). In such applications, SNNs essentially function as traditional neuralnetworks with quantized activation values. This article focuses on an anotheralternative perspective,viewing SNNs as binary-activated recurrent neuralnetworks (RNNs) for sequential modeling tasks.From this viewpoint, current SNNarchitectures face several fundamental challenges in sequence modeling: (1)Traditional models lack effective memory mechanisms for long-range sequencemodeling; (2) The biological-inspired components in SNNs (such as resetmechanisms and refractory period applications) remain theoreticallyunder-explored for sequence tasks; (3) The RNN-like computational paradigm inSNNs prevents parallel training across different timesteps.To address thesechallenges, this study conducts a systematic analysis of the fundamentalmechanisms underlying reset operations and refractory periods inbinary-activated RNN-based SNN sequence models. We re-examine whether suchbiological mechanisms are strictly necessary for generating sparse spikingpatterns, provide new theoretical explanations and insights, and ultimatelypropose the fixed-refractory-period SNN architecture for sequence modeling.