cs.AINov 10, 2026

Recurrent Modular Structure in Spiking Neural Network Reinforcement Learning

Authors: Vikram Ramavarapu, Himanshu Udupi, Ashish Gautam

Organizations: Siebel School of Computing and Data Science University of Illinois Urbana-Champaign Champaign, IL, USA · Computer Science and Mathematics Division Oak Ridge National Laboratory Oak Ridge, TN, USA

Abstract

Biological neural circuits are both modular and recurrent, with dense communities sparsely linked to one another. Spiking neural networks (SNNs) for reinforcement learning are usually feed-forward, and it is unclear how to obtain functional modules in a recurrent SNN or what they buy. In this work, we introduce a class of spiking networks, called Community Recurrent SNNs (CoRe SNNs): recurrent networks partitioned into communities with fixed roles. A controller community receives the stimulus, each action has its own community, and latent communities carry internal state. Spiking activity is then a walk between communities, and the action is the community the walk reaches. All learning is local and three-factor, and the recurrent communities are closer to cortical microcircuits than a feed-forward actor. We compare three regimes for obtaining the communities: dynamic modularity, in which a new learning rule we introduce, Map-STDP, encourages modular structure during learning; partial modularity, in which a stochastic block model (SBM) initialization enforces it partly; and full modularity, in which sealing enforces it fully. On CartPole, across three spiking neuron models, Map-STDP improves learning over the same network without it matches full modularity (sealed communities) on two of the three models, and makes the observed spike walk more compressible. Replacing the feed-forward actor of a TD-STDP actor-critic with recurrent action communities improves on the feed-forward network on CartPole and LunarLander, and matches it on Acrobot. How strongly to enforce modularity depends on the task: full modularity helps most on Acrobot, while on LunarLander the dynamic, partial and full regimes perform alike. Overall, these recurrent modular networks can match or even improve upon feed-forward approaches, with improvements in bio-plausibility and energy efficiency.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Scalable Learning in Structured Recurrent Spiking Neural Networks without Backpropagation

    May 1, 2026Bo Tang, Weiwei XieRecurrent Neural NetworksSpiking Neural Networks

  2. Phase State Space Models: Parallel, Surrogate-Free Training of Spiking Networks

    Aug 7, 2026Wilkie Olin-AmmentorpSpiking Neural NetworksState Space Models

  3. Exploring napping paradigm for Recurrent Spiking Neural Networks

    Sep 12, 2026Andreas Massey, Stefano Nichele, Aliaksandr HubinSpiking Neural NetworksRecurrent Neural Networks