cs.LGMay 10, 2026

Uncertainty-Aware Token Importance Estimation in Spiking Transformers

Authors: Wenxuan Liu, Zecheng Hao, Tong Bu, Yuran Wang, Zhaofei Yu

Organizations: School of Computer Science, Peking University · School of Computer Science, Peking University. · Institute for Artificial Intelligence, Peking University · Beijing, China · Peking University

Abstract

Spiking transformers have shown strong potential for neuromorphic vision, yet their token processing across multiple spiking steps still introduces substantial redundancy and inference cost. Existing token reduction methods mainly rely on response based cues, such as activation magnitude, firing statistics, or feature similarity. Although effective, these criteria do not explicitly characterize token importance from the perspective of temporally evolving class evidence. In spiking transformers, token representations are progressively formed across multiple spiking steps rather than determined at a single instant, suggesting that token importance should be evaluated not only by instantaneous responses but also by temporal uncertainty patterns. Our key observation is that tokens exhibit heterogeneous uncertainty trajectories over time, and that their temporally aggregated uncertainty statistics provide an effective cue for distinguishing informative tokens from redundant ones. Motivated by this, we propose Uncert, a training free and plug and play token importance estimation framework for spiking transformers. Specifically, Uncert models token wise class evidence with a Dirichlet distribution and summarizes each token temporal uncertainty using its mean and fluctuation across spiking steps, yielding an uncertainty aware importance score for token reduction during inference. Experiments on both static and neuromorphic benchmarks show that Uncert achieves favorable accuracy and efficiency tradeoffs, with the most consistent gains observed under token pruning. Further analysis reveals a clear empirical connection between temporal uncertainty patterns and token contribution, offering new insights into token dynamics in spiking transformers.

Explore similar work

CardsList
Apr 28, 2026cs.CV

Vision SmolMamba: Spike-Guided Token Pruning for Energy-Efficient Spiking State-Space Vision Models

Spiking Transformers have shown strong potential for long-range visual modeling through spike-driven self-attention. However, their quadratic token interactions remain fundamentally misaligned with the sparse and event-driven nature of spiking neural computation. To address this limitation, we propose Vision SmolMamba, an energy-efficient spiking state-space architecture that integrates spike-driven dynamics with linear-time selective recurrence. The key idea is a Spike-Guided Spatio-Temporal Token Pruner (SST-TP), which estimates token importance using both spike activation strength and first-spike latency. This mechanism progressively removes redundant tokens while preserving salient spatio-temporal information, enabling efficient scaling with token sparsity. Based on this mechanism, the proposed SmolMamba block incorporates spike events directly into bidirectional state-space recurrence, forming a spiking state-space vision backbone for efficient long-range modeling. Extensive experiments on both static and event-based benchmarks, including ImageNet-1K, CIFAR10/100, CIFAR10-DVS, and DVS128 Gesture, demonstrate that Vision SmolMamba consistently achieves superior accuracy-efficiency trade-offs. In particular, it reduces the estimated energy cost by at least 1.5x compared with prior spiking Transformer baselines and a Spiking Mamba variant while maintaining competitive or improved accuracy. These results demonstrate that combining spike-guided token sparsity with state-space modeling offers a scalable and energy-efficient paradigm for spiking vision systems.
Dewei Bai, Hongxiang Peng, Yunyun Zeng +3
Self-Supervised Vision TransformersVisual Token PruningBidirectional MambaPruningDinov3Energy-Efficient
AskPDF
Apr 17, 2026cs.LG

Closing the Theory-Practice Gap in Spiking Transformers via Effective Dimension

Spiking transformers achieve competitive accuracy with conventional transformers while offering 3838-57×57\times energy efficiency on neuromorphic hardware, yet no theoretical framework guides their design. This paper establishes the first comprehensive expressivity theory for spiking self-attention. We prove that spiking attention with Leaky Integrate-and-Fire neurons is a universal approximator of continuous permutation-equivariant functions, providing explicit spike circuit constructions including a novel lateral inhibition network for softmax normalization with proven O(1/T)O(1/\sqrt{T}) convergence. We derive tight spike-count lower bounds via rate-distortion theory: ε\varepsilon-approximation requires Ω(Lf2nd/ε2)Ω(L_f^2 nd/\varepsilon^2) spikes, with rigorous information-theoretic derivation. Our key insight is input-dependent bounds using measured effective dimensions (deff=47d_{\text{eff}}=47--8989 for CIFAR/ImageNet), explaining why T=4T=4 timesteps suffice despite worst-case T≥10,000T \geq 10{,}000 predictions. We provide concrete design rules with calibrated constants (C=2.3C=2.3, 95% CI: [1.9,2.7][1.9, 2.7]). Experiments on Spikformer, QKFormer, and SpikingResformer across vision and language benchmarks validate predictions with R2=0.97R^2=0.97 (p<0.001p<0.001). Our framework provides the first principled foundation for neuromorphic transformer design.
Dongxin Guo, Jikun Wu, Siu Ming Yiu
Neuromorphic HardwareTransformer ArchitecturesExpressivityKnowledge Gaps
AskPDF
Sep 22, 2026cs.NE

Rethinking Pairwise Token Interaction in Spiking Transformers

Spiking Transformers inherit token interaction mechanisms from conventional Transformers, yet their sparse binary representations fundamentally alter how token-to-token communication is established. In particular, spike-based query-key matching produces highly sparse and input-dependent interaction patterns, coupling information propagation to the instantaneous availability of matching spike events. This motivates a different interaction paradigm in which long-range communication does not rely solely on pairwise spike coincidence. We therefore propose Gated Spike Axial Propagation (GSAP), a spike-native token interaction mechanism that decouples information propagation from context selection. Instead of directly determining communication through query-key matching, GSAP first propagates spike-based context along the horizontal and vertical axes, allowing information to reach distant tokens through structured sequential propagation. A receiver-conditioned gate then determines how much of the propagated context is incorporated at each token, while a lightweight local pathway preserves fine-grained neighborhood information. In this way, GSAP reformulates token interaction as a propagate-then-select process, enabling structured long-range communication while retaining the sparse event-driven nature of spiking representations. Code is available at https://github.com/Fancyssc/GSAP.
Sicheng Shen, Dongcheng Zhao, Zhiyuan Li +5
Topic pending
AskPDF