Organizations: University of Electronic Science and Technology of China, China · Shenzhen Loop Area Institute, China · China Nanhu Academy of Electronics and Information Technology, China
Event-driven spiking neural networks (SNNs) match the sparse spatiotemporal output of event cameras, making spike-driven trackers highly attractive for neuromorphic edge deployment. However, events reveal a target only through their spatial distribution, whose extent varies more than tenfold within one sequence. Event tracking therefore demands multi-scale representation, which existing spike-driven trackers lack. Transferring multi-scale schemes from artificial neural networks either breaks spike-driven inference with floating-point multiplication or repeats backbone computation across input scales. To address this, we propose Spike-driven Multi-Scale Interaction (SDMI), which performs cross-scale interaction in a single spiking forward pass. Its core, Dyadic Spike Sampling (DSS), restricts sampling coordinates to dyadic fractions and thus requires only shifts and additions. The resulting M-SDTrack is fully spike-driven at inference. For tighter edge budgets, we further propose Polarity-Privileged Distillation (PPD), which distils M-SDTrack into EM-SDTrack. PPD treats the event counts cancelled by the voxel representation as privileged information. Its core constraint, Polarity-Guided Curvature Distillation (PGCD), aligns the curvature of the template-search relation under polarity-guided perturbations. On FE108, FELT and VisEvent, M-SDTrack achieves state-of-the-art accuracy with an AUC of 65.1, 41.7 and 42.0. EM-SDTrack retains 99% of the accuracy of its teacher M-SDTrack-Tiny with 27% of its energy and 18% of its parameters. On the Lynxi KA200 neuromorphic chip, it runs losslessly in real time at 116 FPS and 0.12 W.
Figures & tables
Figure 1: Scale variation in event tracking and the cost of existing remedies. (a) Ground-truth box area of one target along a sequence, which ranges from 197 to 2577 pixels. (b) A multi-scale module inserted into an SNN backbone adds floating-point sampling and weighting, so inference is no longer spike-driven. (c) Repeating the network at several input scales stays spike-driven but multiplies the computation. SDMI avoids both costs.
Figure 2: The two training stages. Stage 1 trains the teacher M-SDTrack, which integrates SDMI, on the voxel representation alone. Stage 2 distils it into the student EM-SDTrack through PPD, where both networks still take the voxel representation as input and the training-only event counts serve solely as the probe along which they are compared. At inference, the student runs on the voxel representation alone.
Figure 3: The SDMI block, drawn for the first Transformer stage. The upper part builds the buffer M once. The second block reuses the buffer returned by the first block, after a spiking linear layer maps it to the width of the second stage.
Figure 4: AUC on VisEvent versus the mean pixel cancellation, the first factor of Eq. ( 9 ) averaged over the pixels of the ground-truth box. The frames of all test sequences form one bin for zero values and five quantile bins for positive values.
Methods
Param. (M)
Spiking Neuron
Timesteps
Energy (mJ)
FE108
FELT
VisEvent
AUC(%)
PR(%)
AUC(%)
PR(%)
AUC(%)
PR(%)
STARK [ 36 ]
28.23
–
1
58.88
57.4
89.2
39.3
50.8
34.1
46.8
SimTrack [ 3 ]
88.64
–
1
93.84
56.7
88.3
36.8
47.0
34.6
47.6
OSTrack 256 [ 41 ]
92.52
–
1
98.90
54.6
87.1
35.9
45.5
32.7
46.4
ARTrack 256 [ 34 ]
202.56
–
1
174.80
56.6
88.5
39.5
49.4
33.0
43.8
SeqTrack-B 256 [ 5 ]
90.60
–
1
302.68
53.5
85.5
33.0
42.0
28.6
43.3
Table 1: Comparison with ANN-based and SNN-based trackers on FE108, FELT and VisEvent. Energy is estimated theoretically. The best three results are shown in red , blue and green .
Table 2: Ablation studies on FE108. (a) Components of SDMI. (b) Components of PPD. Row 2 uses the tracking loss alone, rows 3–7 set one weight of Eq. ( 14 ) to zero in turn, and row 8 removes the surplus Transformer blocks before training instead of dropping them progressively. (c) Index ls of the block kept in Transformer stage s , out of six and two. Row 1 of each sub-table is the default.
Figure 5: Sensitivity of EM-SDTrack on FE108 to (a) the loss functions of the terms La , Lc and Lo weighted by λa , λc and λo in Eq. ( 14 ), (b) the loss weights, and (c) the internal hyperparameters of PPD: η , ϑ and the smooth- L1 threshold.
Figure 6: (a) Relation of Eq. ( 11 ) averaged over the upper, middle and lower parts of the template target, on a colour scale shared by the three trackers. Circles mark the maxima. (b) EM-SDTrack running on the Lynxi KA200 chip 2 2 2 https://www.lynxi.com/ (right), with a programmable DC power supply (bottom) and a power meter (top).
Spiking Neural Networks (SNNs) promise energy-efficient vision, but applying them to RGB visual tracking remains difficult: Existing SNN tracking frameworks either do not fully align with spike-driven computation or do not fully leverage neurons' spatiotemporal dynamics, leading to a trade-off between efficiency and accuracy. To address this, we introduce SpikeTrack, a spike-driven framework for energy-efficient RGB object tracking. SpikeTrack employs a novel asymmetric design that uses asymmetric timestep expansion and unidirectional information flow, harnessing spatiotemporal dynamics while cutting computation. To ensure effective unidirectional information transfer between branches, we design a memory-retrieval module inspired by neural inference mechanisms. This module recurrently queries a compact memory initialized by the template to retrieve target cues and sharpen target perception over time. Extensive experiments demonstrate that SpikeTrack achieves the state-of-the-art among SNN-based trackers and remains competitive with advanced ANN trackers. Notably, it surpasses TransT on LaSOT dataset while consuming only 1/26 of its energy. To our knowledge, SpikeTrack is the first spike-driven framework to make RGB tracking both accurate and energy efficient. The code and models are available at https://github.com/faicaiwawa/SpikeTrack.
Qiuyang Zhang, Jiujun Cheng, Qichao Mao +5
Tongji University · Nova University of Lisbon · Stockholm University +2
Continuous-time, event-native spiking neural networks (SNNs) operate strictly on spike events, treating spike timing and ordering as the representation rather than an artifact of time discretization. This viewpoint aligns with biological computation and with the native resolution of event sensors and neuromorphic processors, while enabling compute and memory that scale with the number of events. However, two challenges hinder practical, end-to-end trainable event-based SNN systems: 1) exact charge--fire--reset dynamics impose inherently sequential processing of input spikes, and 2) precise spike times must be solved without time bins. We address both. First, we use parallel associative scans to consume multiple input spikes at once, yielding up to 44x speedups over sequential simulation while retaining exact hard-reset dynamics. Second, we implement differentiable spike time solvers that compute spike times to machine-precision without discrete-time approximations or restrictive analytic assumptions. We demonstrate the viability of training SNNs using our solutions on four event-based datasets on GPUs.
Todd Morrill, Christian Pehle, Anthony Zador
Department of Computer Science, Columbia University, New York, NY, USA · Department of Neuroscience, Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, USA
Event cameras follow a retina-inspired sensing principle, reporting local intensity changes asynchronously with hightemporal resolution and a wide dynamic range. Spiking Neural Networks (SNNs) complement these sparse event streams through brain-inspired dynamics, using sparse spikes and leaky membrane potentials to integrate information over time. However, many SNN object detectors process isolated event intervals with a single label and reset the network state after each prediction, thereby underusing temporal information in continuous event streams. We introduce Sequence-SOD, a sequence-aware SNN object detector that processes extended event sequences containing labels at multiple time points. Events are accumulated into short intervals, discretized into temporal steps, and fed sequentially to an SSD-style Spiking DenseNet while preserving membrane potentials across intervals within a sequence, so that detection is driven by an evolving neural state instead of independently reset input windows. On the Gen1 Automotive Detection Dataset, sequence-aware training improves mAP from 23.38 for single-interval training to 25.30 without augmentation and to 26.88 withevent-data augmentation. The model achieves a theoretical prediction frequency of 40 Hz. Training and evaluating SNN object detectors on extended event sequences improves their ability to exploit temporal cues while preserving the energy-efficiency benefits of sparse spiking computation. The results highlight sequence-aware training as a complementary direction to architectural improvements for event-based SNN detection.
Katharina Bendig, René Schuster, Didier Stricker
RPTU – Technical University Kaiserslautern-Landau, Gottlieb-Daimler-Straße 47, Kaiserslautern 67663, Germany · DFKI – German Research Center for Artificial Intelligence, Trippstadter Str. 122, Kaiserslautern 67663, Germany