EM-SDTrack: Efficient Multi-Scale Spike-driven Tracking for Event Cameras
Organizations: University of Electronic Science and Technology of China, China · Shenzhen Loop Area Institute, China · China Nanhu Academy of Electronics and Information Technology, China
Abstract
Event-driven spiking neural networks (SNNs) match the sparse spatiotemporal output of event cameras, making spike-driven trackers highly attractive for neuromorphic edge deployment. However, events reveal a target only through their spatial distribution, whose extent varies more than tenfold within one sequence. Event tracking therefore demands multi-scale representation, which existing spike-driven trackers lack. Transferring multi-scale schemes from artificial neural networks either breaks spike-driven inference with floating-point multiplication or repeats backbone computation across input scales. To address this, we propose Spike-driven Multi-Scale Interaction (SDMI), which performs cross-scale interaction in a single spiking forward pass. Its core, Dyadic Spike Sampling (DSS), restricts sampling coordinates to dyadic fractions and thus requires only shifts and additions. The resulting M-SDTrack is fully spike-driven at inference. For tighter edge budgets, we further propose Polarity-Privileged Distillation (PPD), which distils M-SDTrack into EM-SDTrack. PPD treats the event counts cancelled by the voxel representation as privileged information. Its core constraint, Polarity-Guided Curvature Distillation (PGCD), aligns the curvature of the template-search relation under polarity-guided perturbations. On FE108, FELT and VisEvent, M-SDTrack achieves state-of-the-art accuracy with an AUC of 65.1, 41.7 and 42.0. EM-SDTrack retains 99% of the accuracy of its teacher M-SDTrack-Tiny with 27% of its energy and 18% of its parameters. On the Lynxi KA200 neuromorphic chip, it runs losslessly in real time at 116 FPS and 0.12 W.
Figures & tables
| Methods | Param. (M) | Spiking Neuron | Timesteps | Energy (mJ) | FE108 | FELT | VisEvent | |||
|---|---|---|---|---|---|---|---|---|---|---|
| AUC(%) | PR(%) | AUC(%) | PR(%) | AUC(%) | PR(%) | |||||
| STARK [ 36 ] | 28.23 | – | 1 | 58.88 | 57.4 | 89.2 | 39.3 | 50.8 | 34.1 | 46.8 |
| SimTrack [ 3 ] | 88.64 | – | 1 | 93.84 | 56.7 | 88.3 | 36.8 | 47.0 | 34.6 | 47.6 |
| OSTrack 256 [ 41 ] | 92.52 | – | 1 | 98.90 | 54.6 | 87.1 | 35.9 | 45.5 | 32.7 | 46.4 |
| ARTrack 256 [ 34 ] | 202.56 | – | 1 | 174.80 | 56.6 | 88.5 | 39.5 | 49.4 | 33.0 | 43.8 |
| SeqTrack-B 256 [ 5 ] | 90.60 | – | 1 | 302.68 | 53.5 | 85.5 | 33.0 | 42.0 | 28.6 | 43.3 |
Event-Based Vision
Event-based vision uses asynchronous camera measurements to perceive motion and scene changes with high temporal resolution and dynamic range.
See all 250 papers