cs.CVOct 12, 2026

Removing the Compute Bottleneck for Event-based Visual Place Recognition with Spiking Neural Networks

Authors: Somayeh Hussaini, Tobias Fischer, Michael Milford

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Neuromorphic sensing and computing promise low latency and high energy efficiency for robotics, yet most event-based visual place recognition (VPR) pipelines reintroduce dense, frame-like computation downstream of an inherently sparse sensor, forfeiting the sensor's latency advantage. In this paper, we address the problem of resolving an event-based VPR query in tens of microseconds on a commodity CPU. We train a two-layer spiking neural network (SNN) with additive spike-timing-dependent plasticity (STDP), which drives the learned weight matrix towards a strongly bimodal distribution, amenable to aggressive magnitude pruning. At inference, our pipeline exploits this sparsity by reading only the pruned weights corresponding to the query's active pixels. As surviving weights share a near-uniform magnitude, each neuron's score reduces to an integer count of matching synapses instead of a weighted sum, lowering per-query latency. This computation is small enough to remain resident in a CPU's fast on-chip cache. On three event-based VPR benchmarks, our sparse SNN exceeds the recall of LENS, an existing event-based SNN, while reducing per-query latency by 5.5-6.2x to tens of microseconds. It matches the recall of a dense variant of our network and of the sum-of-absolute-differences (SAD) baseline, while reading 10x fewer weights than the dense variant. We further characterise how recall and latency scale with the number of active pixels and reference places. These results suggest that the sparsity of both event-based input and learned weights can already be exploited for low-latency inference on conventional processors.

Figures & tables

Fig. 1 : Method overview. Preprocessing: event windows are accumulated into frames, downsampled, and reduced to their top- KK active pixels. Training: additive STDP with winner-take-all mechanism assigns one neuron per reference place and makes the weights bimodal. A threshold between the modes prunes the weight matrix and leaves ③ a binary weight matrix. Sparse inference: a query’s ① KK active pixels select ② only KK rows of the weight matrix. With each weight stored as ④ one uint8 , ⑤ a single 256-bit SIMD instruction sums 32 weights using ⑥ integer additions. The argmax gives the prediction ⑦ in tens of microseconds on one CPU core.
Fig. 2 : R@1 and latency performance comparisons across all methods and datasets. [top] Top-1 R@1: our sparse SNN exceeds LENS’s R@1 and matches dense SNN’s and SAD’s R@1 across all datasets. [bottom] Average per query latency, divided into preprocessing (light gray) and network time (solid colour). Labels give each method’s total (black) and network-only (solid color) speedup over LENS. LENS’s network stage dominates its latency, whereas our sparse SNN and SAD reduce network time to single digits. Dense SNN has a higher network latency than our sparse SNN but it is still substantially lower than LENS’s.
Fig. 3 : Learned weights across varying number of active pixels. Solid lines show the median line of the inactive pixel and surviving weights, with shaded bands of the same colour spanning the 25–75th (darker) and 5–95th (lighter) percentiles. The dashed line is the pruning threshold η\eta ( Equation 6 ). From 5 to 200 px, surviving weights are saturated at WmaxW_{\text{max}} and clearly separated from the inactive weights. With few active pixels, the inactive weights remain close to their uniform initialisation, U(0,0.5Wmax)U(0,0.5W_{\text{max}}) . From 400 px, the active pixels need more than each neuron’s fixed weight budget. Then, normalisation drives the inactive weights to zero, while the surviving weights fall below WmaxW_{\text{max}} and become more variable (median 0.36 WmaxW_{\text{max}} at 1024 px).
 DenseSparseRatio
Per-query cost
Element operationsNinNoutN_{\text{in}}N_{\text{out}}KNoutKN_{\text{out}} 
 1.64×1061.64\times 10^{6}1.60×1051.60\times 10^{5}10.2×10.2\times fewer
Weight bytes read4NinNout4N_{\text{in}}N_{\text{out}}KNoutKN_{\text{out}} 
 6.556.55 MB0.160.16 MB41.0×41.0\times fewer
Vector instructionsNinNout/8N_{\text{in}}N_{\text{out}}/8KNout/32KN_{\text{out}}/32 
TABLE I : Per-query inference cost of the dense and sparse backends on QCR-Event ( Nin=1024N_{\text{in}}=1024 , Nout=1600N_{\text{out}}=1600 , K=100K=100 ). The top panel gives the work each backend must perform, in arithmetic, memory traffic and issued AVX2 instructions (256-bit registers hold 8 float32 or 32 uint8 lanes), which were verified with hardware performance counters. Every ratio is the dense value divided by the sparse value. The bottom panel is a breakdown of the measured 6.73×6.73\times kernel speedup compared to dense SNN into its contributing factors.
Fig. 4 : [Top] Breakdown of the measured 6.73×6.73\times kernel speedup of our sparse SNN over the dense SNN on QCR-Event ( Nin=1024N_{\text{in}}=1024 , Nout=1600N_{\text{out}}=1600 , k=100k=100 ) into the factors of Table I . [Bottom] Share of total per-query latency attributed to each pipeline stage, for the same setting. The stages are preprocessing, per-call overhead, thread fork/join, weight accumulation, and readout (argmax). Preprocessing takes similar absolute time in both backends, so its larger share for sparse (85% vs 41%) reflects the sparse network’s lower cost. Thread fork/join appears only for the dense backend, which parallelises across 8 cores. Sparse runs single-threaded. Accumulation dominates the dense’s network cost (44%), whereas the sparse’s integer-add accumulation is comparatively small (9%).
Fig. 5 : Scalability of per-query latency with the number of reference places for our sparse SNN, dense SNN, SAD and LENS. Our sparse SNN scales similar to SAD throughout, flat and lowest below ∼103\sim 10^{3} places, and increasing roughly linearly while remaining the fastest. Dense SNN’s latency increases moderately until ∼103\sim 10^{3} places and grows the fastest beyond that. LENS stays flat at 130−140μs130-140\mu s while its fixed GPU overhead dominates its negligible per query compute, then increases linearly once that compute becomes significant.
Fig. 6 : Scalability experiment on how increasing the number of active pixels per frame impacts the R@1 and the per-query latency of our sparse SNN, dense SNN, SAD and LENS. R@1 increases with active pixel count and plateaus at 100–200 pixels (9.8–19.5% of the image). We chose 100 for balanced accuracy and sparsity. Beyond this, SAD and LENS stay stable, but our dense and sparse models degrade at high pixel counts as denser inputs become harder to discriminate, impacting sparse more as it collapses all active weights to a single value. Latency-wise, sparse tracks SAD up to ∼\sim 200 pixels before rising with pixel count, while dense stays flat throughout due to always processing the full frame regardless of sparsity.
More in this topic

Visual Place Recognition

Visual Place Recognition matches camera observations to mapped places despite changes in viewpoint, lighting, weather, or time.

See all 84 papers

Explore similar work

CardsList
Mar 24, 2026

Event-Driven Neuromorphic Vision Enables Energy-Efficient Visual Place Recognition

Reliable visual place recognition (VPR) under dynamic real-world conditions is critical for autonomous robots, yet conventional deep networks remain limited by high computational and energy demands. Inspired by the mammalian navigation system, we introduce SpikeVPR, a bio-inspired and neuromorphic approach combining event-based cameras with spiking neural networks (SNNs) to generate compact, invariant place descriptors from few exemplars, achieving robust recognition under extreme changes in illumination, viewpoint, and appearance. SpikeVPR is trained end-to-end using surrogate gradient learning and incorporates EventDilation, a novel augmentation strategy enhancing robustness to speed and temporal variations. Evaluated on two challenging benchmarks (Brisbane-Event-VPR and NSAVP), SpikeVPR achieves performance comparable to state-of-the-art deep networks while using 50 times fewer parameters and consuming 30 and 250 times less energy, enabling real-time deployment on mobile and neuromorphic platforms. These results demonstrate that spike-based coding offers an efficient pathway toward robust VPR in complex, changing environments.

Geoffroy Keime, Nicolas Cuperlier, Benoit R. Cottereau

IPAL, CNRS IRL 2955, Singapore · Univ Toulouse, CNRS, CerCo UMR 5549, Toulouse, France · Laboratoire ETIS UMR 8051, CY Cergy-Paris Université, ENSEA, CNRS, Cergy, France

Visual Place RecognitionNeuromorphic ComputingSpiking Neural NetworksEfficient Neural Network InferenceEvent-Based Vision
AskPDF
Jul 15, 2026

Visual Place Recognition Using Rate-Encoded Spiking Neural Networks with Discrete STDP Learning

Spiking Neural Networks (SNNs) trained through unsupervised Spike-Timing-Dependent Plasticity (STDP) have been explored as solutions to visual loop closure problems, driven by the prospect of efficient on-device inference on neuromorphic devices. State-of-the-art STDP-based models deliver high classification accuracy but fail to reach the high Recall at 100% Precision (R@100P) needed for reliable autonomous navigation. We present a discrete, tensor-native implementation of the STDP-based SNN-VPR pipeline using PyTorch with snnTorch and evaluate it on a 100-place Nordland dataset using 15 independently-trained networks. The contribution of three decisions in the implementation is investigated. First, we show how to perform neuron assignment with a closed-form, deterministic tensor pipeline and show that it provides significantly higher R@100P than a standard argmax procedure. However, some of this gain comes from implementation differences compared to prior continuous-time models, which we measure independently. Second, ablation in isolation shows that state reset after each query helps improve R@100P regardless of the way neurons are assigned. Third, velocity-compensated sliding window aggregation over k consecutive frames reaches R@100P = 100.00% at k = 5 for constant-velocity traversal and an additional 0.20 ms latency. Taken together, these findings show the impact of inference stage design decisions in STDP-based SNN-VPR on recall precision, although the separate contribution of each mechanism and implementation differences is only partially disentangled and needs further examination.

Altzi Tsanko, Oikonomou Katerina Maria, Antonios Gasteratos

Department of Production and Management Engineering, Democritus University of Thrace, Xanthi 67100, Greece

Visual Place RecognitionSpiking Neural NetworksSpike-Timing-Dependent Plasticity
AskPDF
Aug 25, 2026

Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Neuromorphic Virtual Sensing

Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in generalization, latency, and energy efficiency for edge deployment. Neural operators offer a promising alternative but remain reliant on power-intensive hardware. Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges. We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing. Within a trunk-based NOMAD architecture, SAR achieves over a fivefold improvement in the combined Latency-Error-Energy (LEE) metric compared with Variable Spiking Neuron (VSN) and Leaky Integrate-and-Fire (LIF) implementations. We further analyze spiking entropy and feature usage and introduce synthetic knowledge distillation, reducing the LEE score by more than twofold. Finally, we improve VSN through a ReLU-based spiking loss and graph-neighbor thresholding. On the Heat Exchanger dataset, these approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation. Overall, the work presented is a step towards energy-efficient virtual sensing by providing an alternative framework that can be positioned towards neuromorphic or other edge device integration that can be a gold standard to compare latency, energy, and error performance for future efficient designs that are sparsity or brain-inspired spiking based.

William Howes, Farid Ahmed, Syed Bahauddin Alam

Grainger College of Engineering, Nuclear, Plasma & Radiological Engineering Department, University of Illinois Urbana-Champaign, Urbana, IL, USA · National Center for Supercomputing Applications, Urbana, IL, USA

Activation SparsityNeuromorphic ComputingNeural OperatorsVirtual Sensing
AskPDF