Reliable visual place recognition (VPR) under dynamic real-world conditions is critical for autonomous robots, yet conventional deep networks remain limited by high computational and energy demands. Inspired by the mammalian navigation system, we introduce SpikeVPR, a bio-inspired and neuromorphic approach combining event-based cameras with spiking neural networks (SNNs) to generate compact, invariant place descriptors from few exemplars, achieving robust recognition under extreme changes in illumination, viewpoint, and appearance. SpikeVPR is trained end-to-end using surrogate gradient learning and incorporates EventDilation, a novel augmentation strategy enhancing robustness to speed and temporal variations. Evaluated on two challenging benchmarks (Brisbane-Event-VPR and NSAVP), SpikeVPR achieves performance comparable to state-of-the-art deep networks while using 50 times fewer parameters and consuming 30 and 250 times less energy, enabling real-time deployment on mobile and neuromorphic platforms. These results demonstrate that spike-based coding offers an efficient pathway toward robust VPR in complex, changing environments.
Figures & tables
Figure 1: Visual place recognition (VPR) in classical frame-based, biological, and bio-inspired systems. Classical systems (illustrated in blue in the first row) typically rely on RGB images captured at a fixed sampling rate (e.g., 30 or 60 Hz). These images are processed by deep neural networks, such as ResNet or VGG, to extract discriminative descriptors of the different locations (middle panel). The descriptor of a query image is then compared with those stored in memory using a similarity metric to identify the closest match. Although this approach achieves strong retrieval performance, it remains challenging to deploy on resource-constrained platforms because it depends on tens of millions of real-valued parameters, leading to high computational and memory demands that limit their practicality for portable implementations. In contrast, visual place recognition in biological systems (shown in green in the second row) is far more efficient. The retina primarily transmits sparse information as spikes, which occur mostly when changes in illumination, either increments or decrements, are detected in the visual scene. Because spikes are all-or-none signals, their processing along the visual pathway is highly efficient, and the entire system is estimated to consume only around twenty watts. At the end of this processing, cortical structures such as the entorhinal cortex encode environmental features that support the formation of hippocampal place cells, neurons that represent specific locations in the explored environment. The inset above the rat illustrates the place field of one such neuron. Our proposed approach, SpikeVPR (shown in orange in the last row), draws direct inspiration from biological systems. It employs an event-based camera, which, like the biological retina, detects changes in illumination in near real-time. The resulting ‘on’ and ‘off’ spikes are processed by a spiking neural network (SNN) that, similar to the brain, encodes scene descriptors using only binary values. In our implementation, the SNN is built on a SEW ResNet architecture, and scene descriptors are extracted using Spiking MixVPR (see the Materials and Methods).
Figure 2: Overview of the SpikeVPR architecture and its training procedure, illustrated using the Brisbane VPR dataset. Given the event stream corresponding to the vehicle’s current location (purple), SpikeVPR generates a 4096-dimensional descriptor through a spiking neural network (SNN). The architecture consists of an encoder built from Spike-Element-Wise (SEW) ResNet blocks (shown in the lower part of the middle panel) followed by an aggregation module. The SNN is trained using surrogate gradient learning (SGL) with a contrastive loss. This objective encourages high similarity between the latent representation of the current location and those of positive examples (i.e., places located within 30 meters of the query and sampled across different traversals, in green), while reducing similarity with descriptors corresponding to negative locations (i.e., other places in pink). RGB images are displayed for visualization purposes only and are not used in the processing pipeline.
Figure 3: Recall@N (top row) and Precision (bottom row) curves on the Brisbane VPR dataset. Results obtained with SpikeVPR are shown in orange. For comparison, we include curves estimated using the sum of absolute differences (SAD) method (blue) and principal component analysis (PCA) (green). The values displayed on the left of the first row correspond to the Recall@1. The values displayed on the right correspond to the Recall@5 (first row) and precision Recall@100 (second row). ( a ) Average performance across all traverses (Daytime, Morning, Sunrise, Sunset 2). Shaded regions indicate the standard error of the mean. ( b ) Performance measured separately for each tested traverse.
Figure 4: Performance and model complexity of SpikeVPR (dark orange) compared with two recent event-based deep learning approaches: Ensemble (orange) and EventVPR (light orange). ( a ) Average Recall@1 across the different traverses of the Brisbane-Event-VPR dataset (see Figure 3 for details). Error bars indicate the standard error computed across traverses. ( b ) Number of trainable parameters for each method. ( c ) Estimation of energy consumption of both ANNs (Ensemble, EventVPR) vs SpikeVPR (see Section Energy consumption estimation ).
Figure 5: Recall@N (top row) and Precision (bottom row) curves on the NSAVP dataset on Road 0 going forward and on reverse. The references in respective traverses are R0_FS0 and R0_RS0. ( a ) Average performance across all traverses (R0_FA0 and R0_RA0). Shaded regions indicate the standard error of the mean. ( b ) Performance measured separately for each tested traverse. See Figure 3 for more details.
Augmentation
R@1 (%)
Precision@100 (%)
None
14.22
14.81
D
37.78
42.81
X
18.96
22.22
E
20.59
22.96
X + E
24.74
27.26
D + E
37.93
45.19
Table 1: Effect of data augmentations on SpikeVPR performance. Recall at rank 1 (R@1, %) and Area under the precision-recall (PR) curve are reported under different augmentation settings. D: event dilation; X: x-axis transform; E: EventDrop.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Supplementary Figure 1: Illustration of three core challenges in VPR, shown for both event-based and RGB modalities on the Brisbane-Event-VPR Dataset. (a) Occlusion: The same place is captured with and without a significant foreground obstruction (a vehicle windshield pillar), leading to large differences in visual input. (b) Dynamic Objects: A parked truck partially occludes the scene in one traverse, generating spurious high-activity regions in the event frame that are absent in the reference, even though the underlying place remains the same. (c) Perceptual Aliasing: Two structurally similar but geographically distinct locations (1.4 km apart) produce similar appearances, creating a risk of false matches for VPR systems. In event data, red and blue spikes indicate brightness increases and decreases, respectively. RGB frames are shown for illustration only and are not used in our study.
Supplementary Figure 2: Illustration from the NSAVP dataset showing how the motion of a vehicle equipped with an event-based camera affects VPR. Each pair of images depicts the same location captured during two different traversals. (a) Stop with moving objects: When the vehicle is stationary, static scene elements generate no events, leaving only moving objects (e.g., passing cars) visible. This causes a near-complete loss of place-descriptive information, with dynamic distractors dominating the event representation. (b) Slowing Down: As the vehicle decelerates, event rates drop, producing sparse and noisy frames with reduced scene coverage compared to normal-speed traverses. In both cases, the event representation of the same place differs markedly between traverses, highlighting the sensitivity of event-based descriptors to vehicle velocity changes and dynamic objects.
Supplementary Figure 3: Examples of SpikeVPR predictions on the Brisbane-Event-VPR dataset. For each query (shown in purple on the left), the top three retrieved places are displayed. For each of these places, we report the cosine similarity score and the euclidean distance (in meters) to the query. Correct matches (within 30 meters) are shown in green, whereas incorrect matches are shown in red.
Supplementary Figure 4: Visual place recognition performance on the Brisbane-Event-VPR dataset with and without pretraining on NSAVP dataset. ( a ), Mean Recall@N (top) and Precision-Recall curves (bottom) across all four traverses, with shaded regions indicating ± 1 standard error (n = 4 traverses). ( b ), Per-traverse Recall@N (top) and Precision-Recall curves (bottom) for Daytime, Morning, Sunrise and Sunset conditions. SpikeVPR is evaluated both with pretraining on the NSAVP dataset (dark orange) and without pretraining (light orange), alongside SAD (blue) and PCA (teal) baselines. Values at curve endpoints denote Recall@1, Recall@5 and final precision, respectively.
Supplementary Figure 5: Examples of SpikeVPR predictions on the NSAVP dataset. For each query (shown in purple on the left), the top three retrieved places are displayed. For each of these places, we report the cosine similarity score and the spatial distance (in meters) to the query. Correct matches (within 30 meters) are shown in green, whereas incorrect matches are shown in red.
Neuromorphic sensing and computing promise low latency and high energy efficiency for robotics, yet most event-based visual place recognition (VPR) pipelines reintroduce dense, frame-like computation downstream of an inherently sparse sensor, forfeiting the sensor's latency advantage. In this paper, we address the problem of resolving an event-based VPR query in tens of microseconds on a commodity CPU. We train a two-layer spiking neural network (SNN) with additive spike-timing-dependent plasticity (STDP), which drives the learned weight matrix towards a strongly bimodal distribution, amenable to aggressive magnitude pruning. At inference, our pipeline exploits this sparsity by reading only the pruned weights corresponding to the query's active pixels. As surviving weights share a near-uniform magnitude, each neuron's score reduces to an integer count of matching synapses instead of a weighted sum, lowering per-query latency. This computation is small enough to remain resident in a CPU's fast on-chip cache. On three event-based VPR benchmarks, our sparse SNN exceeds the recall of LENS, an existing event-based SNN, while reducing per-query latency by 5.5-6.2x to tens of microseconds. It matches the recall of a dense variant of our network and of the sum-of-absolute-differences (SAD) baseline, while reading 10x fewer weights than the dense variant. We further characterise how recall and latency scale with the number of active pixels and reference places. These results suggest that the sparsity of both event-based input and learned weights can already be exploited for low-latency inference on conventional processors.
Spiking Neural Networks (SNNs) trained through unsupervised Spike-Timing-Dependent Plasticity (STDP) have been explored as solutions to visual loop closure problems, driven by the prospect of efficient on-device inference on neuromorphic devices. State-of-the-art STDP-based models deliver high classification accuracy but fail to reach the high Recall at 100% Precision (R@100P) needed for reliable autonomous navigation. We present a discrete, tensor-native implementation of the STDP-based SNN-VPR pipeline using PyTorch with snnTorch and evaluate it on a 100-place Nordland dataset using 15 independently-trained networks. The contribution of three decisions in the implementation is investigated. First, we show how to perform neuron assignment with a closed-form, deterministic tensor pipeline and show that it provides significantly higher R@100P than a standard argmax procedure. However, some of this gain comes from implementation differences compared to prior continuous-time models, which we measure independently. Second, ablation in isolation shows that state reset after each query helps improve R@100P regardless of the way neurons are assigned. Third, velocity-compensated sliding window aggregation over k consecutive frames reaches R@100P = 100.00% at k = 5 for constant-velocity traversal and an additional 0.20 ms latency. Taken together, these findings show the impact of inference stage design decisions in STDP-based SNN-VPR on recall precision, although the separate contribution of each mechanism and implementation differences is only partially disentangled and needs further examination.
Spiking Neural Networks (SNNs) promise energy-efficient vision, but applying them to RGB visual tracking remains difficult: Existing SNN tracking frameworks either do not fully align with spike-driven computation or do not fully leverage neurons' spatiotemporal dynamics, leading to a trade-off between efficiency and accuracy. To address this, we introduce SpikeTrack, a spike-driven framework for energy-efficient RGB object tracking. SpikeTrack employs a novel asymmetric design that uses asymmetric timestep expansion and unidirectional information flow, harnessing spatiotemporal dynamics while cutting computation. To ensure effective unidirectional information transfer between branches, we design a memory-retrieval module inspired by neural inference mechanisms. This module recurrently queries a compact memory initialized by the template to retrieve target cues and sharpen target perception over time. Extensive experiments demonstrate that SpikeTrack achieves the state-of-the-art among SNN-based trackers and remains competitive with advanced ANN trackers. Notably, it surpasses TransT on LaSOT dataset while consuming only 1/26 of its energy. To our knowledge, SpikeTrack is the first spike-driven framework to make RGB tracking both accurate and energy efficient. The code and models are available at https://github.com/faicaiwawa/SpikeTrack.
Qiuyang Zhang, Jiujun Cheng, Qichao Mao +5
Tongji University · Nova University of Lisbon · Stockholm University +2