EventGeM: Global-to-Local Feature Matching for Event-Based Visual Place Recognition
Authors: Adam D. Hines, Gokul B. Nair, Nicolás Marticorena, Michael Milford, Tobias Fischer
Organizations: QUT Centre for Robotics, School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, QLD 4000, Australia.
Event cameras are rapidly rising in popularity for robotic and computer vision tasks because their sparse activation delivers energy-efficient, high-dynamic-range, and fast sensing. Event cameras have been used in robotic navigation and localization tasks where positioning must occur in real time with sufficient accuracy. However, current event-based localization methods suffer from poor spatial understanding and are not viewpoint tolerant. In this paper, we address the problem of viewpoint-robust place recognition directly from event streams. We present EventGeM, a global-to-local feature fusion pipeline for event-based visual place recognition that combines whole-image feature detection to shortlist top candidates for 2D homography-based re-ranking with random sample consensus (RANSAC). We also contribute a regional generalized mean pooling (GeM) layer that learns to return the most relevant spatial features using per-row exponents to pool event streams into a compact global descriptor, trained on the NYC-Event-VPR dataset. These contributions overcome shortfalls in currently available event-based localization methods that fail to recognize similar places with large changes in viewpoint. To evaluate viewpoint-robust localization, we contribute a new event-based dataset that includes repeated traverses with a severe lateral shift. EventGeM improves absolute Recall@1 by 7 to 43 percentage points over the strongest baseline in each experiment. We also deploy EventGeM on a robotic platform, demonstrating real-time performance of our hierarchical pipeline. The code for EventGeM is available at https://github.com/AdamDHines/Event-GeM.
Figures & tables
Fig. 1: Schematic overview of EventGeM. A query event stream is converted to a multi-channel time surface (MCTS) and passed through the SuperEvent [ 11 ] backbone. Global features are pooled by our multi-exponent generalized mean (GeM) layer [ 12 ] and ranked by cosine similarity to produce a top- k shortlist. Local features from the SuperEvent keypoint detector head then re-rank the shortlist by 2D homography estimation with RANSAC, returning the matched database place.
Fig. 2: Multi-patch GeM pooling. (Top) The 128×30×40 feature map is split into a 4×4 grid; each row is pooled with its own learned exponent γr (values from Sect. IV-A ), giving four 512-dimensional blocks that are ℓ2 -normalized and concatenated. (Bottom) Pooled activations for one event frame under a global γ=5 , a 4×4 grid at γ=5 , and a 4×4 grid with learned per-row exponents. Warmer colors denote stronger activation.
Sunset1 (14,481)
Morning (13,456)
Daytime (14,321)
Method (Ref: Sunset2, 12,828)
R@1
R@5
R@10
R@1
R@5
R@10
R@1
R@5
R@10
LENS [ 9 ]
0.09
0.16
0.20
0.05
0.10
0.14
0.04
0.08
0.13
Sparse-Event-VPR [ 6 ]
0.05
0.08
0.13
0.02
0.04
0.05
0.05
0.08
0.09
EventVLAD [ 5 ]
0.43
0.57
0.64
0.09
0.17
0.22
0.08
0.16
0.22
LENS [ 9 ] +Re-rank
0.19
0.22
0.24
0.12
0.15
0.18
0.09
0.12
0.16
Sparse-Event-VPR [ 6 ] +Re-rank
0.11
0.12
0.15
0.03
0.04
0.05
0.05
0.09
0.10
TABLE I: Recall at K (R@K) on Brisbane-Event-VPR [ 4 ] , with Sunset2 as the reference. Bold is best, underline second best, per column. EventGeM (global only) is our descriptor before re-ranking. Bracketed values are the number of places.
R0-FS0 (18,218)
R0-FN0 (17,793)
Method (Ref: R0-FA0, 20,957)
R@1
R@5
R@10
R@1
R@5
R@10
LENS [ 9 ]
0.17
0.24
0.30
0.03
0.08
0.11
Sparse-Event-VPR [ 6 ]
0.14
0.24
0.29
0.02
0.08
0.11
EventVLAD [ 5 ]
0.20
0.28
0.33
0.05
0.11
0.14
LENS [ 9 ] +Re-rank
0.27
0.32
0.35
0.04
0.08
0.12
Sparse-Event-VPR [ 6 ] +Re-rank
0.19
0.27
0.31
0.04
0.08
0.11
TABLE II: Recall at K (R@K) on NSAVP [ 20 ] , with R0-FA0 as the reference. R0-FS0 is a daytime and R0-FN0 a night-time query. Bold is best, underline second best, per column. EventGeM (global only) is our descriptor before re-ranking. Bracketed values are the number of places.
Fig. 3: Example places from the left and right traverses of the Gardens-Point-Event dataset, separated by a lateral offset of approximately 5 m. Top: RGB frames. Middle: event frames, with red and blue denoting positive and negative polarity. Bottom: green lines are mutual nearest-neighbor keypoint matches from SuperEvent [ 11 ] .
Right (9,759)
Method (Ref: Left, 10,271)
R@1
R@5
R@10
LENS [ 9 ]
0.09
0.14
0.21
Sparse-Event-VPR [ 6 ]
0.06
0.14
0.21
EventVLAD [ 5 ]
0.18
0.40
0.53
EventGeM (global only)
0.37
0.58
0.67
EventGeM (ours)
0.55
0.70
0.75
TABLE III: Recall at K (R@K) on the Gardens-Point-Event dataset, Left as reference and Right as query. Bold is best, underline second best, per column. EventGeM (global only) is our descriptor before re-ranking. Bracketed values are the number of places. Only the event-native baselines are shown to compare differences in viewpoint tolerance across established event baselines.
Fig. 4: Per-query throughput against average Recall@1 (R@1) on Brisbane-Event-VPR [ 4 ] for EventGeM and the baselines [ 9 , 6 , 5 , 29 , 25 , 17 , 11 ] ; methods further right are faster. Each point averages over the Sunset1, Morning, and Daytime queries with Sunset2 as the reference.
Fig. 5: Online deployment of EventGeM on a robotic platform. Top, left to right: the AgileX Scout robot; its iniVation DAVIS346 dynamic vision sensor (DVS) and Jetson Orin AGX; the on-board pipeline; and the floor plan, with the teleoperated path in red, traversed once for the database and once as the query. Bottom, left to right: ground-truth correspondences between 1,400 queries (Q) and 1,600 database places (DB); the descriptor distance matrix, darker being closer; Recall@K; and per-query rate distribution, with median and mean marked.
Δt (ms)
R@1
R@5
R@10
1
0.13
0.18
0.20
10
0.58
0.61
0.62
50
0.87
0.88
0.89
100
0.92
0.93
0.94
250
0.95
0.96
0.96
TABLE IV: Effect of the MCTS accumulation window Δt on Recall at K (R@K) for EventGeM on the Brisbane-Event-VPR [ 4 ] dataset. The reference traverse was Sunset2 and the query traverse was Morning.
Condition
R@1
R@5
R@10
Dim.
γ=1
0.32
0.36
0.39
128
γ=2
0.53
0.57
0.58
128
γ=3
0.59
0.63
0.65
128
γ=5
0.59
0.64
0.66
128
γ=5 + 2×2 patch
0.78
0.80
0.81
512
γ=5 + 4×4 patch
0.81
0.83
0.83
2048
TABLE V: GeM pooling variants, evaluated on Brisbane-Event-VPR [ 4 ] with Sunset2 as reference and Morning as query; all models trained on NYC-Event-VPR [ 32 ] per Sect. III-D . Bold is best, underline second best, per column. R@K is Recall at K and Dim. the descriptor dimensionality. Rows marked global are scored before re-ranking.
Robot localization is an ongoing challenge that demands mapping and positioning systems that are tolerant to viewpoint change. Event cameras are attracting increasing interest and adoption in robotics; however, dealing with viewpoint variance is an under-investigated problem in existing event-based localizers. In addition, event-based datasets that emphasize viewpoint variance for challenging localization situations are scarce. Here, we introduce an event-based visual place recognition (VPR) system that performs robustly under viewpoint changes. We converted five large-scale geo-tagged datasets, conventionally used to train frame-based localization systems, into synthetic event streams using Image-to-Event (I2E) conversion, and used them to fine-tune a pre-trained event-based vision transformer backbone with a multi-loss function, yielding a system we call MegaEvent that learns viewpoint-robust features for place recognition. We achieved an average Recall@1 of 82% across three existing event-based localization datasets, leading the next best event-based method by 20 recall points, and frame-based VPR models applied directly to event frames by 8 to 26 recall points. We introduce a new, challenging dataset - Springfield-Event-VPR - which features a 3.7km walking route recorded in three camera orientations for a total of 11.1km, which MegaEvent outperforms the strongest baseline by 9 recall points. The code for MegaEvent is available at https://github.com/AdamDHines/megaevent.
Adam D. Hines, Michael Milford, Tobias Fischer
QUT Centre for Robotics, Queensland University of Technology, Brisbane QLD 4001
Event cameras have recently shown promising capabilities in instantaneous motion estimation due to their robustness to low light and fast motions. However, computing wide-baseline correspondence between two arbitrary views remains a significant challenge, since event appearance changes substantially with motion, and learning-based approaches are constrained by both scalability and limited wide-baseline supervision. We therefore introduce the first event matching model that achieves cross-dataset wide-baseline correspondence in a zero-shot manner: a single model trained once is deployed on unseen datasets without any target-domain fine-tuning or adaptation. To enable this capability, we introduce a motion-robust and computationally efficient attention backbone that learns multi-timescale features from event streams, augmented with sparsity-aware event token selection, making large-scale training on diverse wide-baseline supervision computationally feasible. To provide the supervision needed for wide-baseline generalization, we develop a robust event motion synthesis framework to generate large-scale event-matching datasets with augmented viewpoints, modalities, and motions. Extensive experiments across multiple benchmarks show that our framework achieves a 37.7% improvement over the previous best event feature matching methods. Code and data are available at: https://github.com/spikelab-jhu/Match-Any-Events.
Ruijun Zhang, Hang Su, Kostas Daniilidis +1
Johns Hopkins University, Baltimore MD 21218, USA · ShanghaiTech University, Shanghai, China · University of Pennsylvania, Philadelphia PA 19104, USA +1
Event cameras offer high-temporal-resolution sensing that remains reliable under high-speed motion and challenging lighting, making them promising for localization from LiDAR point clouds in GPS-denied and visually degraded environments. However, aligning sparse, asynchronous events with dense LiDAR maps is fundamentally ill-posed, as direct correspondence estimation suffers from modality gaps. We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event-depth flow fields to bridge the sensing-modality divide. Instead of treating edges as a post-hoc aid, LEAR couples them with flow estimation through a cross-modal fusion mechanism that injects modality-invariant geometric cues into the motion representation, and an iterative refinement strategy that enforces mutual consistency between the two tasks over multiple update steps. This synergy produces edge-aware, depth-aligned flow fields that enable more robust and accurate pose recovery via Perspective-n-Point (PnP) solvers. On several popular and challenging datasets, LEAR achieves superior performance over the best prior method. The source code, trained models, and demo videos are made publicly available online.
Kuangyi Chen, Jun Zhang, Yuxi Hu +2
Institute of Visual Computing, Graz University of Technology, 8010 Graz, Austria · Neuromorphic Automation and Intelligence Lab (NAIL), School of Artificial Intelligence and Robotics, Hunan University, Changsha 410012, China