LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization
Authors: Kuangyi Chen, Jun Zhang, Yuxi Hu, Yi Zhou, Friedrich Fraundorfer
Organizations: Institute of Visual Computing, Graz University of Technology, 8010 Graz, Austria · Neuromorphic Automation and Intelligence Lab (NAIL), School of Artificial Intelligence and Robotics, Hunan University, Changsha 410012, China
Event cameras offer high-temporal-resolution sensing that remains reliable under high-speed motion and challenging lighting, making them promising for localization from LiDAR point clouds in GPS-denied and visually degraded environments. However, aligning sparse, asynchronous events with dense LiDAR maps is fundamentally ill-posed, as direct correspondence estimation suffers from modality gaps. We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event-depth flow fields to bridge the sensing-modality divide. Instead of treating edges as a post-hoc aid, LEAR couples them with flow estimation through a cross-modal fusion mechanism that injects modality-invariant geometric cues into the motion representation, and an iterative refinement strategy that enforces mutual consistency between the two tasks over multiple update steps. This synergy produces edge-aware, depth-aligned flow fields that enable more robust and accurate pose recovery via Perspective-n-Point (PnP) solvers. On several popular and challenging datasets, LEAR achieves superior performance over the best prior method. The source code, trained models, and demo videos are made publicly available online.
Figures & tables
Fig. 1: Our method ( bottom ) integrates a flow estimator with an edge detector within a mutually reinforcing cycle, enabling more accurate 2D–3D matching (i.e., event–depth flow) compared to the flow-only baseline EVLoc [ 4 ] ( top ).
Fig. 2: The three stages of the proposed method : 1) Image Generation : a depth map is generated using a pinhole camera model D based on a given initial pose guess, and an event image is constructed from the raw event stream using an event image generator E ; 2) Encoder : the depth map and event image are fed into the flow estimation branch ( green module ), and the depth map also serves as input to the edge detection branch ( red module ). Encoded depth features FD and edge features FED are fused across scales via the CFF module to produce edge-aware representations; 3) Iterative Edge and Flow Prediction : these, along with event features FEV and correlation volumes FC (constructed by indexing the element-wise products of FD and FEV using the current flow; see Fig. 5 ), are passed to the IFR module, where flow f and edge representations FED are refined iteratively and jointly, ultimately producing flow estimates f1→N and edge estimates p1→N from the corresponding decoders. The final flow estimate fN is used to recover the camera pose via a PnP solver within a RANSAC loop. The yellow circles in the IFR module denote specific operations. See Sec. IV-C and Fig. 5 for details.
Fig. 3: Architecture of the CFF module. Both the edge detection and flow estimation branches employ five-layer encoders with different spatial resolutions at each layer. Feature fusion is performed at the middle three layers, as illustrated in the diagram.
Fig. 4: Visualization of the encoded depth feature map. Each pixel value in the visualized feature map indicates the mean value of the corresponding feature vector at that location. The depth feature map after applying the CFF module contains richer texture details that align with structures.
Fig. 5: Architecture of the IFR module. Edge features ( FED0→FEDN ) and flow estimates ( f0→fN ) are progressively refined through mutual reinforcement. W, L, and C denote warping, indexing and concatenating, respectively.
CFF
IFR
EPE
Mean Error
Median Error
Transl.[cm] ↓
Rot.[°] ↓
Transl.[cm] ↓
Rot.[°] ↓
Oracle
1.66
4.14
0.39
3.31
0.22
Baseline [ 4 ]
6.10
9.01
1.10
8.11
0.97
Direct Mask
6.11
8.99
1.02
7.80
0.85
(a)
✓
5.75
8.06
0.94
7.20
0.83
(b)
✓
✓
5.52
8.07
0.93
6.85
0.74
TABLE I: Ablation study results. Bold numbers indicate the best performance except the upper bound Oracle .
Fig. 6: Visualization of the predicted edges and flows. Left column: edge predictions; Right column: flow estimates.
Dataset
Test Scene
Time
I2D-Loc [ 3 ]
EVLoc [ 4 ]
Ours
Transl.[cm] ↓
Rot.[°] ↓
Transl.[cm] ↓
Rot.[°] ↓
Transl.[cm] ↓
Rot.[°] ↓
M3ED [ 2 ]
falcon_indoor_flight
-
-
-
8.11
0.97
6.85
0.74
spot_indoor_building_loop
-
-
-
12.88
2.07
10.94
1.63
falcon_outdoor_day_penno_parking
day
-
-
25.11
1.32
22.40
1.18
falcon_forest_into_forest
day
-
-
20.46
1.87
19.35
1.84
spot_outdoor_day_penno_short_loop
day
-
-
17.89
0.88
14.37
0.74
TABLE II: Comparison of localization performance among I2D-Loc, EVLoc, and the proposed method on M3ED and DSEC. Median errors in translation and rotation are used as evaluation metrics. Bold numbers indicate the best performance.
Fig. 7: Comparison of ground-truth poses (green), initial poses (blue), and poses predicted by our method (red) on M3ED. Our predictions align closely with the ground truth, demonstrating high accuracy.
Methods
Stage
Runtime [ms]
Memory [MB]
Transl.[cm]/Rot.[°]
EVLoc [ 4 ]
Feature encoding
4.33
1202
8.11/0.97
Flow estimation (24 iters)
51.80
Ours
Feature encoding
7.71
1232
6.85/0.74
Flow estimation (24 iters)
100.43
Feature encoding
7.71
1232
7.01/0.75
Flow estimation (12 iters)
51.30
TABLE III: Runtime and memory consumption comparison.
Localizing an event camera against a pre-built LiDAR map can be cast as dense optical-flow estimation between a rendered depth view and an event image, followed by a Perspective-n-Point (PnP) solver over the induced 3D-2D correspondences. Existing pipelines rely on geometric consensus during pose estimation, but do not explicitly model the reliability or pose informativeness, i.e., how strongly a correspondence constrains the camera pose, of individual correspondences. We show that the natural way to learn it -- using the per-correspondence error to constrain the learning of confidence -- suffers from a depth-dependent bias: small pixel errors reside predominantly at large depths and do not lead to high pose informativeness. Instead, in our method (CELL), we learn a per-correspondence confidence end-to-end through the pose, using a differentiable probabilistic PnP whose log-partition term encourages weight configurations that yield a better-constrained pose distribution. The learned confidence is used in three ways: (i) it reweights the flow supervision in a decoupled training scheme that keeps pose gradients out of the flow/edge backbone; (ii) it drives a probabilistic correspondence selection at test time; and (iii) together with the network's edge-probability it weights a final edge-matching refinement. We further design a partial-completion depth representation that adds signal without hallucinating across large gaps. On M3ED and DSEC our full system improves over the LEAR baseline on the majority of the evaluated sequences: it reduces the median translation error by up to 26.9% and the median rotation error by up to 15.8%.
Panagiotis Kiousis, Kuangyi Chen, Jun Zhang +1
Dept. of Electrical & Computer Engineering, Univ. of Patras, Greece. · Institute of Visual Computing, Graz University of Technology, Graz, Austria.
LiDAR relocalization has attracted increasing attention as it can deliver accurate 6-DoF pose estimation in complex 3D environments. Recent learning-based regression methods offer efficient solutions by directly predicting global poses without the need for explicit map storage. However, these methods often struggle in challenging scenes due to their equal treatment of all predicted points, which is vulnerable to noise and outliers. In this paper, we propose LEADER, a robust LiDAR-based relocalization framework enhanced by a simple, yet effective geometric encoder. Specifically, a Robust Projection-based Geometric Encoder architecture which captures multi-scale geometric features is first presented to enhance descriptiveness in geometric representation. A Truncated Relative Reliability loss is then formulated to model point-wise ambiguity and mitigate the influence of unreliable predictions. Extensive experiments on the Oxford RobotCar and NCLT datasets demonstrate that LEADER outperforms state-of-the-art methods, achieving 24.1% and 73.9% relative reductions in position error over existing techniques, respectively. The source code is released on https://github.com/JiansW/LEADER.
Jianshi Wu, Minghang Zhu, Dunqiang Liu +5
Fujian Key Laboratory of Urban Intelligent Sensing and Computing · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics, Xiamen University, China · School of Engineering Mathematics and Technology, University of Bristol
Estimating dense 2D optical flow and 3D scene flow is essential for dynamic scene understanding. Recent work combines images, LiDAR, and event data to jointly predict 2D and 3D motion, yet most approaches operate in separate heterogeneous feature spaces. Without a shared latent space that all modalities can align to, these systems rely on multiple modality-specific blocks, leaving cross-sensor mismatches unresolved and making fusion unnecessarily complex.Event cameras naturally provide a spatiotemporal edge signal, which we can treat as an intrinsic edge field to anchor a unified latent representation, termed the Event Edge Space. Building on this idea, we introduce x2-Fusion, which reframes multimodal fusion as representation unification: event-derived spatiotemporal edges define an edge-centric homogeneous space, and image and LiDAR features are explicitly aligned in this shared representation.Within this space, we perform reliability-aware adaptive fusion to estimate modality reliability and emphasize stable cues under degradation. We further employ cross-dimension contrast learning to tightly couple 2D optical flow with 3D scene flow. Extensive experiments on both synthetic and real benchmarks show that x2-Fusion achieves state-of-the-art accuracy under standard conditions and delivers substantial improvements in challenging scenarios.
Ruishan Guo, Ciyu Ruan, Haoyang Wang +3
Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology · The University of Hong Kong