LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization

Kuangyi Chen; Jun Zhang; Yuxi Hu; Yi Zhou; Friedrich Fraundorfer

cs.CVMar 2, 2026

LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization

Authors: Kuangyi Chen, Jun Zhang, Yuxi Hu, Yi Zhou, Friedrich Fraundorfer

Organizations: Institute of Visual Computing, Graz University of Technology, 8010 Graz, Austria · Neuromorphic Automation and Intelligence Lab (NAIL), School of Artificial Intelligence and Robotics, Hunan University, Changsha 410012, China

Abstract

Event cameras offer high-temporal-resolution sensing that remains reliable under high-speed motion and challenging lighting, making them promising for localization from LiDAR point clouds in GPS-denied and visually degraded environments. However, aligning sparse, asynchronous events with dense LiDAR maps is fundamentally ill-posed, as direct correspondence estimation suffers from modality gaps. We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event-depth flow fields to bridge the sensing-modality divide. Instead of treating edges as a post-hoc aid, LEAR couples them with flow estimation through a cross-modal fusion mechanism that injects modality-invariant geometric cues into the motion representation, and an iterative refinement strategy that enforces mutual consistency between the two tasks over multiple update steps. This synergy produces edge-aware, depth-aligned flow fields that enable more robust and accurate pose recovery via Perspective-n-Point (PnP) solvers. On several popular and challenging datasets, LEAR achieves superior performance over the best prior method. The source code, trained models, and demo videos are made publicly available online.

Figures & tables

Fig. 1: Our method ( bottom ) integrates a flow estimator with an edge detector within a mutually reinforcing cycle, enabling more accurate 2D–3D matching (i.e., event–depth flow) compared to the flow-only baseline EVLoc [ 4 ] ( top ).

Fig. 2: The three stages of the proposed method : 1) Image Generation : a depth map is generated using a pinhole camera model $D$ based on a given initial pose guess, and an event image is constructed from the raw event stream using an event image generator $E$ ; 2) Encoder : the depth map and event image are fed into the flow estimation branch ( green module ), and the depth map also serves as input to the edge detection branch ( red module ). Encoded depth features $F_{D}$ and edge features $F_{ED}$ are fused across scales via the CFF module to produce edge-aware representations; 3) Iterative Edge and Flow Prediction : these, along with event features $F_{EV}$ and correlation volumes $F_{C}$ (constructed by indexing the element-wise products of $F_{D}$ and $F_{EV}$ using the current flow; see Fig. 5 ), are passed to the IFR module, where flow $\bm{f}$ and edge representations $F_{ED}$ are refined iteratively and jointly, ultimately producing flow estimates $\bm{f}^{1\rightarrow N}$ and edge estimates $p^{1\rightarrow N}$ from the corresponding decoders. The final flow estimate $\bm{f}^{N}$ is used to recover the camera pose via a PnP solver within a RANSAC loop. The yellow circles in the IFR module denote specific operations. See Sec. IV-C and Fig. 5 for details.

Fig. 3: Architecture of the CFF module. Both the edge detection and flow estimation branches employ five-layer encoders with different spatial resolutions at each layer. Feature fusion is performed at the middle three layers, as illustrated in the diagram.

Panel of Figure 4 — Fig. 4: Visualization of the encoded depth feature map. Each pixel value in the visualized feature map indicates the mean value of the corresponding feature vector at that location. The depth feature map after applying the CFF module contains richer texture details that align with structures.

Fig. 5: Architecture of the IFR module. Edge features (

F_{ED}^{0}\to F_{ED}^{N}

) and flow estimates (

\bm{f}^{0}\to\bm{f}^{N}

) are progressively refined through mutual reinforcement. W, L, and C denote warping, indexing and concatenating, respectively.

	CFF	IFR	EPE	Mean Error		Median Error
	CFF	IFR	EPE	Transl.[cm] $\downarrow$	Rot.[°] $\downarrow$	Transl.[cm] $\downarrow$	Rot.[°] $\downarrow$
Oracle			1.66	4.14	0.39	3.31	0.22
Baseline [ 4 ]			6.10	9.01	1.10	8.11	0.97
Direct Mask			6.11	8.99	1.02	7.80	0.85
(a)	$\checkmark$		5.75	8.06	0.94	7.20	0.83
(b)	$\checkmark$	$\checkmark$	5.52	8.07	0.93	6.85	0.74

TABLE I: Ablation study results. Bold numbers indicate the best performance except the upper bound Oracle .

Fig. 6: Visualization of the predicted edges and flows. Left column: edge predictions; Right column: flow estimates.

Dataset	Test Scene	Time	I2D-Loc [ 3 ]		EVLoc [ 4 ]		Ours
Dataset	Test Scene	Time	Transl.[cm] $\downarrow$	Rot.[°] $\downarrow$	Transl.[cm] $\downarrow$	Rot.[°] $\downarrow$	Transl.[cm] $\downarrow$	Rot.[°] $\downarrow$
M3ED [ 2 ]	falcon_indoor_flight	-	-	-	8.11	0.97	6.85	0.74
	spot_indoor_building_loop	-	-	-	12.88	2.07	10.94	1.63
	falcon_outdoor_day_penno_parking	day	-	-	25.11	1.32	22.40	1.18
	falcon_forest_into_forest	day	-	-	20.46	1.87	19.35	1.84
	spot_outdoor_day_penno_short_loop	day	-	-	17.89	0.88	14.37	0.74

TABLE II: Comparison of localization performance among I2D-Loc, EVLoc, and the proposed method on M3ED and DSEC. Median errors in translation and rotation are used as evaluation metrics. Bold numbers indicate the best performance.

Fig. 7: Comparison of ground-truth poses (green), initial poses (blue), and poses predicted by our method (red) on M3ED. Our predictions align closely with the ground truth, demonstrating high accuracy.

Methods	Stage	Runtime [ms]	Memory [MB]	Transl.[cm]/Rot.[°]
EVLoc [ 4 ]	Feature encoding	4.33	1202	8.11/0.97
	Flow estimation (24 iters)	51.80
Ours	Feature encoding	7.71	1232	6.85/0.74
	Flow estimation (24 iters)	100.43
	Feature encoding	7.71	1232	7.01/0.75
	Flow estimation (12 iters)	51.30

TABLE III: Runtime and memory consumption comparison.

LiDAR Localization

LiDAR (light detection and ranging) localization estimates a robot or vehicle’s position and orientation from laser scans and maps.

See all 42 papers

Learning Which Correspondences to Trust: Confidence-Weighted Event-Camera Localization in LiDAR Maps
Oct 8, 2026Panagiotis Kiousis, Kuangyi Chen, Jun Zhang +1Camera Pose Estimation Map-Based Localization
Dept. of Electrical & Computer Engineering, Univ. of Patras, Greece. · Institute of Visual Computing, Graz University of Technology, Graz, Austria.
LEADER: Learning Reliable Local-to-Global Correspondences for LiDAR Relocalization
Apr 13, 2026Jianshi Wu, Minghang Zhu, Dunqiang Liu +5LiDAR Localization Point Cloud Learning
Fujian Key Laboratory of Urban Intelligent Sensing and Computing · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics, Xiamen University, China · School of Engineering Mathematics and Technology, University of Bristol
$x^2$ -Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space
Mar 17, 2026Ruishan Guo, Ciyu Ruan, Haoyang Wang +3Scene Flow Estimation Optical Flow
Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology · The University of Hong Kong

Oct 8, 2026J/K move · Enter open · S save

Learning Which Correspondences to Trust: Confidence-Weighted Event-Camera Localization in LiDAR Maps

Panagiotis Kiousis, Kuangyi Chen, Jun Zhang, Friedrich Fraundorfer

Dept. of Electrical & Computer Engineering, Univ. of Patras, Greece. · Institute of Visual Computing, Graz University of Technology, Graz, Austria.

Camera Pose Estimation Map-Based Localization Event-Based Vision Camera-LiDAR Fusion

Open paper AskPDF arXiv

Localizing an event camera against a pre-built LiDAR map can be cast as dense optical-flow estimation between a rendered depth view and an event image, followed by a Perspective-n-Point (PnP) solver over the induced 3D-2D correspondences. Existing pipelines rely on geometric consensus during pose estimation, but do not explicitly model the reliability or pose informativeness, i.e., how strongly a correspondence constrains the camera pose, of individual correspondences. We show that the natural way to learn it -- using the per-correspondence error to constrain the learning of confidence -- suffers from a depth-dependent bias: small pixel errors reside predominantly at large depths and do not lead to high pose informativeness. Instead, in our method (CELL), we learn a per-correspondence confidence end-to-end through the pose, using a differentiable probabilistic PnP whose log-partition term encourages weight configurations that yield a better-constrained pose distribution. The learned confidence is used in three ways: (i) it reweights the flow supervision in a decoupled training scheme that keeps pose gradients out of the flow/edge backbone; (ii) it drives a probabilistic correspondence selection at test time; and (iii) together with the network's edge-probability it weights a final edge-matching refinement. We further design a partial-completion depth representation that adds signal without hallucinating across large gaps. On M3ED and DSEC our full system improves over the LEAR baseline on the majority of the evaluated sequences: it reduces the median translation error by up to 26.9% and the median rotation error by up to 15.8%.

Abstract

LiDAR Localization

Explore similar work

Learning Which Correspondences to Trust: Confidence-Weighted Event-Camera Localization in LiDAR Maps

LEADER: Learning Reliable Local-to-Global Correspondences for LiDAR Relocalization

x2x^2x2-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space

$x^2$ -Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space