$x^2$-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space

Ruishan Guo; Ciyu Ruan; Haoyang Wang; Zihang Gong; Jingao Xu; Xinlei Chen

cs.CVMar 17, 2026

$x^2$ -Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space

Authors: Ruishan Guo, Ciyu Ruan, Haoyang Wang, Zihang Gong, Jingao Xu, Xinlei Chen

Organizations: Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology · The University of Hong Kong

Abstract

Estimating dense 2D optical flow and 3D scene flow is essential for dynamic scene understanding. Recent work combines images, LiDAR, and event data to jointly predict 2D and 3D motion, yet most approaches operate in separate heterogeneous feature spaces. Without a shared latent space that all modalities can align to, these systems rely on multiple modality-specific blocks, leaving cross-sensor mismatches unresolved and making fusion unnecessarily complex.Event cameras naturally provide a spatiotemporal edge signal, which we can treat as an intrinsic edge field to anchor a unified latent representation, termed the Event Edge Space. Building on this idea, we introduce $x^2$ -Fusion, which reframes multimodal fusion as representation unification: event-derived spatiotemporal edges define an edge-centric homogeneous space, and image and LiDAR features are explicitly aligned in this shared representation.Within this space, we perform reliability-aware adaptive fusion to estimate modality reliability and emphasize stable cues under degradation. We further employ cross-dimension contrast learning to tightly couple 2D optical flow with 3D scene flow. Extensive experiments on both synthetic and real benchmarks show that $x^2$ -Fusion achieves state-of-the-art accuracy under standard conditions and delivers substantial improvements in challenging scenarios.

Figures & tables

Figure 1: Fusion paradigms for multimodal perception. (a) RPEFlow [ 8 ] : stage-wise fusion across 2D/3D spaces. (b) CMX [ 9 ] :image-like encodings with pairwise rectification/attention at each backbone stage. (c) VisMoFlow [ 10 ] : separate luminance/structure/correlation spaces with dedicated modules. (d) Ours: preserves native domains and learns an Event Edge Space , a shared edge-centric latent space guided by a frozen event teacher, that aligns image, LiDAR, and events before fusion.

Figure 2: Overview of $x^{2}$ -Fusion. Given image, events and LiDAR, we first pretrain an Event Edge Encoder to distill motion-aware edge features. We then freeze the encoder and use its embeddings as edge prototypes to symmetrically regularize channel-wise representations across modalities, aligning them into a shared Event Edge Space (Sec. 3.1 ). Within this space, Reliability-aware Adaptive Fusion estimates global and local reliability and fuses modalities via a cross-attention block to produce 2D/3D features (Sec. 3.2 ). Finally, Cross-dimension Contrast Learning enforces inter-frame coherence and 2D–3D consistency, and the task heads output optical and scene flow (Sec. 3.3 ).

Figure 3: Our proposed event edge encoder pretraining learns explicit, high-fidelity motion-aware edge representations by predicting edge strength from voxelized event streams.

Figure 4: The proposed reliability-aware adaptive fusion module adaptively integrates image, LiDAR, and event features through hierarchical reliability weighting and cross-modal attention.

Modality	Method	#Params(M)	EKubric						DSEC
Modality	Method	#Params(M)	$\text{EPE}_{\text{2D}}$ $\downarrow$	$\text{ACC}_{\text{1px}}$ $\uparrow$	Fl $\downarrow$	$\text{EPE}_{\text{3D}}$ $\downarrow$	$\text{ACC}_{\text{.05}}$ $\uparrow$	$\text{ACC}_{\text{.10}}$ $\uparrow$	$\text{EPE}_{\text{2D}}$ $\downarrow$	$\text{ACC}_{\text{1px}}$ $\uparrow$	Fl $\downarrow$	$\text{EPE}_{\text{3D}}$ $\downarrow$	$\text{ACC}_{\text{.05}}$ $\uparrow$	$\text{ACC}_{\text{.10}}$ $\uparrow$
Img	RAFT [ 29 ]	5.3M	0.838	93.31%	2.36%	-	-	-	0.586	88.98%	1.47%	-	-	-
Img	FlowFormer [ 4 ]	16.2M	0.702	92.58%	2.07%	-	-	-	0.567	89.82%	1.33%	-	-	-
PC (Point Cloud)	PV-RAFT [ 50 ]	239.2K	-	-	-	0.093	82.42%	92.60%	-	-	-	0.190	32.74%	55.62%
EV	E-RAFT [ 51 ]	5.3M	-	-	-	-	-	-	0.481	91.75%	1.31%	-	-	-
Img+Depth	RAFT-3D [ 52 ]	7.7M	0.714	94.39%	-	0.049	92.88%	-	0.572	89.55%	-	0.144	49.30%	-

Table 1: Quantitative evaluation on both standard and degraded scenarios of EKubric and DSEC datasets.

\dagger

Reproduced per original methodology (code unavailable). Extended results are provided in the supplementary material.

Figure 5: Visual comparison of optical flows on EKubric and DSEC dataset. $x^{2}$ -Fusion achieves the state-of-the-art performance across various exposure degradation scenarios with clearer motion boundaries and finer details. Please zoom in for details.

Figure 6: Visual comparison of scene flows on EKubric and DSEC dataset. $x^{2}$ -Fusion demonstrates achieves lower end-point errors across diverse degradation scenarios. Comprehensive results and analyses are provided in the supplementary material. Please zoom in for details.

Settings	$\text{EPE}_{\text{2D}}$ $\downarrow$	$\text{EPE}_{\text{3D}}$ $\downarrow$
w/o Event Edge Space	0.491	0.146
w/o Edge-anchored Regularization	0.393	0.119
w/o Event Edge Encoder	0.378	0.114
w/ Event Edge Space	0.322	0.094

Table 2: Effects of the Event Edge Space.

Figure 7: t-SNE visualization of cross-modal feature alignment. The edge-anchored symmetric regularization reduces tri-model feature divergence through pyramid-level alignment.

Variants	Setting	$\text{EPE}_{\text{2D}}$ $\downarrow$	$\text{EPE}_{\text{3D}}$ $\downarrow$
Model#A	Indep. 2D + 3D	0.404	0.119
Model#B	Joint 2D & 3D	0.386	0.113
Model#C	Joint 2D & 3D + CCL	0.325	0.103

Table 3: Effects of cross-dimension learning and joint task.

Figure 8: Qualitative analysis of joint task performance and contrastive learning effects. (Top) t-SNE visualization of fused 2D-3D feature distributions; (Bottom) Visualization of (left) 2D optical flow and (right) 3D scene flow error maps.

Scene Flow Estimation

See all 21 papers

LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization
Mar 2, 2026Kuangyi Chen, Jun Zhang, Yuxi Hu +2LiDAR Localization Camera-LiDAR Fusion
Institute of Visual Computing, Graz University of Technology, 8010 Graz, Austria · Neuromorphic Automation and Intelligence Lab (NAIL), School of Artificial Intelligence and Robotics, Hunan University, Changsha 410012, China
SF3D-RGB: Scene Flow Estimation from Monocular Camera and Sparse LiDAR
Feb 25, 2026Rajai Alhimdiat, Ramy Battrawy, René Schuster +2Scene Flow Estimation 3D Scene Understanding
Islamic University of Gaza, Gaza, Palestine · German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany
On the Geometry of Learned Representations in Event-Based Multi-Modal Egomotion Estimation
Jul 17, 2026Stefano Silvestrini, Michele CeresoliRepresentation Geometry Event-Based Vision
Politecnico di Milano, Via Giuseppe La Masa, 34, 20156, Milan, Italy

Mar 2, 2026J/K move · Enter open · S save

LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization

Kuangyi Chen, Jun Zhang, Yuxi Hu, Yi Zhou+1

Institute of Visual Computing, Graz University of Technology, 8010 Graz, Austria · Neuromorphic Automation and Intelligence Lab (NAIL), School of Artificial Intelligence and Robotics, Hunan University, Changsha 410012, China

LiDAR Localization Camera-LiDAR Fusion Event-Based Vision Event-Based Optical Flow

Open paper AskPDF arXiv

Event cameras offer high-temporal-resolution sensing that remains reliable under high-speed motion and challenging lighting, making them promising for localization from LiDAR point clouds in GPS-denied and visually degraded environments. However, aligning sparse, asynchronous events with dense LiDAR maps is fundamentally ill-posed, as direct correspondence estimation suffers from modality gaps. We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event-depth flow fields to bridge the sensing-modality divide. Instead of treating edges as a post-hoc aid, LEAR couples them with flow estimation through a cross-modal fusion mechanism that injects modality-invariant geometric cues into the motion representation, and an iterative refinement strategy that enforces mutual consistency between the two tasks over multiple update steps. This synergy produces edge-aware, depth-aligned flow fields that enable more robust and accurate pose recovery via Perspective-n-Point (PnP) solvers. On several popular and challenging datasets, LEAR achieves superior performance over the best prior method. The source code, trained models, and demo videos are made publicly available online.

Abstract

Scene Flow Estimation

Explore similar work

LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization

SF3D-RGB: Scene Flow Estimation from Monocular Camera and Sparse LiDAR

On the Geometry of Learned Representations in Event-Based Multi-Modal Egomotion Estimation