OCA: ODE-Driven Cross-Attention for Image-to-Point-Cloud Registration

Pei An; Jiaqi Yang; Yulong Wang; Siwen Quan; Liangliang Nan

cs.CVSep 29, 2026

OCA: ODE-Driven Cross-Attention for Image-to-Point-Cloud Registration

Authors: Pei An, Jiaqi Yang, Yulong Wang, Siwen Quan, Liangliang Nan

Organizations: Huazhong University of Science and Technology, China · Northwestern Polytechnical University, China · Huazhong Agricultural University, China · Chang’an University, China · Delft University of Technology, Netherlands

Abstract

Cross-attention is a crucial component in learning-based image-to-point-cloud (I2P) registration. Although existing cross-attention mechanisms have achieved promising progress, attention ambiguity remains a fundamental challenge that hinders the learning of discriminative 2D-3D correspondences. To address this problem, we revisit cross-attention and establish ordinary differential equations (ODEs) to model the ideal I2P feature interaction. Based on this formulation, we develop an ODE-driven cross-attention (OCA) module that refines feature representations and attention matrices through ODEs. In practice, OCA can be seamlessly integrated into existing I2P registration frameworks. To validate its effectiveness, we incorporate OCA into five state-of-the-art baselines and evaluate on four public benchmark datasets. Experimental results demonstrate that OCA improves registration recall by up to 5%, 9%, and 15% under the standard, fine-tuning, and zero-shot settings, respectively.

Figures & tables

Figure 1 : Motivation of ODE-driven cross-attention (OCA). Although existing cross-attention models have made progress, attention ambiguity caused by the modality gap remains inevitable. To overcome this problem, we propose OCA to enhance I2P feature representations by propagating features and the attention matrix through ODEs. IR denotes the inlier ratio. Green and red lines represent inliers and outliers, respectively.

Figure 2 : Overview of Sec. 3 and 4 . OCA is designed to alleviate attention ambiguity by approximating the ideal feature interaction formulated in Eq. 1 .

Figure 3 : Overview of OCA. (a) From the convergence condition (C1), we discover that solutions in Eqs. 7 , 8 , and 9 can guide the assignment ODEs to rapidly approach the ground-truth assignment matrix. After that, we propose OCA to unify the solutions in Eqs. 7 , 8 , and 9 . (b) The overall pipeline of OCA. For computation efficiency, Eqs. 10 , 11 , and 12 are adopted to approximate Eqs. 8 and 9 .

Methods	IR	RR
Matr [ 17 ]	0.453	0.472
Matr [ 17 ] + Ref	0.475	0.501
Matr [ 17 ] +OCA	0.501	0.552 +5%
Flow-I2P [ 2 ]	0.469	0.511
Flow-I2P [ 2 ] + Ref	0.459	0.562
Flow-I2P [ 2 ] +OCA	0.530	0.596 +3%

Table 1 : Standard evaluation of the proposed OCA module with five baselines on the 7-Scene dataset [ 11 ] .

Methods	IR	RR
Matr [ 17 ]	0.453	0.472
Matr [ 17 ] + Ref	0.475	0.501
Matr [ 17 ] +OCA	0.501	0.552 +5%
Flow-I2P [ 2 ]	0.469	0.511
Flow-I2P [ 2 ] + Ref	0.459	0.562
Flow-I2P [ 2 ] +OCA	0.530	0.596 +3%

Table 1 : Standard evaluation of the proposed OCA module with five baselines on the 7-Scene dataset [ 11 ] .

Methods	IR	RR
Matr [ 17 ]	0.341	0.457
Matr [ 17 ] + Ref	0.372	0.498
Matr [ 17 ] +OCA	0.375	0.537 +4%
Flow-I2P [ 2 ]	0.361	0.547
Flow-I2P [ 2 ] + Ref	0.379	0.573
Flow-I2P [ 2 ] +OCA	0.387	0.580 +1%

Table 2 : Fine-tuning evaluation of the proposed OCA module with five baselines on the RGBD-v2 dataset [ 15 ] .

Methods	IR	RR
Matr [ 17 ]	0.568	0.472
Matr [ 17 ] + Ref	0.629	0.647
Matr [ 17 ] +OCA	0.703	0.705 +8%
Flow-I2P [ 2 ]	0.666	0.677
Flow-I2P [ 2 ] + Ref	0.703	0.732
Flow-I2P [ 2 ] +OCA	0.737	0.775 +4%

Table 3 : Fine-tuning evaluation of the proposed OCA module with five baselines on the TUM dataset [ 27 ] .

Methods	IR	RR
Matr [ 17 ]	0.568	0.472
Matr [ 17 ] + Ref	0.629	0.647
Matr [ 17 ] +OCA	0.703	0.705 +8%
Flow-I2P [ 2 ]	0.666	0.677
Flow-I2P [ 2 ] + Ref	0.703	0.732
Flow-I2P [ 2 ] +OCA	0.737	0.775 +4%

Table 3 : Fine-tuning evaluation of the proposed OCA module with five baselines on the TUM dataset [ 27 ] .

Methods	IR	RR
Matr [ 17 ]	0.342	0.664
Matr [ 17 ] + Ref	0.376	0.691
Matr [ 17 ] +OCA	0.387	0.711 +2%
Flow-I2P [ 2 ]	0.386	0.763
Flow-I2P [ 2 ] + Ref	0.391	0.770
Flow-I2P [ 2 ] +OCA	0.442	0.789 +2%

Table 4 : Fine-tuning evaluation of the proposed OCA module with five baselines on the ScanNet dataset [ 9 ] .

Matr [ 17 ]	+ Ref	+OCA	+ Zero -OCA	Flow-I2P [ 17 ]	+ Ref	+OCA	+ Zero -OCA
IR	0.259	0.253	0.215	IR	0.310	0.287	0.210
RR	0.184	0.217 +3%	0.276 +9%	RR	0.263	0.342 +8%	0.414 +15%
Bridge [ 7 ]	+ Ref	+OCA	+ Zero -OCA	LDF-I2P [ 23 ]	+ Ref	+OCA	+ Zero -OCA
IR	0.330	0.307	0.295	IR	0.304	0.293	0.210
RR	0.224	0.342 +12%	0.316 +9%	IR	0.309	0.349 +4%	0.428 +12%

Table 5 : Zero-shot registration evaluation of the proposed OCA module with representative baselines on the ScanNet dataset [ 9 ] .

Figure 4 : Qualitative comparisons of Matr [ 17 ] and Matr+OCA on the 7-Scenes dataset [ 11 ] . Across various scenes with different overlap ratios, the proposed OCA module increases the number of inliers.

Figure 5 : Qualitative comparison on the TUM dataset [ 27 ] under different viewpoints with consecutive timestamps. (a) CA-I2P [ 8 ] and (b) CA-I2P+OCA. With the integration of OCA, the I2P registration results become more stable and accurate.

Figure 6 : Failure case of the proposed OCA module in extremely texture-less scenes. In the second column, the IR of Matr+OCA decreases to $7.6\%$ .

Figure 7 : Visualization of $\mathbf{Y}_{\text{norm}}[k]$ . As $\mathbf{Y}_{\text{norm}}[k]=\mathbf{y}_{\text{norm}}[k]\cdot\mathbf{y}_{\text{norm}}[k]^{T}$ , a sparse $\mathbf{Y}_{\text{norm}}[k]$ indicates that the point feature is less redundant with other points, suggesting greater feature discriminability. During ODE propagation, $\mathbf{Y}_{\text{norm}}[k]$ tends to become sparser, which empirically supports the effectiveness of the proposed OCA module.

Methods	Matr+OCA	Matr+ Simple DDPM [ 13 ]	Matr+ Simple FM [ 19 ]	Diff-Reg [ 32 ]
IR	0.703	0.465	0.448	0.623
RR	0.705	0.460	0.459	0.602

Table 6 : Comparisons with existing propagation-based methods on the TUM dataset [ 27 ] .

$T$	1	2	3	4	5
IR	0.571	0.554	0.501	0.433	0.415
RR	0.489	0.510	0.552	0.575	0.539

Table 7 : Ablation study on the iteration number

T

on the 7-Scenes dataset [ 11 ] .

$T$	1	2	3	4	5
IR	0.571	0.554	0.501	0.433	0.415
RR	0.489	0.510	0.552	0.575	0.539

Table 7 : Ablation study on the iteration number

T

on the 7-Scenes dataset [ 11 ] .

$\tau$	0.05	0.10	0.15	0.20	0.25
IR	0.562	0.501	0.443	0.400	0.379
RR	0.495	0.552	0.541	0.497	0.494

Table 8 : Ablation study on the time step

\tau

on the 7-Scenes dataset [ 11 ] .

$\gamma$	1	2	4	8
IR	0.531	0.501	0.472	0.435
RR	0.527	0.552	0.561	0.532

Table 9 : Parameter selection of the

\gamma

$\gamma$	1	2	4	8
IR	0.531	0.501	0.472	0.435
RR	0.527	0.552	0.561	0.532

Table 9 : Parameter selection of the

\gamma

$\omega$	0.05	0.10	0.15	0.20
IR	0.492	0.501	0.527	0.511
RR	0.531	0.552	0.542	0.525

Table 10 : Parameter selection of the

\omega

	Time
Matr [ 17 ]	0.132
Matr+OCA	0.138

Table 11 : Runtime analysis (seconds).

Figure 8 : Visualization of input data (RGB image and 3D point cloud) across different scenes. Three key challenges for real-world I2P registration are: (1) small overlap between the image and point cloud, (2) noisy point clouds, and (3) low-texture scenes.

Figure 9 : Additional qualitative comparisons of the proposed OCA module integrated with five representative baselines: Matr [ 17 ] , Flow-I2P [ 2 ] , Bridge [ 7 ] , LDF-I2P [ 23 ] , and CA-I2P [ 8 ] . Results are shown for (a) 7-Scenes [ 11 ] , (b) ScanNet [ 9 ] , and (c) RGBD-v2 [ 15 ] datasets. Our ODE-based cross-attention module significantly increases the number of inliers across diverse scenarios.

Figure 10 : I2P registration for augmented reality (AR) applications. With the OCA layer, virtual objects are rendered with high spatiotemporal consistency with the real world across continuous timestamps.

Methods	Matr [ 17 ]	Matr [ 17 ] +OCA	Top-I2P [ 1 ]	Top-I2P [ 1 ] +OCA
IR	0.568	0.703	0.683	0.742
RR	0.472	0.705	0.732	0.764

Table 12 : Comparisons with the state-of-the-art prior-knowledge-based registration method [ 1 ] on the TUM dataset [ 27 ] .

Inlier Ratio $\uparrow$
Number of Sampled Points	5000	2500	1000	500	250
GeoTransformer [ 24 ]	71.9%	75.2%	76.0%	82.2%	85.1%
GeoTransformer+OCA (Ours)	73.2 %	75.8 %	77.1 %	82.7 %	87.3 %
Registration Recall $\uparrow$
GeoTransformer [ 24 ]	92.0%	91.8%	91.8%	91.4%	91.2%
GeoTransformer+OCA (Ours)	92.4 %	92.3 %	92.6 %	91.7 %	92.2 %

Table 13 : Verification of the proposed OCA layer in 3D point cloud registration task on the 3DMatch dataset [ 35 ] .

Explore similar work

Cross-Coordinate Correspondence Pruning for Image-to-Point Cloud Registration
Jul 19, 2026Xin Liu, Rong Qin, Huipeng Lin +5Point Cloud Registration Point Clouds
VCIP & TMCC & DISSec, College of Computer Science, Nankai University, Tianjin, China. · AutoCity (Shenzhen) Autonomous Driving Co., Ltd, Shenzhen, China. · School of Intelligent Science and Technology, University of Science and Technology Beijing, Beijing, China. +4
Angle-I2P: Angle-Consistent-Aware Hierarchical Attention for Cross-Modality Outlier Rejection
May 6, 2026Muyao Peng, Shun Zou, Pei An +2Point Cloud Registration Point Clouds
School of Electronic Information and Communications, Huazhong University of Science and Technology
Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration
Sep 16, 2026Zhixin Cheng, Jiacheng Deng, Xiaotian Yin +3Point Cloud Registration Point Clouds
School of Computer Science and Technology, Hefei University of Technology, Hefei 230601, China · School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China · Institute of Advanced Technology, University of Science and Technology of China, Hefei 230027, China

Jul 19, 2026, cs.CVJ/K move · Enter open · S save

Cross-Coordinate Correspondence Pruning for Image-to-Point Cloud Registration

Xin Liu, Rong Qin, Huipeng Lin, Leizhi Shu+4

VCIP & TMCC & DISSec, College of Computer Science, Nankai University, Tianjin, China. · AutoCity (Shenzhen) Autonomous Driving Co., Ltd, Shenzhen, China. · School of Intelligent Science and Technology, University of Science and Technology Beijing, Beijing, China.+4

Point Cloud Registration Point Clouds Ground-Truth Correspondence Outliers Cloud Ensemble

Open paper AskPDF arXiv

Recent detection-free approaches have shown significant efficacy in image-to-point cloud (I2P) registration by employing a coarse-to-fine matching pipeline. In the coarse stage, down-sampled image features and voxelized point cloud features are typically fused to establish initial coarse correspondences for subsequent refinement. However, existing methods largely overlook the critical role of point cloud density, which fundamentally dictates the quality of coarse correspondences and the final registration results. Specifically, excessively sparse point clouds lead to an insufficient number of inliers, while overly dense ones often introduce a high outlier ratio. Consequently, this creates an inherent density trade-off, thereby significantly limiting the registration accuracy of current approaches. For mitigating this trade-off, we propose a novel Cross-Coordinate Correspondences Pruning (CCP) strategy to acquire sufficient inliers while ensuring a low outlier ratio. To minimize interference from inter-modal coordinate discrepancies, we first project cross-coordinate coarse correspondences to the 2D image coordinate system for spatial unification. Subsequently, a lightweight pruning network is responsible for predicting the inlier confidences, which are used to filter coarse outliers, from coordinate geometric and modal feature dimensions. To maximize inlier recall, we further design a Multi-Density Point Ensemble (MDPE) strategy that consolidates and deduplicates pruned coarse correspondences across varying point cloud densities. Our method achieves a significant performance improvement, surpassing existing state-of-the-art methods by at least 8.6% in Registration Recall across various benchmarks.

Abstract

Explore similar work

Cross-Coordinate Correspondence Pruning for Image-to-Point Cloud Registration

Angle-I2P: Angle-Consistent-Aware Hierarchical Attention for Cross-Modality Outlier Rejection

Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration