Organizations: School of Software, Xi’an Jiaotong University, No. 28 Xianning West Road, China · CEPRI, China Electric Power Research Institute No. 15 Xiaoying East Road, Qinghe, Haidian District, Beijing, China
Robust point cloud registration is a fundamental task in 3D computer vision and geometric deep learning, essential for applications such as large-scale 3D reconstruction, augmented reality, and scene understanding. However, the performance of established learning-based methods often degrades in complex, real world scenarios characterized by incomplete data, sensor noise, and low overlap regions. To address these limitations, we propose CMHANet, a novel Cross-Modal Hybrid Attention Network. Our method integrates the fusion of rich contextual information from 2D images with the geometric detail of 3D point clouds, yielding a comprehensive and resilient feature representation. Furthermore, we introduce an innovative optimization function based on contrastive learning, which enforces geometric consistency and significantly improves the model's robustness to noise and partial observations. We evaluated CMHANet on the 3DMatch and the challenging 3DLoMatch datasets. \rev{Additionally, zero-shot evaluations on the TUM RGB-D SLAM dataset verify the model's generalization capability to unseen domains.} The experimental results demonstrate that our method achieves substantial improvements in both registration accuracy and overall robustness, outperforming current techniques. We also release our code in https://github.com/DongXu-Zhang/CMHANet.
Figures & tables
Figure 1: The pixel-to-point correspondence between the point cloud and the image has been established by external parameter calibration, so this method is reasonable, and can effectively combine the global texture features of the image with the local geometric characteristics of the point cloud.
Figure 2: The Registration Recall (RR) is plotted on the x-axis for 3DMatch and on the y-axis for 3DLoMatch. CMHANet stands out by consistently achieving the highest RR.
Figure 3: The workflow overview of CMHANet. Our method processes raw point clouds and images through feature extraction, employs a hybrid-attention mechanism for superpoint matching, refines to dense point correspondences, and finally computes the rigid transformation for alignment.
Figure 4: Details of the superpoint matching module. This module employs a multi-stage attention mechanism: Self-Attention refines individual modality features, followed by Aggregation-Attention for structured intra-modal feature integration, and finally Cross-Attention to fuse features across modalities, ultimately generating a similarity matrix for robust superpoint matching. For visualization, the blue blocks represent the source point cloud features ( F^P ), the orange blocks represent the target point cloud features ( F^Q ), and the yellow lines indicate the attention interactions.
Figure 5: Computation graph of the Hybrid Attention mechanism. In the diagrams, the label F denotes feature representations ( e.g.,F^PC for generic point cloud features, F^P and F^Q for source and target point cloud features (in a, c), F^I for image features), and E represents geometric positional embeddings. The three modules serve distinct roles. (a) Geometric Self-Attention captures global structural relationships within a single point cloud; (b) Geometric Aggregation-Attention fuses 2D visual context from images into 3D geometric features; and (c) Geometric Cross-Attention establishes consistency and searches for correspondences between the source and target point clouds.
3DMatch
3DLoMatch
# Samples
5000
2500
1000
500
250
5000
2500
1000
500
250
Feature Matching Recall (%) ↑
PerfectMatch [ 18 ]
95.0
94.3
92.9
90.1
82.9
63.6
61.7
53.6
45.2
34.2
FCGF [ 13 ]
97.4
97.3
97.0
96.7
96.6
76.6
75.4
74.2
71.7
67.3
D3Feat [ 5 ]
95.6
94.3
95.6
94.3
91.0
66.7
64.7
61.6
60.0
56.6
SpinNet [ 3 ]
97.6
97.2
96.8
95.5
96.5
75.3
74.9
72.5
70.0
63.6
Table 1: Evaluation results of our method on 3DMatch and 3DLoMatch datasets. The results are reported for two key metrics: Feature Matching Recall (%) and Registration Recall (%). The evaluation is conducted across varying sample sizes (5000, 2500, 1000, 500 and 250) to assess the robustness of each method under different constraints.
Model
Estimator
3DMatch
3DLoMatch
RRE (°)
RTE (m)
RRE (°)
RTE (m)
Predator [ 25 ]
RANSAC- 50k
2.029
0.064
3.048
0.093
CoFiNet [ 60 ]
RANSAC- 50k
2.002
0.064
3.271
0.090
PCR-CG [ 70 ]
RANSAC- 50k
1.993
0.061
3.002
0.087
MAC [ 69 ]
RANSAC-free
2.003
0.065
3.010
0.085
Hybrid [ 27 ]
RANSAC-free
1.992
0.061
3.002
0.087
Table 2: RRE and RTE on 3DMatch (3DM) and 3DLoMatch (3DLM) benchmarks. We report Relative Rotation Error (RRE, °) and Relative Translation Error (RTE, m) for various methods using RANSAC-50k or RANSAC-free estimators. CMHANet achieves the lowest RRE and RTE on both datasets.
Model
3DMatch
3DLoMatch
FMR
RR
FMR
RR
IMFNet [ 26 ]
98.5
91.0
80.5
48.4
PCR-CG [ 70 ]
97.4
89.4
80.4
66.3
CMHANet ( ours )
98.6
92.4
87.7
75.5
Table 3: Comparison With Multimodal Point Cloud Registration
Model
Estimator
# Samples
RR (%)
Times (s)
3DM
3DLM
Model
Pose
Total
FCGF [ 13 ]
RANSAC-50k
5000
85.1
40.1
0.052
3.326
3.378
D3Feat [ 5 ]
RANSAC-50k
5000
81.6
37.2
0.024
3.088
3.112
SpinNet [ 3 ]
RANSAC-50k
5000
88.6
59.8
0.248
0.388
60.636
Predator [ 25 ]
RANSAC-50k
5000
89.0
59.8
0.032
5.120
5.152
CoFiNet [ 60 ]
RANSAC-50k
5000
89.3
67.5
0.115
1.807
1.922
Table 4: Comparison of Model Registration Performance with Different Estimators and Sample sizes On 3DMatch(3DM) and 3DLomatch(3DLM). We report Registration Recall (RR, %) and computation times (Model, Pose, and Total, in seconds) for various methods. Evaluations are conducted with RANSAC-50k, Weighted SVD, and LGR to assess robustness and efficiency. CMHANet consistently achieves superior performance in both metrics.
Figure 6: The result of CMHANet. We visualized representative scenes from the 3DMatch dataset. The first row shows the initial arrangement of the two point clouds in their raw, unaligned positions. The middle row depicts the ground truth alignment of the two point clouds, representing the ideal registration outcome. The bottom row showcases the registration results predicted by CMHANet, offering a direct comparison to both the initial misalignment and the ground truth. We also report the Root Mean Square Error (RMSE) values for each scene, quantifying the spatial deviation between CMHANet’s predicted alignments and the ground truth.
Method
xyz
360
teddy
desk
plant
dishes
coke
flower
mean
ICP [ 6 ]
2.1
5.1
2.1
2.3
1.6
3.7
3.1
2.7
2.8
AA-ICP [ 34 ]
2.1
5.1
2.1
2.3
1.6
3.7
3.1
2.7
2.8
Sparse ICP [ 7 ]
1.6
4.8
1.8
1.8
0.88
3.9
3.3
3.2
2.66
Robust ICP [ 68 ]
0.5
2.2
1
1.2
0.65
3.2
2.4
2.4
1.69
Teaser++ [ 57 ]
3.4
20
11
8
6.1
21
23
20
14.06
DCP [ 51 ]
6.5
10
6.6
7
5.6
7.4
7.5
6.3
7.11
Table 5: Registration Results on Eight Sequences from the TUM RGB-D SLAM Dataset. We report the Average RMSE ( ×10−2 ). Our method (CMHANet) is trained on 3DMatch and directly tested on TUM without fine-tuning.
Module
3DMatch
3DLoMatch
Loss
HA
IM
PIR
FMR
IR
RR
PIR
FMR
IR
RR
✓
-
-
83.8
97.4
68.2
89.9
50.9
86.0
41.2
71.9
-
-
✓
84.2
97.8
65.9
90.1
51.2
86.3
40.8
72.1
-
✓
✓
85.1
98.5
69.2
91.4
51.7
86.8
42.9
74.2
✓
-
✓
84.0
98.2
68.8
90.5
52.2
86.5
41.8
72.4
✓
✓
✓
86.8
98.6
70.7
92.4
54.8
87.7
43.1
75.5
Table 6: Ablation experiments of the CMHANet. The results are measured in %. We also assess the impact of including ( ✓ ) or excluding (-) our custom Loss function, Hybrid Attention (HA) module, and Image Module (IM) on PIR, FMR, IR, and RR metrics. Best performance is highlighted in bold.
Model
3DMatch
3DLoMatch
PIR
FMR
IR
RR
PIR
FMR
IR
RR
w/o IM Module
83.8
97.4
68.2
89.9
50.9
86.0
41.2
71.9
Resnet-34
84.9
97.8
68.7
90.9
51.2
86.3
41.8
73.1
Resnet-101
87.0
98.5
71.2
92.4
54.7
87.2
42.9
75.2
CMHANet
86.8
98.6
70.7
92.4
54.8
87.7
43.1
75.5
Table 7: Ablation Experiments on Image Module in CMHANet. Best performance is highlighted in bold. The second-best performance results are indicated by underlines.
Model
3DMatch
3DLoMatch
PIR
FMR
IR
RR
PIR
FMR
IR
RR
w/o HA Module
84.0
98.2
68.8
90.5
52.2
86.5
41.8
72.4
w/o Aggre Att
85.9
98.3
70.2
91.3
53.7
86.7
42.8
73.6
CMHANet
86.8
98.6
70.7
92.4
54.8
87.7
43.1
75.5
Table 8: Ablation Experiments on Hybrid Attention Module in CMHANet. Best performance is highlighted in bold.
Figure 7: Visual ablation comparison demonstrating the impact of different feature transformation modules on registration performance. From left to right, w/o IM: Without the Image Module, the network relies solely on geometry; w/o HA: Removing the Hybrid Attention module hinders effective global feature interaction; CMHANet: Our full model leverages cross-modal feature transformation; Ground Truth: Shown for reference.
Point cloud registration (PCR) is a fundamental task in 3D vision and provides essential support for applications such as autonomous driving, robotics, and environmental modeling. Despite its widespread use, existing methods often fail when facing real-world challenges like heavy noise, significant occlusions, and large-scale transformations. These limitations frequently result in compromised registration accuracy and insufficient robustness in complex environments. In this paper, we propose IGASA as a novel registration framework constructed upon a Hierarchical Pyramid Architecture (HPA) designed for robust multi-scale feature extraction and fusion. The framework integrates two pivotal components consisting of the Hierarchical Cross-Layer Attention (HCLA) module and the Iterative Geometry-Aware Refinement (IGAR) module. The HCLA module utilizes skip attention mechanisms to align multi-resolution features and enhance local geometric consistency. Simultaneously, the IGAR module is designed for the fine matching phase by leveraging reliable correspondences established during coarse matching. This synergistic integration within the architecture allows IGASA to adapt effectively to diverse point cloud structures and intricate transformations. We evaluate the performance of IGASA on four widely recognized benchmark datasets including 3D(Lo)Match, KITTI, and nuScenes. Our extensive experiments consistently demonstrate that IGASA significantly surpasses state-of-the-art methods and achieves notable improvements in registration accuracy. This work provides a robust foundation for advancing point cloud registration techniques while offering valuable insights for practical 3D vision applications. The code for IGASA is available in https://github.com/DongXu-Zhang/IGASA.
We introduce C-GenReg, a training-free framework for 3D point cloud registration that leverages the complementary strengths of world-scale generative priors and registration-oriented Vision Foundation Models (VFMs). Current learning-based 3D point cloud registration methods struggle to generalize across sensing modalities, sampling differences, and environments. Hence, C-GenReg augments the geometric point cloud registration branch by transferring the matching problem into an auxiliary image domain, where VFMs excel, using a World Foundation Model to synthesize multi-view-consistent RGB representations from the input geometry. This generative transfer, preserves spatial coherence across source and target views without any fine-tuning. From these generated views, a VFM pretrained for finding dense correspondences extracts matches. The resulting pixel correspondences are lifted back to 3D via the original depth maps. To further enhance robustness, we introduce a "Match-then-Fuse" probabilistic cold-fusion scheme that combines two independent correspondence posteriors, that of the generated-RGB branch with that of the raw geometric branch. This principled fusion preserves each modality inductive bias and provides calibrated confidence without any additional learning. C-GenReg is zero-shot and plug-and-play: all modules are pretrained and operate without fine-tuning. Extensive experiments on indoor (3DMatch, ScanNet) and outdoor (Waymo) benchmarks demonstrate strong zero-shot performance and superior cross-domain generalization. For the first time, we demonstrate a generative registration framework that operates successfully on real outdoor LiDAR data, where no imagery data is available.
Image-to-point cloud registration methods typically follow a coarse-to-fine pipeline, extracting patch-level correspondences and refining them into dense pixel-to-point matches. However, in scenes with repetitive patterns, images often lack sufficient 3D structural cues and alignment with point clouds, leading to incorrect matches. Moreover, prior methods usually overlook structural consistency, limiting the full exploitation of correspondences. To address these issues, we propose two novel modules: the Local Geometry Enhancement (LGE) module and the Graph Distribution Consistency (GDC) module. LGE enhances both image and point cloud features with normal vectors, injecting geometric structure into image features to reduce mismatches. GDC constructs a graph from matched points to update features and explicitly constrain similarity distributions. Extensive experiments and ablations on two benchmarks, RGB-D Scenes v2 and 7-Scenes, demonstrate that our approach achieves state-of-the-art performance in image-to-point cloud registration.
Zhixin Cheng, Jiacheng Deng, Xinjun Li +5
School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China · Meituan Inc, Beijing 100000, China · Institute of Advanced Technology, University of Science and Technology of China, Hefei 230027, China