RegFormer++: An Efficient Large-Scale 3D LiDAR Point Registration Network with Projection-Aware 2D Transformer

Jiuming Liu; Guangming Wang; Zhe Liu; Chaokang Jiang; Haoang Li; Mengmeng Liu; Tianchen Deng; Marc Pollefeys

cs.CVMar 15, 2026

RegFormer++: An Efficient Large-Scale 3D LiDAR Point Registration Network with Projection-Aware 2D Transformer

Authors: Jiuming Liu, Guangming Wang, Zhe Liu, Chaokang Jiang, Haoang Li, Mengmeng Liu, Tianchen Deng, Marc Pollefeys,

Organizations: School of Automation and Intelligent Sensing, State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis, Shanghai Jiao Tong University, Shanghai 200240, China. · Department of Engineering, University of Cambridge, UK. · China University of Mining and Technology. · Thrust of Robotics and Autonomous Systems and the Thrust of Intelligent Transportation, Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China. ·

Abstract

Although point cloud registration has achieved remarkable advances in object-level and indoor scenes, large-scale LiDAR registration methods has been rarely explored before. Challenges mainly arise from the huge point scale, complex point distribution, and numerous outliers within outdoor LiDAR scans. In addition, most existing registration works generally adopt a two-stage paradigm: They first find correspondences by extracting discriminative local descriptors and then leverage robust estimators (e.g. RANSAC) to filter outliers, which are highly dependent on well-designed descriptors and post-processing choices. To address these problems, we propose a novel end-to-end differential transformer network, termed RegFormer++, for large-scale point cloud alignment without requiring any further post-processing. Specifically, a hierarchical projection-aware 2D transformer with linear complexity is proposed to project raw LiDAR points onto a cylindrical surface and extract global point features, which can improve resilience to outliers due to long-range dependencies. Because we fill original 3D coordinates into 2D projected positions, our designed transformer can benefit from both high efficiency in 2D processing and accuracy from 3D geometric information. Furthermore, to effectively reduce wrong point matching, a Bijective Association Transformer (BAT) is designed, combining both cross attention and all-to-all point gathering. To improve training stability and robustness, a feature-transformed optimal transport module is also designed for regressing the final pose transformation. Extensive experiments on KITTI, NuScenes, and Argoverse datasets demonstrate that our model achieves state-of-the-art performance in terms of both accuracy and efficiency.

Figures & tables

Figure 1: Comparison with previous point cloud registration works. Previous methods (A) extract local descriptors and establish explicit correspondences, while our RegFormer++ rely on globally-aware features and learn cross-frame association by cross attention requiring no correspondences. Furthermore, prior literature resorts to the RANSAC [ 24 ] algorithm for outlier filtering. In contrast, our method leverage attention mechanism to softly remove outliers.

Figure 2: The overall architecture of RegFormer++. We first project point clouds onto a 2D surface and feed its patches into three cascaded feature extraction transformers. For the cross-frame association, we design a Bijective Association Transformer module which includes a cross attention module and an all-to-all point gathering. Finally, transformations are refined iteratively.

Figure 3: Cylindrical projection. We project 3D point clouds onto a 2D surface and fill each pixel with its raw $x,y,z$ coordinates. A projection mask is also proposed to remove invalid positions.

Figure 4: Visualization of kernel-based efficient patch embedding. For each center point, its surrounding neighbor points within a predefined 2D kernel are indexed and grouped. Then, outliers that are harmful to registration are filtered. Above grouping and filtering in 2D space actually happen in 3D space based on the cylindrical projection process.

Figure 5: All-to-all point gathering. After the cross-attention mechanism leveraged for preliminary information exchange between two frames, geometric characteristics of conditioned features ${\tilde{F}}^{S}_{3},{\tilde{F}}^{T}_{3}$ are fully considered to generate the initial motion embeddings. The initial embeddings include coordinate information, point and neighbor similarity features, and content features.

Module	Layer type	$N$	Sample rate	Channels	MLP width	Heads	Dropout
	Point Swin Transformer (stage 1)	2	$[H/4\times W/8]$	16	[16,64,16]	2	0.1
	Point Swin Transformer (stage 2)	2	$[H/8\times W/16]$	32	[32,128,32]	4	0.1
Feature Extraction Transformer	Point Swin Transformer (stage 3)	6	$[H/16\times W/32]$	64	[64,256,64]	8	0.1
	self-attention layer	6	—	64	[64,256,64]	8	0.1
Bijective Association Transformer	cross-attention layer	6	—	64	[64,256,64]	8	0.1

Table 1: Detailed network settings in transformer layers.

N

denotes the number of attention blocks. MLP width indicates the expansion layer of MLP in feed-forward networks.

Module	Layer type		$K$	Sample rate	MLP width
	All-to-all point gathering		112	1	[128,64,64], [128,64]
	Shared MLP for $ME_{3}$		—	1	[128,64]
	$FC_{1}$ for $q_{3}$ , $FC_{2}$ for $t_{3}$		—	1	[4], [3]
Estimation of Initial Transformation		Attentive cost volume	4, 6	1	[128,64,64], [128,64]
		Set upconv	8	$2\times 2$	[128,64], [64]
		Shared MLP for $ME_{2}$	—	1	[128,64]

Table 2: Detailed network parameters in the initial pose estimation and iterative refinement layers.

K

points are selected in the

K

Nearest Neighbors (KNN) of the all-to-all point gathering layer, set upconv layer, and attentive cost volume layer. MLP width means the number of output channels for each layer of MLP.

	RTE(m)		RRE(°)
Method	AVG	STD	AVG	STD	RR(%)	Time(ms)/Points	NT(ms)
FGR [ 112 ]	0.93	0.59	0.96	0.81	39.4%	506.1/11445	44.22
RANSAC [ 24 ]	0.13	0.07	0.54	0.40	91.9%	549.6/16384	33.54
DCP [ 88 ]	1.03	0.51	2.07	1.19	47.3%	46.4/1024	45.31
IDAM [ 37 ]	0.66	0.48	1.06	0.94	70.9%	33.4/4096	8.15
FMR [ 32 ]	0.66	0.42	1.49	0.85	90.6%	85.5/12000	7.13

Table 3: Comparison results with state-of-the-arts on the KITTI dataset. Input point pairs are 10-frame away prepared by [ 57 ] . The best performance is highlighted in bold. Registration Recall (RR) is defined as the success ratio where RRE

<

5°and RTE

<

2m. ‘NT’ means normalized time per thousand points.

Method	Estimator	RTE(cm)	RRE(°)	RR(%)	Time(s)
3DFeat-Net [ 95 ]	RANSAC	25.9	0.25	96.0%	3.4
FCGF [ 18 ]	RANSAC	9.5	0.30	96.6%	3.4
D3Feat [ 7 ]	RANSAC	7.2	0.30	99.8%	3.1
Predator [ 30 ]	RANSAC	6.8	0.27	99.8%	5.2
CoFiNet [ 99 ]	RANSAC	8.5	0.41	99.8%	1.9
SpinNet [ 4 ]	RANSAC	9.9	0.47	99.1%	60.6

Table 4: Comparison results with methods using traditional estimators. Methods are classified into two main categories: RANSAC-based and Graph-based. † indicates methods using optimization algorithm. Other methods including ours are learning-based. Input point pairs are at least 10m away as in [ 71 ] The best performances are respectively highlighted in bold. RR is defined as the success ratio where RRE

<

5°and RTE

<

2m.

Figure 6: Registration recall with different RRE and RTE thresholds on the KITTI dataset.

Method	RTE(m)	RRE(°)	Recall(%)	Time/Points	NT(ms)
FGR [ 112 ]	0.71	1.01	32.2%	284.6/11445	24.87
RANSAC [ 24 ]	0.21	0.74	60.9%	268.2/8192	32.74
DCP [ 88 ]	1.09	2.07	58.6%	45.5/1024	44.43
IDAM [ 37 ]	0.47	0.79	88.0%	32.6/4096	7.96
FMR [ 32 ]	0.60	1.61	92.1%	61.1/12000	5.09
DGR [ 17 ]	0.21	0.48	98.4%	523.0/8192	63.84

Table 5: Comparison results with state-of-the-arts on the NuScenes dataset. Input point pairs are 10-frame away prepared by [ 57 ] . The best performance is highlighted in bold. Registration Recall (RR) is defined as the success ratio where RRE

<

5°and RTE

<

2m. ‘NT’ means normalized time per thousand points.

Figure 7: Registration results on KITTI and Nuscenes datasets. Point clouds colored red and green indicate the input source and target point clouds. Transformed source points by the estimated pose of HRegNet and ours are colored pink and blue respectively. The ground truth is colored purple. Our RegFormer++ can align low-overlap input points even with large translations (b) and (d) or large rotations (a) and (c).

Figure 8: Visualization of registration errors. Point clouds colored yellow and blue indicate transformed source points by the ground truth (GT) and our estimated pose. Registration errors are visualized by a red vector pointing from estimated points to the GT ones.

	RTE(m)		RRE(°)
Method	AVG	STD	AVG	STD	RR(%)
HRegNet [ 57 ]	1.12	0.53	1.04	1.12	25.7%
RegFormer++	0.01	0.01	0.11	0.27	85.5%

Table 6: Comparison results on the Argoverse dataset [ 12 ] . We follow the standard dataset settings to evaluate the performance and train both HRegNet and our method from scratch.

	Gazebo		Wood
Method	Summer	Winter	Autumn	Summer	Avg.
FPFH [ 74 ]	38.6%	14.2%	14.8%	20.8%	22.1%
SHOT [ 80 ]	73.9%	45.7%	60.9%	64.0%	61.1%
3DMatch [ 102 ]	22.8%	8.3%	13.9%	22.4%	16.9%
CGF [ 35 ]	37.5%	13.8%	10.4%	19.2%	20.2%
PerfectMatch [ 29 ]	91.3%	84.1%	67.8%	72.8%	79.0%

Table 7: Generalization results on the ETH dataset. All models are pre-trained on other datasets and then directly evaluated on the unseen ETH dataset [ 68 ] .

	00		01		02		03		04		05		06		07 †		08 †		09 †		10 †		Mean on 07-10
Method	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$	$t_{rel}$	$r_{rel}$
ICP-po2po	6.88	2.99	11.21	2.58	8.21	3.39	11.07	5.05	6.64	4.02	3.97	1.93	1.95	1.59	5.17	3.35	10.04	4.93	6.93	2.89	8.91	4.74	7.36	3.41
ICP-po2pl	3.80	1.73	13.53	2.58	9.00	2.74	2.72	1.63	2.96	2.58	2.29	1.08	1.77	1.00	1.55	1.42	4.42	2.14	3.95	1.71	6.13	2.60	4.74	1.93
GICP [ 75 ]	1.29	0.64	4.39	0.91	2.53	0.77	1.68	1.08	3.76	1.07	1.02	0.54	0.92	0.46	0.64	0.45	1.58	0.75	1.97	0.77	1.31	0.62	1.92	0.73
CLS [ 81 ]	2.11	0.95	4.22	1.05	2.29	0.86	1.63	1.09	1.59	0.71	1.98	0.92	0.92	0.46	1.04	0.73	2.14	1.05	1.95	0.92	3.46	1.28	2.15	1.00
LOAM ∗ [ 105 ]	15.99	6.25	3.43	0.93	9.40	3.68	18.18	9.91	9.59	4.57	9.16	4.10	8.91	4.63	10.87	6.76	12.72	5.77	8.10	4.30	12.67	8.79	10.82	5.43

Table 8: Comparison with the state-of-the-art on the LiDAR odometry task.

t_{rel}

r_{rel}

indicate the average translation RMSE (%) and rotation RMSE ( ∘ /100m) respectively on all subsequences in the length of

100,200,...,800m

. ‘ ∗ ’ means that only odometry results without mapping are listed for a fair comparison. ‘ † ’ means the evaluation sequences. ‘

{NG}

’ means results are not given. We re-evaluate the results of PWCLO [ 86 ] and obtain the same results as in [ 1 ] . The best results for each sequence are bold , and the second best is underlined .

Figure 9: Visualization of 2D and 3D trajectories of LiDAR odometry. The blue and red lines are respectively our estimated and the ground truth trajectory.

Figure 10: Comparison results with traditional LiDAR odometry LOAM.

Model	RTE(m)	RRE(°)	Recall(%)
(a) Transformation $q_{3}$ , $t_{3}$	0.79 $\pm$ 0.47	1.46 $\pm$ 0.98	74.9%
(b) Transformation $q_{2}$ , $t_{2}$	0.53 $\pm$ 0.37	1.37 $\pm$ 0.91	88.9%
(c) Transformation $q_{1}$ , $t_{1}$	0.20 $\pm$ 0.19	0.57 $\pm$ 0.50	99.7%
(d) Transformation $q_{0}$ , $t_{0}$	0.04 $\pm$ 0.06	0.14 $\pm$ 0.13	100%

Table 9: Ablation studies of the hierarchical architecture.

Model	RTE(m)	RRE(°)	Recall(%)
(a) w/o projection mask	0.14 $\pm$ 0.13	0.41 $\pm$ 0.43	99.5%
(b) replaced by 2D CNN	0.51 $\pm$ 0.39	0.76 $\pm$ 0.73	83.6%
(c) replaced by PointNet++ [ 70 ]	0.21 $\pm$ 0.20	0.47 $\pm$ 0.51	93.6%
(d) replaced by Point transformer [ 110 ]	0.11 $\pm$ 0.14	0.24 $\pm$ 0.26	99.6%
Ours (Full)	0.04 $\pm$ 0.06	0.14 $\pm$ 0.13	100%

Table 10: Ablation studies of components in Point Swin Transformer (PST).

Model	RTE(m)	RRE(°)	Recall(%)
(a) w/o cross attention	0.09 $\pm$ 0.11	0.20 $\pm$ 0.29	98.8%
(b) w/o all-to-all point gathering	0.86 $\pm$ 0.39	1.69 $\pm$ 1.37	67.2%
(c) replaced with cost volume in [ 86 ]	0.69 $\pm$ 0.33	0.98 $\pm$ 0.82	71.2%
(d) replaced with cost volume in [ 83 ]	0.15 $\pm$ 0.17	0.21 $\pm$ 0.25	94.8%
Ours (Full)	0.04 $\pm$ 0.06	0.14 $\pm$ 0.13	100%

Table 11: Ablation studies of components in Bijective Association Transformer (BAT) module.

$SF$	FE trans	RTE(m)	RRE(°)	Recall(%)
		0.07 $\pm$ 0.12	0.20 $\pm$ 0.21	99.7%
✓		0.06 $\pm$ 0.10	0.19 $\pm$ 0.17	99.8%
	✓	0.05 $\pm$ 0.06	0.18 $\pm$ 0.15	99.9%
✓	✓	0.04 $\pm$ 0.06	0.14 $\pm$ 0.13	100%

Table 12: Ablation studies of the transformed features in the optimal transport (OT) layer.

Figure 11: Visualization of attention weights. We give two samples here, where the first two rows respectively represent corresponding pictures and projected point clouds. Attention weights are visualized in the last row. Dynamic objects (red box) have lower attention weights, and static objects (green box) have higher weights.

Figure 12: Visualization of the cross attention mechanism in BAT. Points A and B are corresponding points respectively in the source and target frame. Point C is the transformed position of A by conditioned features after cross-attention block.

Point Cloud Registration

Point cloud registration aligns 3D point sets by estimating the transformations and correspondences that make their geometry consistent.

See all 111 papers

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration
Aug 20, 2026Eunsoo Im, Junghun Suh, Gyeonggwan Lee +1Point Cloud Registration Point Cloud Learning
DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers
Sep 11, 2026Lanke Frank Tarimo Fu, Maurice Fallon3D ViTs Camera Pose Estimation
University of Oxford, Oxford OX2 6NN, United Kingdom
Unsupervised Point Cloud Registration via Training-Time Semantic Guidance
Sep 14, 2026Kezheng Xiong, Shiyun Xu, Sheng Ao +3Point Cloud Registration
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China

Aug 20, 2026J/K move · Enter open · S save

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

Eunsoo Im, Junghun Suh, Gyeonggwan Lee, Seunghwan Hong

Point Cloud Registration Point Cloud Learning LiDAR Localization Cross-Modal Knowledge Distillation

Open paper AskPDF arXiv

Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics. We propose CVSD-Reg, a robust global LiDAR registration framework that distills visual semantic priors from a vision foundation model into LiDAR representations. In Stage 1, a Point Transformer V3 student learns from a frozen DINOv2 teacher through contrastive distillation and spherical-manifold alignment, which preserves the hyperspherical geometry of the teacher embedding space. Self-supervised InfoNCE consistency and soft

\mathrm{SE}(3)

invariance further encourage viewpoint-robust descriptors. In Stage 2, the distilled representation is adapted to registration through correspondence learning, density-aware point-dropout augmentation, and end-to-end pose optimization. With a single checkpoint, CVSD-Reg generalizes to both single-sensor and zero-shot cross-sensor scenarios without sensor-specific adaptation and remains entirely camera-free at inference. On KITTI, nuScenes, and HeLiPR, CVSD-Reg achieves strict success rate ([email protected],m/

1^\circ

) of 97.7

\%

, 99.0

\%

, and 99.3

\%

, respectively, including 97.3

\%

on sparse 16-beam Velodyne scans. It outperforms state-of-the-art geometric registration methods by up to 44.0 percentage points without requiring camera inputs or post-hoc ICP refinement.

Abstract

Point Cloud Registration

Explore similar work

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers

Unsupervised Point Cloud Registration via Training-Time Semantic Guidance