Toward Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups

Hyungtae Lim; Minkyun Seo; Luca Carlone; Jaesik Park

cs.CVJan 6, 2026

Toward Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups

Authors: Hyungtae Lim, Minkyun Seo, Luca Carlone, Jaesik Park

Organizations: Laboratory for Information and Decision Systems (LIDS), Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Computer Science and Engineering, Seoul National University, Seoul, Republic of Korea

Abstract

Some deep learning-based point cloud registration methods struggle with zero-shot generalization, often requiring target-domain retraining, fine-tuning, or dataset-specific metric parameter tuning for new environments. We identify three critical limitations: (a) fixed user-defined parameters (e.g., voxel size, search radius) that fail to generalize across varying scales, (b) learned keypoint detectors exhibit poor cross-domain transferability, and (c) absolute coordinates amplify scale mismatches between datasets. To address these three issues, we present BUFFER-X, a registration framework whose descriptor model is trained once and then applied without parameter tuning or target-domain retraining, which achieves zero-shot generalization through: (a) geometric bootstrapping for automatic hyperparameter estimation, (b) distribution-aware farthest point sampling to replace learned detectors, and (c) patch-level coordinate normalization to ensure scale consistency. Our approach employs hierarchical multi-scale matching to extract correspondences across local, middle, and global receptive fields, enabling robust registration in diverse environments. For efficiency-critical applications, we present BUFFER-X-Lite from a holistic system perspective, integrating the BUFFER-X front-end and adaptive early exit with the fast and robust pose solver to reduce total computation time by 43% relative to BUFFER-X while preserving comparable accuracy. We evaluate on a comprehensive benchmark comprising 12 datasets spanning object-scale, indoor, and outdoor scenes, including cross-sensor registration between heterogeneous LiDAR configurations. Results demonstrate that our approach generalizes effectively across diverse scales, scenes, and sensor setups. Code: https://github.com/MIT-SPARK/BUFFER-X.

Figures & tables

Panel of Figure 1 — Fig. 2: (a) Variation in the number of points after voxelization with different voxel sizes $v$ across datasets. Even in indoor scenes, point counts vary significantly depending on the sensor type ( i.e., TIERS [ 6 ] vs. 3DMatch [ 2 ] ). Notably, TIERS and KITTI [ 4 ] , both using omnidirectional LiDARs, yield different point densities due to indoor vs. outdoor environments. (b) Empirical distribution of the datasets’ maximum range.

Fig. 3: Overview of our BUFFER-X , which mainly consists of three steps. (a) Geometric bootstrapping ( Sec. IV-A ) to determine the appropriate voxel size and radii for the given source $\mathcal{P}$ and target $\mathcal{Q}$ clouds. (b) Multi-scale patch embedder ( Sec. IV-B ) to generate patch-wise descriptor $\mathcal{S}_{\xi}$ for multiple scales $\xi\in\{l,m,g\}$ , where $l$ , $m$ , and $g$ represent local, middle, and global scales, respectively. Specifically, Mini-SpinNet [ 14 ] outputs cylindrical feature maps $\mathcal{C}_{\xi}$ and vector feature sets $\mathcal{F}_{\xi}$ . (c) Hierarchical inlier search ( Sec. IV-C ), which first performs nearest neighbor-based intra-scale matching using $\mathcal{F}^{\mathcal{P}}_{\xi}$ and $\mathcal{F}^{\mathcal{Q}}_{\xi}$ at each scale, followed by pairwise transformation estimation. Finally, it identifies globally consistent inliers $\mathcal{I}$ across all scales to refine correspondences based on consensus maximization [ 87 , 88 ] .

Fig. 4: (a) Visual description of local ( $r_{l}$ ), middle ( $r_{m}$ ), and global ( $r_{g}$ ) radii for the same point to illustrate scale differences and (b) normalized patches ranging from $[-1,1]$ . Note that their reference frames follow the eigenvectors obtained from principal component analysis (PCA) [ 91 , 90 ] . The $z$ -axis is assigned to the eigenvector ${\bm{v}}_{3}$ , which corresponds to the smallest eigenvalue.

Panel of Figure 4 — Fig. 5: Dataset statistics and characteristics analysis, which demonstrates the diversity of point cloud characteristics across different environments, sensor types, and acquisition scenarios in our generalizability benchmark. (a)-(b) Box plots of mean and 99th percentile range per frame across different datasets, respectively. (c) Distribution of the number of point clouds and the number of selected test pairs across indoor and outdoor environments. (d)-(e) Gaussian distribution of ranges for indoor and outdoor scenes, respectively, and (f)-(g) those for different sensor types in the TIERS and KAIST sequences.

Panel of Figure 5 — Fig. 6: (a) Different scales of sequences from the TIERS dataset [ 6 ] in our experiments. While each scan is utilized for our evaluation, we build and then visualize map clouds using LiDAR point cloud scans and corresponding poses to illustrate the different scales of the surroundings. (b) Scale comparison of three sequences: Oxford from the NewerCollege dataset [ 7 ] , MIT from Kimera-Multi [ 9 ] , and KAIST from the HeLiPR dataset [ 11 ] at the same scale ( i.e., 500 m). Note that although these sequences fall under the same campus category, their scales differ. For clarity, the map clouds are visualized with respect to their $z$ values. (c) (T-B) Examples of visualized LiDAR scans from different LiDAR sensors in the Indoor10 of TIERS dataset [ 6 ] and KAIST05 sequence of the HeLiPR dataset [ 11 ] . Note that even in the same environment, differences in the number of LiDAR rays and the field of view result in point clouds with different patterns.

Param.	Description	Value
$\kappa_{\text{spheric}}$	Coefficient for voxel size when sphericity is high	0.15
$\kappa_{\text{disc}}$	Coefficient for voxel size when sphericity is low	0.10
$\tau_{v}$	Threshold in Eq. 3	0.05
$\tau_{l}$	Threshold for the local ( $l$ ) search radius	0.005
$\tau_{m}$	Threshold for the middle ( $m$ ) search radius	0.02
$\tau_{g}$	Threshold for the global ( $g$ ) search radius	0.05

TABLE I: Global configuration used by BUFFER-X. The same values are fixed for all 12 datasets without any per-dataset tuning.

Panel of Table 7 — TABLE II: Quantitative performance comparison of registration success rates to evaluate generalization capability. Deep learning-based models were trained only on 3DMatch [ 2 ] and KITTI [ 4 ] , respectively, and to ensure a fair comparison, RANSAC was employed for pose estimation across all learning-based approaches, with a maximum of 50K iterations. For conventional methods, FPFH [ 102 ] was used for feature extraction.

Panel of Figure 8 — Fig. 7: Qualitative comparison of registration results across diverse scenes (source: orange and target: cyan). All methods were trained on 3DMatch . Top three rows: scan pairs from ScanNet++F , Oxford , and MIT . Bottom four rows: scan pairs with heterogeneous LiDAR sensor setups from TIERS (Vel16 $\rightarrow$ OS128, OS64 $\rightarrow$ Vel16) and KAIST (Aeva $\rightarrow$ Avia, Ouster $\rightarrow$ Aeva), respectively, where A $\rightarrow$ B means that the source acquired by sensor type A is registered to the target point cloud from sensor type B.

Fig. 8: Relative translation error (RTE) and relative rotation error (RRE) of our approach to state-of-the-art methods, all trained on 3DMatch and tested on KITTI , with oracle tuning and scale alignment, corresponding to those in Table II under the + + setting. The **** annotations indicate measurements with a

p

-value

<10^{-4}

after a paired

t

-test. Note that even if other state-of-the-art approaches showed promising high success rates in Table II , but in terms of RTE and RRE, our BUFFER-X showed substantially lower values.

Panel of Table 10 — TABLE III: Registration success rates under heterogeneous sensor configurations. Deep learning-based methods are trained only on 3DMatch [ 2 ] . Conventional methods use FPFH [ 102 ] . To ensure a fair comparison, RANSAC was employed for pose estimation across all learning-based approaches, with a maximum of 50K iterations.

Fig. 9: Visualization of patch-level similarity across heterogeneous LiDAR sensors. Despite large differences in overall point density distributions and fields of view captured by Livox Avia and Ouster OS2-128, local patches from different point clouds (A and B) exhibit consistent feature distributions, showing high cosine similarity close to 1.

Fig. 10: PaCMAP visualization of descriptor distributions across datasets. Unlike state-of-the-art approaches [ 73 , 3 , 79 ] , our BUFFER-X shows persistent descriptors that are robust to domain shift, as evidenced by the overlap between 3DMatch (training domain) and other datasets. That is, the closer and more overlapped with sky blue, the better.

Fig. 11: Distribution of correspondences at different scales (global, middle, local) across datasets, showing the contribution of multi-scale matching.

Local	Middle	Global	RTE [cm] $\downarrow$	RRE [°] $\downarrow$	Succ. rate [%] $\uparrow$	Hz $\uparrow$
✓			6.57	2.15	84.06	5.61
	✓		5.87	1.85	93.38	5.47
		✓	6.06	1.91	93.57	5.49
✓	✓		5.73	1.81	94.31	2.35
✓		✓	5.77	1.81	94.02	2.36
	✓	✓	5.78	1.81	94.62	2.33

TABLE IV: Ablation study on multi-scale combinations and early exit strategies evaluated on 3DMatch [ 2 ] . Top: different scale combinations. Bottom: comparison with RANSAC and KISS-Matcher solver [ 45 ] (referred to as our solver ) for early exit.

Fig. 12: Benchmark-average recall obtained by varying one parameter group at a time while fixing all other parameters to the default values in Table I . Red stars indicate the default parameter values used throughout our experiments. OOM ( i.e., out of memory) denotes configurations that exceeded the available GPU memory. Full per-dataset recalls are reported in Tables S2 and S3 of the supplementary material.

Fig. 13: Comparison of success rate and total computation time between BUFFER-X-Lite at different early-exit thresholds and the original BUFFER-X.

Fig. 14: Pose estimation time comparison between RANSAC and KISS-Matcher solver given the same correspondences. RANSAC, which is still widely used for pose estimation in learning-based approaches, exhibits substantially longer computation time for LiDAR-acquired point clouds ( i.e., in TIERS , KAIST , and MIT ), whereas KISS-Matcher solver achieves up to 100

\times

faster performance.

Fig. 15: Peak GPU memory (left) and total runtime (right) versus raw input point count for the default three-scale pipeline, the single-scale variant, and pair-adaptive early exit. Dots denote individual registration pairs, and solid lines denote binned medians. Note that pooling multiple datasets yields uneven point-count coverage and varying dataset composition across bins, thereby producing local non-monotonicity in the median curves.

Panel of Figure 19 — Fig. 16: Comparison of mutual correspondences versus inliers for RANSAC and KISS-Matcher solver across 3DMatch , ScanNet++i , TIERS , and KAIST . The number of estimated inliers in failure cases for RANSAC is relatively widespread across lower values, whereas that of KISS-Matcher solver is more likely to be near-zero.

Fig. 17: Application of our approach to robotic systems: online submap-to-map localization between two sessions on the Cantwell scene from the Gibson dataset [ 106 ] .

Fig. 18: Overlap-conditioned analysis on outdoor benchmarks. Registration recall and pair count of BUFFER-X over all outdoor benchmark pairs grouped by overlap ratio (left) and controlled KITTI evaluation using a protocol adopted in prior work [ 107 ] , where the odometry-based pair-selection distance is increased from 10 m to 20 m, 30 m, and 40 m without cropping or otherwise modifying the scans (right).

Aspect	Prior work	This extended manuscript
Core BUFFER-X components	Geometric bootstrapping, distribution-aware FPS, patch-level coordinate normalization, hierarchical multi-scale matching, and cross-scale consensus maximization.	Inherited without being presented as a new contribution of this extension; retained to provide a self-contained description.
Primary research focus	Establishes the feasibility of zero-shot transfer from a single training dataset to unseen indoor and outdoor scenes.	Shifts the focus from whether transfer is possible to why it works and under which conditions it remains reliable.
Efficiency extension	Not included.	Presents BUFFER-X-Lite through a system-level integration of the BUFFER-X front-end, pair-adaptive early exit, and the previously developed KISS-Matcher correspondence-pruning and pose-estimation modules; Algorithm 2 provides the complete procedure.
Evaluation scope	Focuses primarily on homogeneous indoor and outdoor registration.	Expands to a 12-dataset benchmark spanning object-scale, indoor, and outdoor scenes, including ModelNet40 and six heterogeneous LiDAR source–target directions: OS128 $\rightarrow$ OS64, OS64 $\rightarrow$ Vel16, Vel16 $\rightarrow$ OS128, Aeva $\rightarrow$ Avia, Avia $\rightarrow$ Ouster, and Ouster $\rightarrow$ Aeva.
New quantitative analyses	Component-level ablations supporting the initial zero-shot feasibility study.	Adds five analysis categories: descriptor-distribution consistency, correspondence complementarity across scales, benchmark-wide parameter sensitivity, pipeline-wide efficiency, and overlap-conditioned failure behavior.
New practical material	Not included.	Adds one submap-to-map localization application, complete BUFFER-X-Lite pseudocode, and an analysis of global-optimum ambiguity under severe partial overlap.

TABLE S1: Explicit comparison between our prior work and this extended manuscript. This extension retains the BUFFER-X backbone for a self-contained presentation while adding a broader evaluation scope, quantitative analyses, and an efficiency-oriented system-level integration of the front-end and solver.

Fig. S1: Registration recall versus measured pair overlap for pooled indoor benchmarks ( 3DMatch , 3DLoMatch , ScanNet++i , and ScanNet++F ). Bars show the number of evaluated pairs in each overlap bin.

Panel of Figure 28 — Fig. S2: Illustration of the global optimum ambiguity problem in point cloud registration. (a) Two misaligned L-shaped point clouds and (b)–(d) possible alignments: overlap along the longer segment ( L ), overlap along the shorter segment ( S ), and full overlap ( F ), respectively. (e) German–McClure (GM) surrogate costs at different control parameters $\mu$ and (f) the corresponding loss values. (g) Truncated least-squares (TLS) surrogate costs and (h) their corresponding loss values. The fully overlapping case produces the lowest loss for all tested kernel shapes.

Panel of Figure 29 — Fig. S3: Qualitative low-overlap registration examples on 3DLoMatch . The top four rows show the behavior of BUFFER-X in failed cases, where the prediction can favor a geometrically plausible but incorrect high-overlap alignment. The bottom two rows show successful low-overlap cases, where distinctive anchor objects or structures provide sufficient geometric evidence for correct registration.

$N_{\mathrm{FPS}}$
	Object	Indoor					Outdoor						Avg.
Setting	ModelNet40	3DMatch	3DLoMatch	ScanNet++i	ScanNet++F	TIERS	KITTI	WOD	KAIST	MIT	ETH	Oxford	Avg.
500	99.13	93.67	69.54	94.07	99.36	92.30	99.10	100.00	93.02	93.48	89.76	97.67	93.43 (-2.71 %p)
1,000	99.45	94.36	72.33	95.03	99.80	92.76	99.82	100.00	98.79	98.70	99.02	99.67	95.81 (-0.33 %p)
1,500	99.84	95.58	74.18	94.99	99.90	93.45	99.82	100.00	99.15	97.39	99.72	99.67	96.14
2,000	99.45	94.79	72.68	95.56	99.95	91.95	99.82	100.00	99.20	97.83	100.00	99.34	95.88 (-0.26 %p)

TABLE S2: Full per-dataset recalls (%) for the one-factor parameter sweeps in Fig. 12. Shaded rows denote default settings. Red numbers indicate a decrease from the default parameter setting, while green numbers indicate an increase.

	Outdoor						Avg.
$r_{\mathrm{max}}$	KITTI	WOD	KAIST	MIT	ETH	Oxford	Avg.
2.5 m	99.82	100.00	97.94	96.52	99.86	99.00	98.86 (-0.43 %p)
5.0 m	99.82	100.00	99.15	97.39	99.72	99.67	99.29
7.5 m	99.82	100.00	99.35	99.13	99.86	99.67	99.64 (+0.35 %p)
10.0 m	99.82	100.00	99.35	97.83	100.00	99.67	99.44 (+0.15 %p)

TABLE S3: Outdoor-specific sensitivity to the radius cap

r_{\mathrm{max}}

. This parameter applies to large-scale LiDAR scans; therefore, only outdoor datasets are reported. Shaded rows indicate default settings. Values are registration recall (%). Red numbers indicate a decrease from the default parameter setting, while green numbers indicate an increase.

Point Cloud Registration

Point cloud registration aligns 3D point sets by estimating the transformations and correspondences that make their geometry consistent.

See all 110 papers

Oct 20, 2026

Register Anything Model for Generalizable and Robust Point Cloud Registration

We present Register Anything Model (RAM), a unified model for generalizable 3D point cloud registration that robustly estimates 6-DoF transformations across diverse domains spanning sensors and environments. Existing registration methods are typically tailored to a single domain, with carefully tuned architectures and hyperparameters, and thus degrade markedly when applied to novel domains. We bridge this gap by training a single model on diverse data from indoor, outdoor, and object domains. A central challenge is the severe geometric discrepancy across domains, i.e., differences in scene scale, sensor noise, point density, and structural patterns, which hinders processing with a unified point-based model. To address this, we introduce a geometry-aware normalization strategy that jointly voxelizes and normalizes point clouds into a standardized geometric space, adaptively based on their geometric properties to preserve salient local structures while standardizing resolution. Built on this normalization, RAM follows a hierarchical coarse-to-fine paradigm, and employs an efficient geometric transformer that models local geometric consistency instead of globally over all points. This design mitigates discretization noise introduced by voxelization, and improves computational efficiency. Extensive experiments on 12 benchmarks demonstrate state-of-the-art performance in terms of inlier ratio and registration recall. Notably, our method outperforms the counterparts trained on each separate domain, showing superior generalization capability.

Zheng Qin, Junhua Xi, Yan Huang +5

Defense Innovation Institute, Academy of Military Sciences, China · College of Computer Science and Technology, National University of Defense Technology, China · College of Data Intelligence, Information Support Force Engineering University, China +1

Point Cloud Registration Point Cloud Learning Domain Generalization

Ask PDF

Apr 17, 2026

C-GenReg: Training-Free 3D Point Cloud Registration by Multi-View-Consistent Geometry-to-Image Generation with Probabilistic Modalities Fusion

We introduce C-GenReg, a training-free framework for 3D point cloud registration that leverages the complementary strengths of world-scale generative priors and registration-oriented Vision Foundation Models (VFMs). Current learning-based 3D point cloud registration methods struggle to generalize across sensing modalities, sampling differences, and environments. Hence, C-GenReg augments the geometric point cloud registration branch by transferring the matching problem into an auxiliary image domain, where VFMs excel, using a World Foundation Model to synthesize multi-view-consistent RGB representations from the input geometry. This generative transfer, preserves spatial coherence across source and target views without any fine-tuning. From these generated views, a VFM pretrained for finding dense correspondences extracts matches. The resulting pixel correspondences are lifted back to 3D via the original depth maps. To further enhance robustness, we introduce a "Match-then-Fuse" probabilistic cold-fusion scheme that combines two independent correspondence posteriors, that of the generated-RGB branch with that of the raw geometric branch. This principled fusion preserves each modality inductive bias and provides calibrated confidence without any additional learning. C-GenReg is zero-shot and plug-and-play: all modules are pretrained and operate without fine-tuning. Extensive experiments on indoor (3DMatch, ScanNet) and outdoor (Waymo) benchmarks demonstrate strong zero-shot performance and superior cross-domain generalization. For the first time, we demonstrate a generative registration framework that operates successfully on real outdoor LiDAR data, where no imagery data is available.

Yuval Haitman, Amit Efraim, Joseph M. Francos

Ben-Gurion University, Beer-Sheva, Israel

Point Cloud Registration Cross-Modal Feature Matching

Ask PDF

Aug 20, 2026

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics. We propose CVSD-Reg, a robust global LiDAR registration framework that distills visual semantic priors from a vision foundation model into LiDAR representations. In Stage 1, a Point Transformer V3 student learns from a frozen DINOv2 teacher through contrastive distillation and spherical-manifold alignment, which preserves the hyperspherical geometry of the teacher embedding space. Self-supervised InfoNCE consistency and soft

\mathrm{SE}(3)

invariance further encourage viewpoint-robust descriptors. In Stage 2, the distilled representation is adapted to registration through correspondence learning, density-aware point-dropout augmentation, and end-to-end pose optimization. With a single checkpoint, CVSD-Reg generalizes to both single-sensor and zero-shot cross-sensor scenarios without sensor-specific adaptation and remains entirely camera-free at inference. On KITTI, nuScenes, and HeLiPR, CVSD-Reg achieves strict success rate ([email protected],m/

1^\circ

) of 97.7

\%

, 99.0

\%

, and 99.3

\%

, respectively, including 97.3

\%

on sparse 16-beam Velodyne scans. It outperforms state-of-the-art geometric registration methods by up to 44.0 percentage points without requiring camera inputs or post-hoc ICP refinement.

Eunsoo Im, Junghun Suh, Gyeonggwan Lee +1

Point Cloud Registration Point Cloud Learning LiDAR Localization Cross-Modal Knowledge Distillation

Ask PDF

Abstract

Point Cloud Registration

Explore similar work

Register Anything Model for Generalizable and Robust Point Cloud Registration

C-GenReg: Training-Free 3D Point Cloud Registration by Multi-View-Consistent Geometry-to-Image Generation with Probabilistic Modalities Fusion

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration