cs.CVMar 2, 2025

SemICP: Semantic Non-Rigid Point Cloud Registration with Elastic Energy Regularization

Authors: Wanwen Chen, Qi Zeng, Carson Studders, Zongze Li, Jamie J. Y. Kwon, Tara Kemper, Emily H. T. Pang, Eitan Prisman, 

Organizations: Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada. · Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada. · School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.

Abstract

Purpose: Accurate point cloud registration is essential in computer-aided interventions (CAI) to align multi-modal medical images for intraoperative guidance. Classical methods, such as Iterative Closest Point (ICP), remain attractive for their explainability and minimal training requirements, but typically ignore anatomical semantics and biomechanical properties during regularization. Methods: We present Semantic ICP (SemICP), a novel non-rigid point cloud registration framework that combines semantically informed point matching with linear elastic energy regularization. Semantic labels are used to improve correspondence matching by constraining correspondences to be anatomically consistent. A novel control-point deformation representation with linear-elastic energy regularization is introduced to encourage biomechanically plausible deformations. SemICP was evaluated on a simulation phantom and four datasets against established baselines. It was also tested with labels from AI-based segmentation in a fully automatic segmentation-registration pipeline. Results: On the finite element phantom, SemICP recovered 86% of the interior displacement from surface observations alone. On the real datasets, SemICP achieved the lowest target registration error on Liver MR-US and the lowest surface distances on TORS, and ranked second on AbdominalMR-CT and HippocampusMR, where NR-ICP achieved lower surface distances. The fully automatic registration pipeline was shown to be effective for US-MR registration and to improve the alignment of expert-annotated structures. Conclusion: SemICP improves interior accuracy at competitive surface accuracy by combining semantic correspondence constraints and linear energy regularization. Combined with AI-based segmentation, SemICP provides an effective pipeline for multi-modal registration in CAI.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 27, 2026

SAGE: Semantic-Augmented Geometric Experts for Point Cloud Registration

Point cloud registration is an essential task for scene reconstruction, requiring the alignment of partially overlapping point clouds. Under low-overlap conditions, this task remains challenging because existing geometry-only methods cannot distinguish geometrically similar but semantically distinct structures (e.g., TV corners vs. desk corners and chair legs vs. table legs), resulting in incorrect correspondences. To address this geometric ambiguity, we present Semantic-Augmented Geometric Experts (SAGE), which integrates frozen pre-trained point cloud semantic representations into the registration pipeline without requiring additional sensors or fine-tuning the semantic encoder. To effectively leverage these semantic distinctions, SAGE employs a Semantic-Guided Mixture of Experts (SG-MoE) architecture. Through semantic-aware routing, SG-MoE enables each expert to specialize in distinct semantic-geometric patterns, reducing mismatches that confound geometry-only methods. With a structure-aware estimator, SAGE achieves the highest registration recall on 3DLoMatch while remaining competitive on 3DMatch. When evaluated with the same number of refinement iterations, SAGE further achieves the highest inlier ratio and registration recall on both benchmarks. These results indicate that frozen semantic representations provide effective cues for resolving geometric ambiguity in low-overlap registration.
Sep 14, 2026

Unsupervised Point Cloud Registration via Training-Time Semantic Guidance

Unsupervised registration of large-scale LiDAR point clouds remains challenging due to the geometric ambiguity inherent in outdoor scenes, which degrades pseudo-label quality and leads to suboptimal convergence, particularly for sparse, low-resolution scans such as those from nuScenes. We reveal that registration models intrinsically encode semantic awareness that strongly correlates with registration accuracy, albeit without explicit semantic supervision. However, this native awareness is fragile: noisy supervision arising from geometric ambiguity in unsupervised settings rapidly erodes the learned semantic structure, causing performance collapse. To this end, we propose CAESAR, a teacher-student framework guided by an off-the-shelf 3D segmentation model exclusively during training. We observe that potential inlier matches are often buried just beneath a few spurious neighbors in the noisy feature space, motivating Dual-Cue Guided Re-Matching to recover them through reselection rather than simply rejecting. Building on this, a train-only Semantic-Geometric Label Mining performs lightweight, batch-specific teacher refinement and mines reliable pseudo-labels under semantic guidance. We further introduce Semantic Predictive Distillation to consolidate the student's semantic awareness in the feature space. Extensive experiments on KITTI and nuScenes demonstrate state-of-the-art performance, with pronounced gains on the challenging nuScenes benchmark. Crucially, CAESAR incurs zero inference overhead and requires no semantic annotations on the registration data. Code will be released.
Sep 29, 2026

Context without Commitment: Robust Dense Correspondence under Non-Rigid Deformation

Non-rigid point-cloud registration aims to find the corresponding target point for each point on a deforming source surface. Point-level matching keeps the full target cloud available, but correspondence becomes ambiguous when different regions have similar local geometry. Regional or coarse-to-fine methods provide broader spatial context, but an incorrect regional match can exclude the correct correspondence before dense matching. We propose CoCo-Reg, which uses regional patches to enrich dense point features without allowing patch predictions to restrict the final point-level search. CoCo-Reg constructs farthest-point-sampled patches, exchanges geometric information within and between source and target, supervises patch similarity using identity-corrected point overlap, and projects the resulting regional information back to dense point features. The final registration stage still scores the full target cloud before global point-level candidate selection. On 726 held-out ModelNet10 objects across nine deformation levels, two established learning-based baselines obtain mean correspondence errors of 0.1993 and 0.1921, whereas CoCo-Reg obtains 0.0547. Relative to its point-level baseline, this is a 72.6% reduction. CoCo-Reg achieves lower correspondence error on 92.3% of paired test objects and reduces the mean fraction of points with error above 0.1 from 47.3% to 17.3%. Chamfer distance and HD95 decrease in the same direction, and CoCo-Reg remains lower across all tested deformation levels. These results support using regional context for dense non-rigid correspondence without imposing a hard patch-level restriction on the final search. Because evaluation uses one checkpoint per method, the reported gains characterize the complete systems rather than the isolated causal contribution of an individual component. Code will be made publicly available.