SemICP: Semantic Non-Rigid Point Cloud Registration with Elastic Energy Regularization
Authors: Wanwen Chen, Qi Zeng, Carson Studders, Zongze Li, Jamie J. Y. Kwon, Tara Kemper, Emily H. T. Pang, Eitan Prisman, Septimiu E. Salcudean
Organizations: Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada. · Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada. · School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.
Purpose: Accurate point cloud registration is essential in computer-aided interventions (CAI) to align multi-modal medical images for intraoperative guidance. Classical methods, such as Iterative Closest Point (ICP), remain attractive for their explainability and minimal training requirements, but typically ignore anatomical semantics and biomechanical properties during regularization. Methods: We present Semantic ICP (SemICP), a novel non-rigid point cloud registration framework that combines semantically informed point matching with linear elastic energy regularization. Semantic labels are used to improve correspondence matching by constraining correspondences to be anatomically consistent. A novel control-point deformation representation with linear-elastic energy regularization is introduced to encourage biomechanically plausible deformations. SemICP was evaluated on a simulation phantom and four datasets against established baselines. It was also tested with labels from AI-based segmentation in a fully automatic segmentation-registration pipeline. Results: On the finite element phantom, SemICP recovered 86% of the interior displacement from surface observations alone. On the real datasets, SemICP achieved the lowest target registration error on Liver MR-US and the lowest surface distances on TORS, and ranked second on AbdominalMR-CT and HippocampusMR, where NR-ICP achieved lower surface distances. The fully automatic registration pipeline was shown to be effective for US-MR registration and to improve the alignment of expert-annotated structures. Conclusion: SemICP improves interior accuracy at competitive surface accuracy by combining semantic correspondence constraints and linear energy regularization. Combined with AI-based segmentation, SemICP provides an effective pipeline for multi-modal registration in CAI.
Figures & tables
Figure 1: SemICP workflow: rigid initialization followed by non-rigid refinement.
Mean
Interior error, mm (%)
SemICP
Load
Mag.
displ. (mm)
SemICP
Oracle
Surface, mm (%)
∣ΔdetJ∣
Compr.
1.5 mm
0.83
0.084 (10.1)
0.034 (4.1)
0.138 (16.5)
0.003
3 mm
1.66
0.131 (7.9)
0.071 (4.3)
0.212 (12.7)
0.004
6 mm
3.32
0.220 (6.6)
0.133 (4.0)
0.355 (10.6)
0.007
Shear
1.5 mm
0.76
0.171 (22.6)
0.013 (1.8)
0.224 (29.4)
0.002
3 mm
1.51
0.303 (20.0)
0.026 (1.7)
0.384 (25.2)
0.002
Table 1: Results on the FEM phantom. The mean interior and surface displacement errors are reported in mm, with the error as a percentage of the mean true displacement of the same points in parentheses. Oracle: SemICP with true surface correspondences in point matching. The surface error and Jacobian determinant error are reported on results estimated by SemICP.
Figure 2: SemICP on the finite element phantom. (a) Mean interior displacement error as a function of the regularization weight α for compression (blue) and shear (orange) of 1.5, 3 and 6 mm (light to dark); mean ± standard deviation over three samplings; dashed: α=10 . (b) Recovered displacement at α=10 for 3 mm compression (top) and 3 mm shear (bottom) on a vertical slice ( y=35 mm). Left to right: displacement magnitude of the finite element ground truth and of SemICP (arrows: in-plane displacement at twice its length), error magnitude, and deformed grid at true scale. Dashed: embedded structures.
Initial
DCPv1
DCPv2
ICP
GO-ICP
CPD
FR-ICP
NR-ICP
AMM-NRR
Fast-RNRR
SemICP
HD95
17.5 (12.8)
30.3 (10.8)
30.3 (11.3)
15.6 (11.6)
15.7 (12.3)
16.4 (11.1)
16.9 (11.5)
17.1 (11.7)
14.0 (11.6)
15.6 (11.8)
13.5 (12.9)
MSD
6.8 (4.4)
14.8 (7.2)
14.6 (7.9)
5.4 (3.9)
5.4 (3.9)
5.7 (3.6)
5.9 (4.2)
6.0 (4.0)
4.4 (2.8)
5.6 (3.7)
4.1 (3.0)
Time
0.1 (0.4)
0.1 (0.4)
0.3 (0.3)
12.7 (0.4)
3.5 (0.9)
15.3 (8.7)
262.2 (373.5)
53.0 (14.8)
14.2 (7.4)
65.8 (24.1)
Table 2: HD95 and MSD in mean (std), mm and runtime (s) on the TORS dataset.
Figure 3: Registration results on a TORS US-CT pair (posterior-anterior view). Dark colors: CT structures (fixed). Light colors: US structures after registration by each method (moving). Each structure has one hue: red, carotid artery; purple, larynx/mandibular bone; blue, jugular vein.
Initial
DCPv1
DCPv2
ICP
GO-ICP
CPD
FR-ICP
NR-ICP
AMM-NRR
Fast-RNRR
SemICP
HD95
38.4 (19.4)
73.6 (28.1)
96.0 (54.8)
18.7 (6.9)
18.9 (7.5)
17.1 (11.5)
23.7 (10.4)
8.2 (7.8)
18.4 (17.6)
17.9 (19.9)
11.1 (8.9)
MSD
17.1 (10.4)
37.1 (22.5)
58.5 (48.0)
7.2 (2.5)
7.2 (2.7)
7.3 (5.8)
8.0 (3.6)
2.2 (1.6)
6.4 (10.0)
6.8 (9.5)
3.5 (1.3)
Time
0.8 (1.9)
0.8 (1.9)
1.1 (0.5)
30.9 (32.6)
25.4 (38.4)
24.7 (14.7)
1053.3 (568.3)
91.6 (24.7)
27.2 (19.4)
84.1 (12.2)
Table 3: HD95 and MSD in mean (std), mm and runtime (s) on AbdominalMRCT.
Figure 4: Registration results on one CT-MR pair in AbdominalMRCT (posterior-anterior view). Dark colors: MR structures (fixed). Light colors: CT structures after registration by each method (moving). Each structure has one hue: red, liver; purple, spleen; blue, right kidney; orange, left kidney.
Initial
DCPv1
DCPv2
ICP
GO-ICP
CPD
FR-ICP
NR-ICP
AMM-NRR
Fast-RNRR
SemICP
HD95
3.8 (1.3)
11.2 (5.8)
13.5 (5.7)
2.5 (1.1)
2.6 (1.3)
2.2 (1.0)
2.8 (1.4)
1.1 (0.8)
2.2 (1.7)
1.8 (1.6)
1.1 (1.2)
MSD
1.7 (0.6)
5.0 (3.3)
6.3 (3.5)
1.0 (0.3)
1.0 (0.3)
0.9 (0.2)
1.0 (0.3)
0.2 (0.2)
0.6 (0.3)
0.5 (0.3)
0.4 (0.2)
Time
0.07 (0.25)
0.08 (0.26)
0.15 (0.03)
25.2 (17.9)
32.0 (13.8)
0.16 (0.06)
20.2 (5.0)
0.7 (0.2)
0.3 (0.08)
13.8 (1.4)
Table 4: HD95 and MSD in mean (std), mm and runtime (s) on hippocampusMR.
Figure 5: Registration results on one MR-MR pair in HippocampusMR (superior-inferior view). Dark colors: fixed MR structures. Light colors: moving MR structures after registration by each method. Each structure has one hue: red, hippocampus head; purple, hippocampus tail.
Initial
KeyMorph
TransMorph
ICP
GO-ICP
CPD
FR-ICP
NR-ICP
SemICP
TRE
14.5 (5.4)
7.4 (5.2)
8.0 (5.9)
10.3 (8.4)
7.8 (8.3)
11.5 (7.9)
7.0 (6.1)
N/A (N/A)
5.5 (3.0)
HD95
25.2 (5.2)
20.0 (7.6)
19.7 (7.5)
18.3 (7.0)
19.2 (8.9)
18.1 (6.8)
17.9 (5.6)
21.6 (6.1)
15.6 (5.6)
MSD
9.8 (3.1)
6.8 (4.2)
6.8 (4.8)
5.5 (2.5)
6.5 (5.9)
5.9 (2.8)
5.4 (2.7)
4.0 (1.6)
4.0 (2.1)
Time
2.2 (0.1)
0.04 (0.01)
1.0 (0.6)
39.5 (161.9)
17.7 (21.6)
15.0 (6.0)
290.5 (174.5)
18.0 (5.9)
Table 5: TRE, HD95 and MSD in mean (std), mm and runtime (s) on Liver.
Figure 6: Registration results on one MR-US pair in Liver MR-US (anterior-posterior view). Dark colors: MR structures (fixed). Light colors: US structures after registration by each method (moving). Each structure has one hue: red, gallbladder; purple, IVC; blue, kidney; orange, vessel.
Figure 7: The workflow overview of integration with learned labelers.
Dataset
Metrics
Gallbladder
IVC
Right Kidney
Vessels
Average
MR T1-W
Dice
0.867(0.073)
0.773(0.095)
0.955(0.010)
0.729(0.085)
0.831(0.066)
IoU
0.762(0.088)
0.630(0.106)
0.918(0.012)
0.578(0.096)
0.722(0.075)
US Bmode
Dice
0.771(0.069)
0.700(0.076)
0.606(0.091)
0.689(0.073)
0.692(0.072)
IoU
0.670(0.075)
0.560(0.081)
0.520(0.102)
0.543(0.084)
0.573(0.082)
Table 6: nnUnet segmentation 5-fold evaluation results reported in mean (std).
Initial
SemICP-rigid
SemICP-final
HD95
MSD
HD95
MSD
HD95
MSD
Gallbladder
Seg
11.21±12.15
5.20±8.92
8.92±7.21
3.96±4.46
3.67±1.29
1.46±0.61
GT
9.43±3.89
4.19±1.90
8.86±4.49
3.99±2.68
5.97±2.84
2.51±1.32
IVC
Seg
11.07±6.60
4.54±2.73
9.77±8.15
4.16±3.63
3.78±2.20
1.44±0.42
GT
11.39±7.42
4.78±3.41
11.48±10.52
4.77±4.31
8.23±6.93
3.09±2.25
Kidney
Seg
21.97±25.26
12.14±24.16
16.60±12.94
7.38±11.86
4.50±4.31
1.91±4.03
Table 7: Unidirectional HD95 and MSD (mean ± std, mm) on liver MR-3DUS. Seg: results on learning-based point clouds. GT: results on expert-labeled point clouds.
Figure 8: Liver registration results with learned labelers (anterior-posterior view). Dark colors: MR structures (fixed). Light colors: US structures after each registration stage (moving). Each structure has one hue: red, gallbladder; purple, IVC; blue, kidney; orange, vessel.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Regularizer
α
Interior (mm)
Surface (mm)
∣ΔdetJ∣
Folding (%)
None (no registration)
–
1.85 ± 0.99
1.86 ± 1.00
0.004
0.00
NL (ours)
0.01
1.00 ± 0.49
1.58 ± 0.75
0.092
0.05
0.1
0.84 ± 0.46
1.39 ± 0.70
0.047
0.01
1
0.60 ± 0.35
0.97 ± 0.46
0.015
0.00
10
0.25 ± 0.17
0.35 ± 0.21
0.004
0.00
100
0.76 ± 0.42
0.88 ± 0.49
0.004
0.00
Appendix
Table S1: Phantom, regularization terms: mean ± standard deviation over 18 registrations (2 loads × 3 magnitudes × 3 samplings). Best weight α per combination in bold (selected with the ground truth).
HD95
MSD
SDLogJ
Folding (%)
Initial
38.45±19.41
17.08±10.39
N/A
N/A
w/o semantic matching + Regels
11.42±9.03∗
3.56±1.28∗
0.015±0.007
0.00±0.00
Semantic w/o Reg
7.41±11.32∗
1.57±1.47∗
2.1±0.9∗
7.87±3.02∗
Semantic + Reggrad
20.61±9.54∗
7.78±2.92∗
0.0017±0.0001∗
0.00±0.00
Semantic + Regmag
20.80±9.88∗
7.87±3.08∗
0.0016±0.0002∗
0.00±0.00
Semantic + Regels (Ours)
11.08±8.89
3.47±1.28
0.015±0.007
0.00±0.00
Appendix
Table S2: HD95 and MSD (mean ± std, mm ), SDLogJ (mean ± std) and folding (mean ± std, % of the grid with J≤0 ) in ablation study on semantic matching and regularization on AbdominalMRCT ( α=10 , E=1 , ν=0.45 ). The best is in bold, and the second best is underlined. ∗ denotes significant difference from Semantic + Regels (Ours) ( p≤0.05 in Wilcoxon test).
Figure S1: Ablation on AbdominalMRCT (MR axial and coronal slices through the left kidney). (a) SemICP with (left) and without (right) semantic matching. White contours: MR segmentation; colored points: registered CT surface points (red, liver; purple, spleen; blue, right kidney; orange, left kidney). (b) Magnitude of the displacement field (mm, shared color scale) for each regularization configuration, overlaid on the MR slices with the MR segmentation contours.
Method
Parameter
Values
Configs
ICP (rigid)
number of sampled points
1000 (Liver: all ), 5000, all
15
landmarks matched per iteration
200 , 500, 1000, 2000, 5000
CPD (rigid)
outlier weight w
0 , 0.05, 0.1, 0.2, 0.3
15
number of sampled points
1000 , 3000, 5000
GO-ICP
convergence threshold (MSE)
0.005, 0.01
10
trimming fraction
0 , 0.05, 0.1, 0.2, 0.3
Appendix
Table S3: Parameter search grids for the cross validation. The default is shown in bold.
Dataset
Cases
Folds
Grouping
AbdominalMR-CT
8
8
leave-one-patient-out (8 patients)
TORS
24
12
leave-one-patient-out (24 sequences from 12 patients)
HippocampusMR
60
30
leave-one-pair-out: each image pair is registered in both directions, and both directions are held out together
Liver MR–3DUS
160
3
the three patient-level folds used to train KeyMorph and TransMorph (16 subjects × 10 samples)
Appendix
Table S4: Cross-validation fold definition and number of folds in each dataset.
Method
Parameters
MR-CT (8)
TORS (12)
Hippocampus (30)
Liver (3)
ICP
points, landmarks
5000, 2000 (4)
5000, 500 (11)
1000, 1000 (30)
5000, 2000; 1000, 2000; 5000, 5000 (1 each)
CPD
w , points
0.3, 1000 (6)
0.3, 1000 (12)
0.2, 3000 (26)
0.3, 1000 (2)
GO-ICP
MSE, trim
0.005, 0.05 (4)
0.005, 0 (11)
0.005, 0 (29)
0.005, 0 / 0.1 / 0.3 (1 each)
FR-ICP
νbegin , νend
1, 0.4 (5)
3, 0.8 (6)
3, 0.8 (30)
1, 0.4 (2)
NR-ICP
αstart→αend
2×102→1 (6)
2×105→1 (11)
2×106→0.1 (30)
2×103→0.1 (2)
AMM-NRR
α , β
100, 1 (6)
100, 1 (12)
1, 10 (30)
–
Appendix
Table S5: The configuration chosen in most folds in each dataset (number of folds in parentheses).
Dataset
Method
Cases
Successful
Failed (reason)
AbdominalMR-CT
AMM-NRR
8
7
1 (initialization did not finish)
Fast-RNRR
8
7
1 (initialization did not finish)
TORS
AMM-NRR
24
7
17 (initialization did not finish)
Fast-RNRR
24
6
18 (17 initializations did not finish, 1 diverged)
NR-ICP
24
23
1 (diverged)
HippocampusMR
AMM-NRR
60
58
2 (initialization did not finish)
Appendix
Table S6: Failed registrations in the final (tuned) runs. All other method–dataset combinations, including SemICP on all datasets, completed every case successfully.
E
1kPa
10kPa
100kPa
500kPa
1MPa
TRE
5.51
6.02 ∗
6.27 ∗
6.28 ∗
6.28 ∗
HD95
15.59
16.37 ∗
17.02 ∗
17.07 ∗
17.08 ∗
MSD
3.99
4.62 ∗
5.02 ∗
5.05 ∗
5.05 ∗
SDLogJ
0.013
0.0022 ∗
0.00031 ∗
0.00018 ∗
0.00017 ∗
Appendix
Table S7: The average TRE, HD95, MSD, and SDLogJ on Liver MR-US for different Young’s Modulus E , while ν=0.45 , α=10 . TRE, HD95 and MSD are reported in mm . ∗ means the results are significantly different compared with E =1kPa under p≤0.05 in a Wilcoxon test.
E
1kPa
10kPa
100kPa
500kPa
1MPa
HD95
11.08
16.83 ∗
20.11 ∗
20.66 ∗
20.75 ∗
MSD
3.47
6.24 ∗
7.59 ∗
7.81 ∗
7.85 ∗
SDLogJ
0.015
0.0034 ∗
0.00049 ∗
0.00023 ∗
0.00023 ∗
Appendix
Table S8: The average HD95, MSD and SDLogJ on AbdominalMRCT for different Young’s Modulus E , while ν=0.45 , α=10 . HD95 and MSD are reported in mm . ∗ means the results are significantly different compared with E =1kPa under p≤0.05 in a Wilcoxon test.
ν
0.35
0.40
0.45
0.499
TRE
5.50
5.50
5.51
5.95 ∗
HD95
15.61
15.58
15.59
16.16 ∗
MSD
3.96 ∗
3.97 ∗
3.99
4.42 ∗
SDLogJ
0.016 ∗
0.014 ∗
0.013
0.010 ∗
Appendix
Table S9: The average TRE, HD95, MSD and SDLogJ on Liver MR-US for different Poisson’s ratio ν , with E =1kPa, α=10 . TRE, HD95 and MSD are reported in mm . ∗ means the results are significantly different compared with ν =0.45 under p≤0.05 in a Wilcoxon test.
ν
0.35
0.40
0.45
0.499
HD95
9.34 ∗
10.11 ∗
11.08
13.39 ∗
MSD
2.91 ∗
3.12 ∗
3.47
4.45 ∗
SDLogJ
0.018 ∗
0.017
0.015
0.014 ∗
Appendix
Table S10: The average HD95, MSD and SDLogJ on AbdominalMRCT for different Poisson’s ratio ν , with E =1kPa, α=10 . HD95 and MSD are reported in mm . ∗ means the results are significantly different compared with ν=0.45 under p≤0.05 in a Wilcoxon test.
Init
ICP
GO-ICP
SemICP-rigid
Vis
Rot
HD95
MSD
TRE
HD95
MSD
TRE
HD95
MSD
TRE
HD95
MSD
TRE
5%
±5∘
13.60
6.31
14.17
43.31
29.53
59.28
9.39
4.02
8.58
9.24 ∗
3.94
8.23
±15∘
32.95
19.52
42.73
36.20
23.70
51.99
12.09
5.85
14.20
10.12 ∗
4.34 ∗
10.42 ∗
±30∘
61.06
43.25
80.26
42.35
28.89
61.36
27.97
18.15
40.61
13.82 ∗
6.27 ∗
19.75 ∗
10%
±5∘
13.74
6.22
13.19
42.49
28.24
55.73
9.27
3.96
8.35
9.23
3.95
8.20
±15∘
30.90
17.45
38.22
35.83
23.23
47.19
11.34
5.20
12.32
10.19
4.41
10.10
Appendix
Table S11: The average unidirectional HD95, unidirectional MSD and TRE of all labels for different rigid ICP methods with different visible ratios of the point cloud. The numbers are reported in mm . The best values are bold and the second best is underlined. ∗ means the results are significantly different compared with the best or the second-best experiments under p≤0.05 in a two-sample t-test.
Medical image registration aligns scans across time, patients, or imaging modalities to track anatomy and support diagnosis, treatment, and image-guided procedures.
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China