Landmark-Free and Scale-Blind Multi-View Registration of Preoperative Liver Models to Intraoperative Surfaces

Jiaming Feng; Gerardo Loza Galindo; Shahid Farid; Sharib Ali

cs.CVOct 16, 2026

Landmark-Free and Scale-Blind Multi-View Registration of Preoperative Liver Models to Intraoperative Surfaces

Authors: Jiaming Feng, Gerardo Loza Galindo, Shahid Farid, Sharib Ali

Organizations: AI in Medicine and Surgery Group, School of Computer Science, University of Leeds, Leeds LS2 9JT, United Kingdom · Department of HPB and Transplant Surgery, St. James’s University Hospital, Leeds, United Kingdom

Abstract

Augmented-reality guidance in laparoscopic liver resection requires aligning preoperative 3D liver models from CT/MRI with intraoperative surfaces. Monocular reconstruction in most cases cannot guarantee scale, albeit rigid-motion registration requires scale as an input. Our proposed framework ``SimCompose'' estimates the full similarity transformation $\Sim$ , including scale, without relying on landmarks and applies the same transformation across multiple views with known relative geometry. However, reliable detection of registration failures remains challenging in surgical settings, where ground-truth alignment is generally unavailable. For this, we propose a geometry-based selector that evaluates candidate transformations without requiring ground truth. Large disagreement among these transformations is used to flag potentially unreliable registrations. Finally, we propose to use a non-rigid refinement based on a low-dimensional deformation model from Neo-Hookean simulations in the Simulation Open Framework Architecture (SOFA), without additional network training. SimCompose is trained exclusively on synthetic partial views. Its encoder processes point clouds normalised to a unit root-mean-square radius, without access to metric scale, and filters each view before multi-view fusion. Evaluation on unseen synthetic data (300 single-view pairs per run; 100 three-view registrations) shows that multi-view integration and candidate selection increase the success rate under stringent $\Sim$ accuracy criteria from 38.4% for a single-view, single-method configuration to 77%. Existing learning-based baselines do not satisfy these criteria even when provided with the exact scale. On five in-vivo patients with 92 keyframes, the rigid registration stage reduces surface error more than three-fold relative to learning-based baselines with external scale information.

Figures & tables

Fig. 1: The registration task. The complete preoperative liver surface is blue; partial laparoscopic observations with surrounding clutter are orange. We estimate one scale, rotation and translation to align the model to observations sharing a coordinate frame and relative scale, without landmarks or supplied metric scale. Non-rigid refinement is optional. Right: surgical overlay, with the transformed model in magenta and the annotated liver boundary in green.

Fig. 2: SimCompose. The blue cloud $\mathcal{M}$ samples the preoperative liver surface; the orange cloud $\mathcal{Q}_{k}$ is the observation from view $k$ . (A) Each cloud is encoded independently to give descriptors $\mathbf{d}$ and probabilities $o$ (model visibility or observation liver membership) for its sampled points. Descriptor colours show a principal-component projection; blue tiles and gold dots denote descriptors and probabilities. (B) Point indices retained by per-view gating, a threshold on each view’s probabilities $o$ , select the original coordinates and descriptors for fusion into the gated observation $\mathcal{Q}_{\mathrm{f}}$ . (C) Candidate similarity transformations $T_{a}$ are estimated, with $a$ indexing candidates. (D) The geometric score $\mathcal{G}$ , a trimmed observation-to-model surface distance, selects $\hat{T}$ . (E) With that transformation fixed, deformation is fitted in a simulation-derived subspace. Lettered ports connect reused outputs; repeated cloud symbols denote the same input coordinates.

		In vivo		Blender
Method	Scale	Dice $\uparrow$	CD $\downarrow$ (mm)	Dice $\uparrow$	CD $\downarrow$ (%)
SimCompose (ours)	blind	79.6 $\pm$ 6.0	12.1 $\pm$ 1.8	73.6	7.8
Self-P2IR [ 8 ] , rigid core †	ext.-ratio	69.3 $\pm$ 6.2	45.9 $\pm$ 11.6	66.1	5.1
Lepard [ 12 ]	ext.-ratio	66.0 $\pm$ 5.2	44.5 $\pm$ 5.3	62.4	5.5
GeoTransformer [ 11 ]	ext.-ratio	56.3 $\pm$ 4.4	74.4 $\pm$ 18.8	48.3	9.5
GeoTr., retrained r	ext.-ratio	63.8	63.4	—	—

TABLE I: Rigid registration on in-vivo and Blender data.

Configuration	Scale	Views	Pass rate (%) $\uparrow$
Self-P2IR / Lepard / GeoTr. / PREDATOR a	ext.-ratio	1	$0$
GeoTr., retrained r	ext.-ratio	1	$0$
Chamfer- $\mathrm{Sim}(3)$ (ref.)	blind	1	$\mathit{2.7}$
FPFH/TEASER++/SC 2 -PCR [ 31 , 37 , 38 ]	blind	1	$\leq 8$
GeoTr. $+$ our solver b	pre-scaled	1	$15.3$
Lepard $+$ our solver b	pre-scaled	1	$15.3$

TABLE II: Registration accuracy on held-out synthetic geometry.

Method	Scale	CD r $\downarrow$ (mm)	CD nr $\downarrow$ (mm)	Dice $\uparrow$	Def. (%)
SimCompose (ours)	blind	14.2	9.9	83.4	5.0
NDP [ 30 ] ‡	blind	88.3	6.0	89.0	34.3
Lepard [ 12 ]	ext.-ratio	45.4	10.7	83.3	12.1
Self-P2IR [ 8 ] , rigid core †	ext.-ratio	48.8	11.2	82.9	12.8
Self-P2IR, complete system e	native	71.7	67.0	29.0	6.6
GeoTransformer [ 11 ]	ext.-ratio	82.5	12.9	73.0	12.5

TABLE III: Non-rigid refinement on in-vivo data.

Fig. 3: Quantitative analyses. (a) CD is the median of five patient medians, in millimetres at the reference scale. Dotted lines show CD at per-patient oracle multipliers:

33.7/43.2

mm for GeoTransformer/Self-P2IR; SimCompose estimates scale internally. Squares use shared multipliers

1.0/1.1

. (b) Three-view fusion uses the full synthetic candidate set and

100

pairs per point; clean strict success is

79\%

, as in Table IV . Error bars show two-sided

95\%

Clopper–Pearson (exact binomial) intervals; grey points and lines are single-view references on the same pairs. The

2\times/5\times

criteria multiply all strict thresholds in Section V-A . (c) Curves show within-sample ranking at nominal coverage, the fraction of registrations retained after screening; stars show achieved coverage under leave-one-patient-out (LOPO) calibration. Synthetic events are strict-criterion failures; in-vivo events are rigid-stage collapses. Filled/open stars denote full/two-method sets.

Fig. 4: Rigid-stage comparisons. Ours : SimCompose; GeoTr.: GeoTransformer. (a) P2I-LReg keyframes; Pnn-fm denotes patient nn, frame m. (b) Blender cases: static organ, zero true deformation. Columns: input, overlays, and 3D registration. Overlay models are magenta; boundaries are green in (a). Registration models are grey; observations use viridis residual colours, dark $=$ low. Each row shares a camera and colour scale, in mm at right; maxima are the four-method 95th percentiles, with larger residuals saturated. Non-rigid stage: Fig. 5 .

Fig. 5: Non-rigid refinement of P01-f10. Overlays and 3D registrations share a camera and colour scale in mm; colours follow Fig. 4 . Mean residual decreases from $10.3$ to $6.7$ mm. Bottom: an $18$ mm-thick slab parallel to the camera $y$ – $z$ plane, centred on the observed point whose rigid residual is closest to the $97$ th percentile. Observations are light discs, rigid points blue squares, and non-rigid points red triangles. Fitting narrows the gap between observations and model points.

Configuration	Pass rate (%) $\uparrow$	$\Delta$ (pp)
Full model (fusion $+$ methods $+$ selection)	$\mathbf{79}$	—
Oracle (upper bound)	$\mathit{89}$	$\mathit{+10}$
w/o candidate selection (best fixed method)	$70$	$-9$
w/o multiple methods (MH only)	$59$	$-20$
w/o multi-view fusion (combined candidates, sv )	$41.3$	$-37.7$
w/o fusion and multiple methods ( sv )	$38.4$	$-40.6$

TABLE IV: Ablation of registration components and supplied scale.

Medical Image Registration

Medical image registration aligns scans across time, patients, or imaging modalities to track anatomy and support diagnosis, treatment, and image-guided procedures.

See all 95 papers

Jul 20, 2026

Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy

Accurate 3D--2D liver registration, which aligns preoperative 3D models to partial, view-dependent intraoperative surface observations, is critical for AR-guided laparoscopic surgery but remains challenging due to severe occlusion, limited visibility, and the lack of 3D ground-truth supervision. Existing landmark-free approaches perform partial-to-complete geometric alignment, yet robust self-supervision under extreme partial visibility remains difficult. We propose Vis2Reg, a visibility-aware registration framework that explicitly constrains deformation using mask-consistent visible regions. We introduce a visibility-aware self-supervision that derives a visible-domain 3D supervision signal from intraoperative masks, enabled by differentiable point rasterization and mask-guided back-projection. This formulation improves robustness under severe occlusion while maintaining fully self-supervised learning. Vis2Reg combines a robust geometric rigid initialization module with an implicit neural deformation field for stable alignment. Vis2Reg achieves a Dice score of 92.6% and a Chamfer Distance of 1.43 mm on real intraoperative datasets, with 111 ms per-frame inference time, demonstrating both accuracy and practical efficiency.

Jiaming Feng, Xukun Zhang, Shahid Farid +1

AI in Medicine and Surgery Group, School of Computer Science, University of Leeds, LS2 9JT, Leeds, United Kingdom · Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pok Fu Lam, Hong Kong · Department of HPB and Transplant Surgery, St. James’s University Hospital, Leeds, United Kingdom

Medical Image Registration Deformable Image Registration 2D-3D Image Registration Laparoscopic Surgery

Ask PDF

Mar 2, 2026

Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints

In laparoscopic liver surgery, augmented reality technology enhances intraoperative anatomical guidance by overlaying 3D liver models from preoperative CT/MRI onto laparoscopic 2D views. However, existing registration methods lack explicit modeling of reliable 2D-3D geometric correspondences supported by latent evidence, leading to limited interpretability and potentially unstable alignment in clinical scenarios. In this work, we introduce Land-Reg, a correspondence-driven deformable registration framework that explicitly learns latent-grounded 2D-3D landmark correspondences as an interpretable intermediate representation to bridge cross-modal alignment. For rigid registration, Land-Reg embraces a Cross-modal Latent Alignment module to map multi-modal features into a unified latent space. Further, an Uncertainty-enhanced Overlap Landmark Detector with similarity matching is proposed to robustly estimate explicit 2D-3D landmark correspondences. For non-rigid registration, we design a novel shape-constrained supervision strategy that anchors shape deformation to matched landmarks through reprojection consistency and incorporates local-isometric regularization to alleviate inherent 2D-3D depth ambiguity, while a rendered-mask alignment enforces global shape consistency. Experimental results on the P2ILF dataset demonstrate the superiority of our method on both rigid pose estimation and non-rigid deformation. Our code will be available at https://github.com/cuiruize/Land-Reg.

Ruize Cui, Jialun Pei, Haiqiao Wang +4

The Hong Kong Polytechnic University, Hong Kong, China · The Chinese University of Hong Kong, Hong Kong, China

2D-3D Image Registration Laparoscopic Surgery Medical Image Registration Cross-Modal Image Registration Deformable Image Registration

Ask PDF

Feb 19, 2026

Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR

Augmented reality (AR) guidance in laparoscopic liver surgery requires accurate registration of preoperative 3D models to intraoperative 2D video, but remains challenging due to partial visibility, specularities, and tissue deformation. Existing methods often rely on contour-based rigid initialization and finite-element (FE) models for deformable registration, increasing modeling and engineering complexity. We present a depth-augmented, FE-free 3D--2D registration pipeline that combines robust rigid initialization with patient-specific non-rigid refinement. For rigid alignment, we adapt the RefineNet module of FoundationPose to laparoscopic liver scenes by using multi-class contour maps and monocular depth for relative pose refinement. For deformable alignment, we construct a patient-specific statistical deformation model from non-rigid ICP (NICP) correspondences and optimize pose and shape parameters using a coarse-to-fine L-BFGS-B strategy. On a public clinical laparoscopic liver dataset, the proposed method achieves a mean target registration error (TRE) of 14.73,mm under a controlled manual-contour setting designed to isolate registration performance. Ablation studies show that monocular depth improves rigid initialization over contour-only inputs, while tumor-mapping analysis indicates that good surface alignment does not necessarily translate into lower target localization error. On an external dataset without ground truth, the method produces visually plausible overlays for qualitative assessment. These results suggest that depth-augmented pose refinement and FE-free statistical deformation modeling provide a promising alternative to FE-based pipelines for controlled 3D--2D liver registration in surgical AR.

Hanyuan Zhang, Lucas He, Runlong He +6

UCL Hawkes Institute, University College London, London WC1E 6BT, UK · Unit for Lifelong Health and Ageing at UCL, University College London, London WC1E 7HB, UK · Medtronic plc., London, UK +1

2D-3D Image Registration AR in Surgery Medical Image Registration Deformable Image Registration Laparoscopic Surgery

Ask PDF

Abstract

Medical Image Registration

Explore similar work

Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy

Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints

Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR