Landmark-Free and Scale-Blind Multi-View Registration of Preoperative Liver Models to Intraoperative Surfaces

Jiaming Feng; Gerardo Loza Galindo; Shahid Farid; Sharib Ali

cs.CVOct 16, 2026

Landmark-Free and Scale-Blind Multi-View Registration of Preoperative Liver Models to Intraoperative Surfaces

Authors: Jiaming Feng, Gerardo Loza Galindo, Shahid Farid, Sharib Ali

Organizations: AI in Medicine and Surgery Group, School of Computer Science, University of Leeds, Leeds LS2 9JT, United Kingdom · Department of HPB and Transplant Surgery, St. James’s University Hospital, Leeds, United Kingdom

Abstract

Augmented-reality guidance in laparoscopic liver resection requires aligning preoperative 3D liver models from CT/MRI with intraoperative surfaces. Monocular reconstruction in most cases cannot guarantee scale, albeit rigid-motion registration requires scale as an input. Our proposed framework ``SimCompose'' estimates the full similarity transformation $\Sim$ , including scale, without relying on landmarks and applies the same transformation across multiple views with known relative geometry. However, reliable detection of registration failures remains challenging in surgical settings, where ground-truth alignment is generally unavailable. For this, we propose a geometry-based selector that evaluates candidate transformations without requiring ground truth. Large disagreement among these transformations is used to flag potentially unreliable registrations. Finally, we propose to use a non-rigid refinement based on a low-dimensional deformation model from Neo-Hookean simulations in the Simulation Open Framework Architecture (SOFA), without additional network training. SimCompose is trained exclusively on synthetic partial views. Its encoder processes point clouds normalised to a unit root-mean-square radius, without access to metric scale, and filters each view before multi-view fusion. Evaluation on unseen synthetic data (300 single-view pairs per run; 100 three-view registrations) shows that multi-view integration and candidate selection increase the success rate under stringent $\Sim$ accuracy criteria from 38.4% for a single-view, single-method configuration to 77%. Existing learning-based baselines do not satisfy these criteria even when provided with the exact scale. On five in-vivo patients with 92 keyframes, the rigid registration stage reduces surface error more than three-fold relative to learning-based baselines with external scale information.

Figures & tables

Fig. 1: The registration task. The complete preoperative liver surface is blue; partial laparoscopic observations with surrounding clutter are orange. We estimate one scale, rotation and translation to align the model to observations sharing a coordinate frame and relative scale, without landmarks or supplied metric scale. Non-rigid refinement is optional. Right: surgical overlay, with the transformed model in magenta and the annotated liver boundary in green.

Fig. 2: SimCompose. The blue cloud $\mathcal{M}$ samples the preoperative liver surface; the orange cloud $\mathcal{Q}_{k}$ is the observation from view $k$ . (A) Each cloud is encoded independently to give descriptors $\mathbf{d}$ and probabilities $o$ (model visibility or observation liver membership) for its sampled points. Descriptor colours show a principal-component projection; blue tiles and gold dots denote descriptors and probabilities. (B) Point indices retained by per-view gating, a threshold on each view’s probabilities $o$ , select the original coordinates and descriptors for fusion into the gated observation $\mathcal{Q}_{\mathrm{f}}$ . (C) Candidate similarity transformations $T_{a}$ are estimated, with $a$ indexing candidates. (D) The geometric score $\mathcal{G}$ , a trimmed observation-to-model surface distance, selects $\hat{T}$ . (E) With that transformation fixed, deformation is fitted in a simulation-derived subspace. Lettered ports connect reused outputs; repeated cloud symbols denote the same input coordinates.

		In vivo		Blender
Method	Scale	Dice $\uparrow$	CD $\downarrow$ (mm)	Dice $\uparrow$	CD $\downarrow$ (%)
SimCompose (ours)	blind	79.6 $\pm$ 6.0	12.1 $\pm$ 1.8	73.6	7.8
Self-P2IR [ 8 ] , rigid core †	ext.-ratio	69.3 $\pm$ 6.2	45.9 $\pm$ 11.6	66.1	5.1
Lepard [ 12 ]	ext.-ratio	66.0 $\pm$ 5.2	44.5 $\pm$ 5.3	62.4	5.5
GeoTransformer [ 11 ]	ext.-ratio	56.3 $\pm$ 4.4	74.4 $\pm$ 18.8	48.3	9.5
GeoTr., retrained r	ext.-ratio	63.8	63.4	—	—

TABLE I: Rigid registration on in-vivo and Blender data.

Configuration	Scale	Views	Pass rate (%) $\uparrow$
Self-P2IR / Lepard / GeoTr. / PREDATOR a	ext.-ratio	1	$0$
GeoTr., retrained r	ext.-ratio	1	$0$
Chamfer- $\mathrm{Sim}(3)$ (ref.)	blind	1	$\mathit{2.7}$
FPFH/TEASER++/SC 2 -PCR [ 31 , 37 , 38 ]	blind	1	$\leq 8$
GeoTr. $+$ our solver b	pre-scaled	1	$15.3$
Lepard $+$ our solver b	pre-scaled	1	$15.3$

TABLE II: Registration accuracy on held-out synthetic geometry.

Method	Scale	CD r $\downarrow$ (mm)	CD nr $\downarrow$ (mm)	Dice $\uparrow$	Def. (%)
SimCompose (ours)	blind	14.2	9.9	83.4	5.0
NDP [ 30 ] ‡	blind	88.3	6.0	89.0	34.3
Lepard [ 12 ]	ext.-ratio	45.4	10.7	83.3	12.1
Self-P2IR [ 8 ] , rigid core †	ext.-ratio	48.8	11.2	82.9	12.8
Self-P2IR, complete system e	native	71.7	67.0	29.0	6.6
GeoTransformer [ 11 ]	ext.-ratio	82.5	12.9	73.0	12.5

TABLE III: Non-rigid refinement on in-vivo data.

Fig. 3: Quantitative analyses. (a) CD is the median of five patient medians, in millimetres at the reference scale. Dotted lines show CD at per-patient oracle multipliers:

33.7/43.2

mm for GeoTransformer/Self-P2IR; SimCompose estimates scale internally. Squares use shared multipliers

1.0/1.1

. (b) Three-view fusion uses the full synthetic candidate set and

100

pairs per point; clean strict success is

79\%

, as in Table IV . Error bars show two-sided

95\%

Clopper–Pearson (exact binomial) intervals; grey points and lines are single-view references on the same pairs. The

2\times/5\times

criteria multiply all strict thresholds in Section V-A . (c) Curves show within-sample ranking at nominal coverage, the fraction of registrations retained after screening; stars show achieved coverage under leave-one-patient-out (LOPO) calibration. Synthetic events are strict-criterion failures; in-vivo events are rigid-stage collapses. Filled/open stars denote full/two-method sets.

Fig. 4: Rigid-stage comparisons. Ours : SimCompose; GeoTr.: GeoTransformer. (a) P2I-LReg keyframes; Pnn-fm denotes patient nn, frame m. (b) Blender cases: static organ, zero true deformation. Columns: input, overlays, and 3D registration. Overlay models are magenta; boundaries are green in (a). Registration models are grey; observations use viridis residual colours, dark $=$ low. Each row shares a camera and colour scale, in mm at right; maxima are the four-method 95th percentiles, with larger residuals saturated. Non-rigid stage: Fig. 5 .

Fig. 5: Non-rigid refinement of P01-f10. Overlays and 3D registrations share a camera and colour scale in mm; colours follow Fig. 4 . Mean residual decreases from $10.3$ to $6.7$ mm. Bottom: an $18$ mm-thick slab parallel to the camera $y$ – $z$ plane, centred on the observed point whose rigid residual is closest to the $97$ th percentile. Observations are light discs, rigid points blue squares, and non-rigid points red triangles. Fitting narrows the gap between observations and model points.

Configuration	Pass rate (%) $\uparrow$	$\Delta$ (pp)
Full model (fusion $+$ methods $+$ selection)	$\mathbf{79}$	—
Oracle (upper bound)	$\mathit{89}$	$\mathit{+10}$
w/o candidate selection (best fixed method)	$70$	$-9$
w/o multiple methods (MH only)	$59$	$-20$
w/o multi-view fusion (combined candidates, sv )	$41.3$	$-37.7$
w/o fusion and multiple methods ( sv )	$38.4$	$-40.6$

TABLE IV: Ablation of registration components and supplied scale.

Medical Image Registration

Medical image registration aligns scans across time, patients, or imaging modalities to track anatomy and support diagnosis, treatment, and image-guided procedures.

See all 96 papers

Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy
Jul 20, 2026Jiaming Feng, Xukun Zhang, Shahid Farid +1Medical Image Registration Deformable Image Registration
AI in Medicine and Surgery Group, School of Computer Science, University of Leeds, LS2 9JT, Leeds, United Kingdom · Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pok Fu Lam, Hong Kong · Department of HPB and Transplant Surgery, St. James’s University Hospital, Leeds, United Kingdom
Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints
Mar 2, 2026Ruize Cui, Jialun Pei, Haiqiao Wang +42D-3D Image Registration Laparoscopic Surgery
The Hong Kong Polytechnic University, Hong Kong, China · The Chinese University of Hong Kong, Hong Kong, China
Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR
Feb 19, 2026Hanyuan Zhang, Lucas He, Runlong He +62D-3D Image Registration AR in Surgery
UCL Hawkes Institute, University College London, London WC1E 6BT, UK · Unit for Lifelong Health and Ageing at UCL, University College London, London WC1E 7HB, UK · Medtronic plc., London, UK +1

Jul 20, 2026J/K move · Enter open · S save

Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy

Jiaming Feng, Xukun Zhang, Shahid Farid, Sharib Ali

AI in Medicine and Surgery Group, School of Computer Science, University of Leeds, LS2 9JT, Leeds, United Kingdom · Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pok Fu Lam, Hong Kong · Department of HPB and Transplant Surgery, St. James’s University Hospital, Leeds, United Kingdom

Medical Image Registration Deformable Image Registration 2D-3D Image Registration Laparoscopic Surgery

Open paper AskPDF arXiv

Accurate 3D--2D liver registration, which aligns preoperative 3D models to partial, view-dependent intraoperative surface observations, is critical for AR-guided laparoscopic surgery but remains challenging due to severe occlusion, limited visibility, and the lack of 3D ground-truth supervision. Existing landmark-free approaches perform partial-to-complete geometric alignment, yet robust self-supervision under extreme partial visibility remains difficult. We propose Vis2Reg, a visibility-aware registration framework that explicitly constrains deformation using mask-consistent visible regions. We introduce a visibility-aware self-supervision that derives a visible-domain 3D supervision signal from intraoperative masks, enabled by differentiable point rasterization and mask-guided back-projection. This formulation improves robustness under severe occlusion while maintaining fully self-supervised learning. Vis2Reg combines a robust geometric rigid initialization module with an implicit neural deformation field for stable alignment. Vis2Reg achieves a Dice score of 92.6% and a Chamfer Distance of 1.43 mm on real intraoperative datasets, with 111 ms per-frame inference time, demonstrating both accuracy and practical efficiency.

Abstract

Medical Image Registration

Explore similar work

Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy

Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints

Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR