Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR

Hanyuan Zhang; Lucas He; Runlong He; Weixi Yi; Abdolrahim Kadkhodamohammadi; Danail Stoyanov; Brian R. Davidson; Evangelos B. Mazomenos

cs.CVFeb 19, 2026

Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR

Authors: Hanyuan Zhang, Lucas He, Runlong He, Weixi Yi, Abdolrahim Kadkhodamohammadi, Danail Stoyanov, Brian R. Davidson, Evangelos B. Mazomenos,

Organizations: UCL Hawkes Institute, University College London, London WC1E 6BT, UK · Unit for Lifelong Health and Ageing at UCL, University College London, London WC1E 7HB, UK · Medtronic plc., London, UK · Division of Surgery and Interventional Science, University College London, London WC1E 6BT, UK

Abstract

Augmented reality (AR) guidance in laparoscopic liver surgery requires accurate registration of preoperative 3D models to intraoperative 2D video, but remains challenging due to partial visibility, specularities, and tissue deformation. Existing methods often rely on contour-based rigid initialization and finite-element (FE) models for deformable registration, increasing modeling and engineering complexity. We present a depth-augmented, FE-free 3D--2D registration pipeline that combines robust rigid initialization with patient-specific non-rigid refinement. For rigid alignment, we adapt the RefineNet module of FoundationPose to laparoscopic liver scenes by using multi-class contour maps and monocular depth for relative pose refinement. For deformable alignment, we construct a patient-specific statistical deformation model from non-rigid ICP (NICP) correspondences and optimize pose and shape parameters using a coarse-to-fine L-BFGS-B strategy. On a public clinical laparoscopic liver dataset, the proposed method achieves a mean target registration error (TRE) of 14.73,mm under a controlled manual-contour setting designed to isolate registration performance. Ablation studies show that monocular depth improves rigid initialization over contour-only inputs, while tumor-mapping analysis indicates that good surface alignment does not necessarily translate into lower target localization error. On an external dataset without ground truth, the method produces visually plausible overlays for qualitative assessment. These results suggest that depth-augmented pose refinement and FE-free statistical deformation modeling provide a promising alternative to FE-based pipelines for controlled 3D--2D liver registration in surgical AR.

Figures & tables

Figure 1: The registration workflow diagram: The first stage adopts the RefineNet component from FoundationPose by Wen et al., with the inputs replaced by the liver’s multi-class contour and depth images. The output is a relative pose update, which is used to initialize the subsequent non-rigid optimization.

Figure 2: Patient-specific statistical deformation model constructed from NICP-registered liver shapes.

Method	Annotation	P1	P2	P3	P4	Avg	Avg w/o P2
MA	N/A	15.14	35.48	30.48	16.29	24.35	20.63
Sil-B [ 1 ]	Manual	8.25	37.25	28.40	15.83	22.43	17.47
LMR [ 11 ]	Manual	17.40	53.80	17.60	17.00	26.45	17.33
NM [ 15 ]	Manual	14.82	51.43	20.15	12.95	24.83	15.87
Opt-B [ 12 ]	Manual	14.87	N/A	22.40	7.23	N/A	14.83
ADeLiR [ 7 ]	Auto	12.45	46.44	17.13	11.20	21.81	13.60

Table 1: Registration performance in TRE (mm, lower is better). Best results are in bold . MA: manual initialization. For each patient, TRE is first averaged over all available frames using the public benchmark evaluation code. For the proposed method, we additionally report mean

\pm

std over 10 runs with different random initializations, where each run yields one case-level mean TRE.

Figure 3: Qualitative overlays on four cases from an independent laparoscopic liver dataset, comparing CMA–ES and L-BFGS-B optimization. In each group, the leftmost column shows the original intraoperative image, followed by CMA–ES and L-BFGS-B results.

(a) Input modality
Method	P1	P2	P3	P4	Mean
Contour	15.25	39.48	18.94	20.21	23.47
Contour+Mask	14.59	28.00	12.52	31.12	21.56
Contour+Depth	13.68	32.32	11.03	18.18	18.80
(b) Tumor mapping
FFD	9.70	31.54	7.65	10.08	14.74

Table 2: Ablation studies (TRE in mm, lower is better). Best in bold .

2D-3D Image Registration

See all 21 papers

Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints
Mar 2, 2026Ruize Cui, Jialun Pei, Haiqiao Wang +42D-3D Image Registration Laparoscopic Surgery
The Hong Kong Polytechnic University, Hong Kong, China · The Chinese University of Hong Kong, Hong Kong, China
Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy
Jul 20, 2026Jiaming Feng, Xukun Zhang, Shahid Farid +1Medical Image Registration Deformable Image Registration
AI in Medicine and Surgery Group, School of Computer Science, University of Leeds, LS2 9JT, Leeds, United Kingdom · Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pok Fu Lam, Hong Kong · Department of HPB and Transplant Surgery, St. James’s University Hospital, Leeds, United Kingdom
Warm-Started Reinforcement Learning for Iterative 3D/2D Liver Registration
Apr 11, 2026Hanyuan Zhang, Lucas He, Zijie Cheng +5Medical Image Registration Cross-Modal Image Registration
UCL Hawkes Institute, University College London, London, UK · Medtronic plc., London, UK

Mar 2, 2026J/K move · Enter open · S save

Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints

Ruize Cui, Jialun Pei, Haiqiao Wang, Jun Zhou+3

The Hong Kong Polytechnic University, Hong Kong, China · The Chinese University of Hong Kong, Hong Kong, China

2D-3D Image Registration Laparoscopic Surgery Medical Image Registration Cross-Modal Image Registration Deformable Image Registration

Open paper AskPDF arXiv

In laparoscopic liver surgery, augmented reality technology enhances intraoperative anatomical guidance by overlaying 3D liver models from preoperative CT/MRI onto laparoscopic 2D views. However, existing registration methods lack explicit modeling of reliable 2D-3D geometric correspondences supported by latent evidence, leading to limited interpretability and potentially unstable alignment in clinical scenarios. In this work, we introduce Land-Reg, a correspondence-driven deformable registration framework that explicitly learns latent-grounded 2D-3D landmark correspondences as an interpretable intermediate representation to bridge cross-modal alignment. For rigid registration, Land-Reg embraces a Cross-modal Latent Alignment module to map multi-modal features into a unified latent space. Further, an Uncertainty-enhanced Overlap Landmark Detector with similarity matching is proposed to robustly estimate explicit 2D-3D landmark correspondences. For non-rigid registration, we design a novel shape-constrained supervision strategy that anchors shape deformation to matched landmarks through reprojection consistency and incorporates local-isometric regularization to alleviate inherent 2D-3D depth ambiguity, while a rendered-mask alignment enforces global shape consistency. Experimental results on the P2ILF dataset demonstrate the superiority of our method on both rigid pose estimation and non-rigid deformation. Our code will be available at https://github.com/cuiruize/Land-Reg.

Abstract

2D-3D Image Registration

Explore similar work

Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints

Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy

Warm-Started Reinforcement Learning for Iterative 3D/2D Liver Registration