Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery

Muhammad Asad; Emanuele Colleoni; Pritesh Mehta; Nicolas Toussaint; Ricardo Sanchez-Matilla; Maria Robu; Faisal Bashir; Rahim Mohammadi

cs.CVMar 3, 2026

Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery

Authors: Muhammad Asad, Emanuele Colleoni, Pritesh Mehta, Nicolas Toussaint, Ricardo Sanchez-Matilla, Maria Robu, Faisal Bashir, Rahim Mohammadi, +2 more

Organizations: Medtronic Surgical, London, UK. · UCL Hawkes Institute, University College London, London,UK.

Abstract

Purpose: Monocular depth estimation (MDE) is vital for scene understanding in minimally invasive surgery (MIS). However, endoscopic video sequences are often contaminated by smoke, specular reflections, blur, and occlusions, limiting the accuracy of MDE models. In addition, current MDE models do not output depth confidence, which could be a valuable tool for improving their clinical reliability. Methods: We propose a novel confidence-aware MDE framework featuring three significant contributions: (i) Calibrated confidence targets: an ensemble of fine-tuned stereo matching models is used to capture disparity variance into pixel-wise confidence probabilities; (ii) Confidence-aware loss: Baseline MDE models are optimized with confidence-aware loss functions, utilizing pixel-wise confidence probabilities such that reliable pixels dominate training; and (iii) Inference-time confidence: a confidence estimation head is proposed with two convolution layers to predict per-pixel confidence at inference, enabling assessment of depth reliability. Results: Comprehensive experimental validation across internal and public datasets demonstrates that our framework improves depth estimation accuracy and can robustly quantify the prediction's confidence. On the internal clinical endoscopic dataset (StereoKP), we improve dense depth estimation accuracy by ~8% as compared to the baseline model. Conclusion: Our confidence-aware framework enables improved accuracy of MDE models in MIS, addressing challenges posed by noise and artifacts in pre-clinical and clinical data, and allows MDE models to provide confidence maps that may be used to improve their reliability for clinical applications.

Figures & tables

Figure 1: Factors such as image acquisition noise, lens contamination/smudges, blur, occluded views, and lighting issues in MIS datasets contribute towards unreliable depth data that impacts MDE models trained on MIS datasets. Depth confidence can be used to identify noise-free regions in confidence-aware training. Examples from the StereoKP dataset.

Figure 2: The proposed confidence-aware training methods. Our proposed blocks are shown in blue. Method (a) is used to add depth confidence to a depth dataset, whereas (b) shows our confidence-aware MDE training framework.

Dataset	Description	Setting	Modalities	Total Frames
StereoKP	Internal, 57 pre-clinical + 6 clinical videos	Pre-clinical + Clinical	Metric depth using Xu et al. (2023) + manually annotated instrument keypoints	10,491
MicroCT-SE	Internal, surrogate and two ex-vivo liver	Lab	Metric depth from MicroCT	16,775
MicroCT-PK	Internal, ex-vivo porcine kidney	Lab	Metric depth using from MicroCT	4046
Hamlyn Recasens et al. (2021)	Public in-vivo stereo endoscope videos Recasens et al. (2021)	Clinical	Metric depth using stereo-matching Recasens et al. (2021)	92,673
DaVinci Ye et al. (2017)	Public in-vivo stereo endoscope videos Ye et al. (2017)	Clinical	Metric depth using Xu et al. (2023)	41,433

Table 1: Summary of stereo endoscopic datasets utilized for experimental validation of proposed confidence-aware MDE methods.

Figure 3: Comparison of frames from different datasets used in this work. StereoKP, Hamlyn Recasens et al. (2021) and DaVinci Ye et al. (2017) are from clinical settings whereas both MicroCT-SE and MicroCT-PK are from lab settings. StereoKP and MicroCT datasets are internal wheres Hamlyn and DaVinci are public datasets.

$\boldsymbol{\sigma}$	ARE (%) $\downarrow$	$\boldsymbol{\delta_{1}<1.25}$ (%) $\uparrow$
0.2	9.58	92.68
0.5	8.75	94.00
0.7	8.86	94.14
1.0	9.04	93.40

Table 2: Sensitivity analysis of the hyperparameter

\sigma

in the confidence probability equation (Eq. 1 ), showing the effect on ARE and

\delta_{1}<1.25

metrics using the StereoKP dataset.

CH	CAL	MAE (mm) $\downarrow$	Acc @ 2mm (%) $\uparrow$	ARE (%) $\downarrow$	$\boldsymbol{\delta_{1}<1.25}$ (%) $\uparrow$
✗	✗	2.04	72.4	12.41	85.83
✗	✓	1.76	78.2	8.99	93.62
✓	✗	1.81	76.5	9.20	93.63
✓	✓	1.79	77.9	8.86	94.14

Table 3: Ablation study showing the impact of Confidence Head (CH) and Confidence-aware Loss (CAL) on model performance across four configurations. Results demonstrate that both components, individually and combined, improve MAE, accuracy at 2mm, ARE, and

\delta_{1}<1.25

over the baseline. Best results are shown in bold .

Dataset	DAv1-B Baseline		DAv1-B CA (proposed)
	ARE $\downarrow$	$\delta_{1}$ $\uparrow$	ARE $\downarrow$	$\delta_{1}$ $\uparrow$
StereoKP	12.41	85.83	8.86	94.14
MicroCT-SE	5.69	99.62	5.04	99.60
MicroCT-PK	18.64	96.98	14.74	97.27
Hamlyn Recasens et al. (2021)	13.97	80.00	13.74	80.10
DaVinci Ye et al. (2017)	8.28	96.60	8.03	96.80

Table 4: Evaluation metrics for baseline and proposed methods across datasets. Bold values shows best results for a given metric and dataset.

Figure 4: Qualitative results on StereoKP dataset, shows ground truth and predicted depth from DAv1-B baseline and DAv1-B CA (proposed) models, additionally showing predicted confidence from CA model. Arrows show regions where CA model performed particularly better as compared to baseline.

Explore similar work

EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images
Aug 25, 2025Xinning Yao, Bo Liu, Bojian Li +3Medical Imaging Monocular Depth Estimation
Image Processing Center, Beihang University, Beijing, 100191, China. · State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology, Beijing 102206, China.
Boosting Generalizable Depth Estimation in Endoscopy by Mixture of Lightweight Experts and Intrinsic Image Alignment
Aug 1, 2026Liangjing Shao, Beilei Cui, Yiming Huang +2Endoscopy Depth Estimation
Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · Shenzhen Loop Area Institute, China
Augmenting Real-time Visualization in Minimally Invasive Surgery with Particle Filtering and Depth-Aware Occlusion Handling
Oct 13, 2026Anuska Sarkar, Partha Acharya, Vishnu Nandury +2Occlusion Reasoning Surgical Video Understanding
School of Medical Science and Technology, Indian Institute of Technology Kharagpur, Kharagpur 721302 India · Computer Science Department, Arizona State University, Tempe, Arizona 85287 USA · Narayana Hospital, Kolkata 700107 India

Aug 25, 2025J/K move · Enter open · S save

EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images

Xinning Yao, Bo Liu, Bojian Li, Jingjing Wang+2

Image Processing Center, Beihang University, Beijing, 100191, China. · State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology, Beijing 102206, China.

Medical Imaging Monocular Depth Estimation Vision Foundation Model Adaptation

Open paper AskPDF arXiv

Depth estimation is a foundational component for 3D reconstruction in minimally invasive endoscopic surgeries. However, existing monocular depth estimation techniques often exhibit limited performance to the varying illumination and complex textures of the surgical environment. While applying foundation models offers a promising approach to enhance the depth estimation performance, the domain gap between the natural images used for pre-training and the target endoscopic images leads to significant semantic perception deficiencies. In this study, EndoUFM is introduced as an unsupervised monocular depth estimation framework that innovatively Utilizes dual Foundation Models for Endoscopic images, thereby enhancing the depth estimation performance by leveraging the powerful pre-learned priors. The framework features a novel adaptive fine-tuning strategy that incorporates Random Vector Low-Rank Adaptation (RVLoRA) to enhance model adaptability, and a Residual block based on Depthwise Separable Convolution (Res-DSC) to improve the capture of fine-grained local features. A mask-guided smoothness loss is also introduced to enforce depth consistency within anatomical structures. Extensive experiments on the SCARED, Hamlyn, SERV-CT, and EndoNeRF datasets confirm that our method achieves state-of-the-art performance while maintaining an efficient model size. This work contributes to augmenting surgeons' spatial perception during minimally invasive procedures, thereby enhancing surgical precision and safety, with crucial implications for augmented reality and navigation systems. Our code is available at https://github.com/RealMindyY/EndoUFM.

Abstract

Explore similar work

EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images

Boosting Generalizable Depth Estimation in Endoscopy by Mixture of Lightweight Experts and Intrinsic Image Alignment

Augmenting Real-time Visualization in Minimally Invasive Surgery with Particle Filtering and Depth-Aware Occlusion Handling