Multi-Modal Monocular Endoscopic Depth and Pose Estimation with Edge-Guided Self-Supervision

Xinwei Ju; Rema Daher; Danail Stoyanov; Sophia Bano; Francisco Vasconcelos

cs.CVFeb 19, 2026

Multi-Modal Monocular Endoscopic Depth and Pose Estimation with Edge-Guided Self-Supervision

Authors: Xinwei Ju, Rema Daher, Danail Stoyanov, Sophia Bano, Francisco Vasconcelos

Organizations: UCL Hawkes Institute, Department of Computer Science University College London, London, UK

Abstract

Monocular depth and pose estimation play an important role in the development of colonoscopy-assisted navigation, as they enable improved screening by reducing blind spots, minimizing the risk of missed or recurrent lesions, and lowering the likelihood of incomplete examinations. However, this task remains challenging due to the presence of texture-less surfaces, complex illumination patterns, deformation, and a lack of in-vivo datasets with reliable ground truth. In this paper, we propose PRISM (Pose-Refinement with Intrinsic Shading and edge Maps), a self-supervised learning framework that leverages anatomical and illumination priors to guide geometric learning. Our approach uniquely incorporates edge detection and luminance decoupling for structural guidance. Specifically, edge maps are derived using a learning-based edge detector (e.g., DexiNed or HED) trained to capture thin and high-frequency boundaries, while luminance decoupling is obtained through an intrinsic decomposition module that separates shading and reflectance, enabling the model to exploit shading cues for depth estimation. Experimental results on multiple real and synthetic datasets demonstrate state-of-the-art performance. We further conduct a thorough ablation study on training data selection to establish best practices for pose and depth estimation in colonoscopy. This analysis yields two practical insights: (1) self-supervised training on real-world data outperforms supervised training on realistic phantom data, underscoring the superiority of domain realism over ground truth availability; and (2) video frame rate is an extremely important factor for model performance, where dataset-specific video frame sampling is necessary for generating high quality training data.

Figures & tables

Figure 1: An overview of our proposed self-supervised monocular depth estimation framework. Our key contribution is to explicitly guide the DepthNet with priors from two parallel branches: a LumNet, inspired by IID-SfMLearner, to extract an illumination-aware luminance map ( $L_{t}$ ), and an EdgeNet, based on DexiNed, to extract a high-fidelity structural edge map ( $E_{t}$ ). The inputs to the DepthNet are the concatenation of the raw image $I_{t}$ , the luminance map $L_{t}$ , and the edge map $E_{t}$ .

Model	RMSE ↓	LogRMSE ↓	MAE ↓	MedAE ↓	AbsRel ↓	SqRel ↓	ATE ↓
Monodepth2 [ 11 ]	6.3138	0.1841	4.3699	3.1279	0.1501	1.0168	2.7025
MonoViT [ 12 ]	6.8544	0.1789	4.6146	3.0384	0.1473	1.0958	2.4298
AF-SfM _pretrained [ 19 ]	17.3395	0.5216	12.0918	8.2881	0.4126	7.2491	-
AF-SfM _finetuned [ 19 ]	9.9636	0.2658	6.3069	3.4845	0.2165	3.7232	4.4841
IID-SfM _pretrained [ 8 ]	17.6461	0.5269	12.3394	8.4590	0.4178	7.4915	-
IID-SfM _finetuned [ 8 ]	8.6977	0.2552	5.6891	3.5772	0.1996	1.9452	3.1358

Table 1: Quantitative comparison of depth estimation on the C3VD test set for models trained with Hyper-Kvasir. Best results are in bold , second best are underlined .

Figure 2: Qualitative evaluation of generalization on real-world endoscopic video frames with challenging improvements boxed in red.

Figure 3: Qualitative comparison of estimated camera trajectories across four models on three representative sequences from the C3VD dataset. Each 3D plot shows the predicted path color-coded by temporal order.

Models	Pose Edge	Pose Lum	Depth Edge	Depth Lum	RMSE ↓	Log RMSE ↓	MAE ↓	MedAE ↓	Abs Rel ↓	Sq Rel ↓	$\delta<1.25$ ↑	$\delta<1.25^{2}$ ↑	$\delta<1.25^{3}$ ↑
Baseline	–	–	–	–	6.2258	0.1845	4.3176	3.0993	0.1523	1.0360	0.7727	0.9741	0.9961
PE	✓	–	–	–	6.2501	0.1824	4.3260	2.9790	0.1505	1.0610	0.7852	0.9665	0.9945
PL	–	✓	–	–	6.7658	0.2072	4.8068	3.5452	0.1719	1.2877	0.7318	0.9489	0.9910
DE	–	–	✓	–	6.6625	0.2111	4.7625	3.4991	0.1755	1.3095	0.7338	0.9415	0.9891
DL	–	–	–	✓	6.7540	0.2005	4.7099	3.4705	0.1649	1.2652	0.7468	0.9576	0.9951
DEPL	–	✓	✓	–	6.9001	0.2105	4.9384	3.5381	0.1739	1.3404	0.7274	0.9483	0.9896

Table 2: Quantitative comparison of depth estimation with models trained on Hyper-Kvasir and tested on the C3VD test set. Best results are in bold and second best results are in underlined .

Figure 4: Qualitative comparison of ablation variants on four subsequences from the EndoMapper dataset. Each row shows a sample frame and the corresponding predicted depth maps from different prior configurations.

Model	Edge Loss	Pose Estimation			Depth Estimation
Model	Edge Loss	ATE ↓	RTE ↓	ROT	RMSE ↓	Log RMSE ↓	MAE ↓	MedAE ↓	AbsRel ↓	SqRel ↓ ↓
Monodepth2 [ 11 ]	–	2.7025	0.2004	0.1250	6.3138	0.1841	4.3699	3.1279	0.1501	1.0168
Monodepth2 [ 11 ]	✓	1.6293	0.2121	0.1286	6.8043	0.1991	4.9267	3.7038	0.1681	1.2604
DLPE	–	2.5101	0.2005	0.1408	6.1246	0.1830	4.3300	3.2582	0.1475	1.0010
	✓	2.5858	0.2015	0.1412	6.5946	0.1761	4.4031	2.9848	0.1388	1.0138
	✓ †	1.9077	0.2087	0.1496	6.1246	0.1830	4.3300	3.2582	0.1475	1.0010

Table 3: Depth and pose estimation results of models trained with Hyper-Kvasir and tested on C3VD test set with different combinations of structural priors and edge-guided loss. Best configuration per model group is in bold .

Figure 5: Qualitative trajectory comparison under modality-only vs. modality + edge-guided SSIM training on cecum_t4b , sigmoid_t3b and trans_t4b .

Model	Training Dataset	Interval	RMSE $\downarrow$	RMSE log $\downarrow$	MAE $\downarrow$	MedAE $\downarrow$	Abs Rel $\downarrow$	Sq Rel $\downarrow$	$\delta<1.25$ $\uparrow$	$\delta<1.25^{2}$ $\uparrow$	$\delta<1.25^{3}$ $\uparrow$
Monodepth2 [ 11 ]	C3VD	1	15.1642	0.4196	10.1482	6.3000	0.3047	4.7856	0.4516	0.7353	0.8783
		10	7.5960	0.2215	5.0917	3.4709	0.1732	1.4608	0.7161	0.9253	0.9827
		20	7.7644	0.2473	5.4523	3.9334	0.2002	1.7340	0.6668	0.8976	0.9725
		30	7.7282	0.2403	5.2392	3.6111	0.1878	1.6480	0.6919	0.8956	0.9696
	Hyper-Kvasir	1	6.3138	0.1841	4.3699	3.1279	0.1501	1.0168	0.7754	0.9720	0.9956
		10	40.5039	1.3728	32.2524	26.0333	1.5865	93.1729	0.1331	0.2595	0.3706

Table 4: Quantitative depth estimation results across models, datasets, and frame intervals. All models are trained on either C3VD or Hyper-Kvasir and always tested on C3VD test data.

Figure 6: Qualitative generalization performance of models trained with Hyper-Kvasir on EndoMapper real dataset. This evaluation highlights model robustness to real-world challenges like specular reflections and complex tissue folds, where Hyper-Kvasir-trained models consistently show an advantage.

Model	RMSE ↓	LogRMSE ↓	MAE ↓	MedAE ↓	AbsRel ↓	SqRel ↓	$\delta<1.25$ ↑	$\delta<1.25^{2}$ ↑	$\delta<1.25^{3}$ ↑
Self-supervised	7.5960	0.2215	5.0917	3.4709	0.1732	1.4608	0.7161	0.9253	0.9827
Supervised (+GT)	8.2805	0.2443	5.6020	3.7030	0.1921	1.7754	0.6797	0.9022	0.9721

Table 5: Comparison between self-supervised and supervised Monodepth2 [ 11 ] models on C3VD (interval = 10). Supervised model adds scale-consistent depth loss with a weight of 0.5. The better one is marked in Bold

Model	Training Set	Interval	RMSE ↓	LogRMSE ↓	MAE ↓	MedAE ↓	AbsRel ↓	SqRel ↓
PRISM	C3VD	1	14.4647	0.4165	9.7689	6.3846	0.3221	4.7946
	C3VD	30	7.7530	0.2460	5.4256	3.9831	0.1972	1.6139
	C3VD_supervised	30	8.2836	0.2523	5.6774	3.8749	0.1976	1.7205
	Hyper-phasir	1	6.1246	0.1830	4.3300	3.2582	0.1475	1.0010

Table 6: Comparison of depth estimation across datasets, temporal intervals, and supervision modes using PRISM with DLPE settings.

Self-Supervised Monocular Depth Estimation

See all 17 papers

EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images
Aug 25, 2025Xinning Yao, Bo Liu, Bojian Li +3Medical Imaging Monocular Depth Estimation
Image Processing Center, Beihang University, Beijing, 100191, China. · State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology, Beijing 102206, China.
Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation
Feb 25, 2026Juan Yang, Yuyan Zhang, Han Jia +2Monocular Depth Estimation Synthetic-to-Real Domain Adaptation
College of Computer Science, Sichuan University, Chengdu, China · Department of Gastroenterology and Hepatology, West China Hospital, Sichuan University, Chengdu, China
CoGE: Sim-to-Real Online Geometric Estimation for Monocular Colonoscopy
May 13, 2026Liangjing Shao, Beilei Cui, Hongliang RenSynthetic-to-Real Domain Adaptation Endoscopy
Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · Shenzhen Loop Area Institute, China

Aug 25, 2025J/K move · Enter open · S save

EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images

Xinning Yao, Bo Liu, Bojian Li, Jingjing Wang+2

Image Processing Center, Beihang University, Beijing, 100191, China. · State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology, Beijing 102206, China.

Medical Imaging Monocular Depth Estimation Vision Foundation Model Adaptation

Open paper AskPDF arXiv

Depth estimation is a foundational component for 3D reconstruction in minimally invasive endoscopic surgeries. However, existing monocular depth estimation techniques often exhibit limited performance to the varying illumination and complex textures of the surgical environment. While applying foundation models offers a promising approach to enhance the depth estimation performance, the domain gap between the natural images used for pre-training and the target endoscopic images leads to significant semantic perception deficiencies. In this study, EndoUFM is introduced as an unsupervised monocular depth estimation framework that innovatively Utilizes dual Foundation Models for Endoscopic images, thereby enhancing the depth estimation performance by leveraging the powerful pre-learned priors. The framework features a novel adaptive fine-tuning strategy that incorporates Random Vector Low-Rank Adaptation (RVLoRA) to enhance model adaptability, and a Residual block based on Depthwise Separable Convolution (Res-DSC) to improve the capture of fine-grained local features. A mask-guided smoothness loss is also introduced to enforce depth consistency within anatomical structures. Extensive experiments on the SCARED, Hamlyn, SERV-CT, and EndoNeRF datasets confirm that our method achieves state-of-the-art performance while maintaining an efficient model size. This work contributes to augmenting surgeons' spatial perception during minimally invasive procedures, thereby enhancing surgical precision and safety, with crucial implications for augmented reality and navigation systems. Our code is available at https://github.com/RealMindyY/EndoUFM.

Abstract

Self-Supervised Monocular Depth Estimation

Explore similar work

EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images

Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation

CoGE: Sim-to-Real Online Geometric Estimation for Monocular Colonoscopy