ColonAdapter: Geometry Estimation Through Foundation Model Adaptation for Colonoscopy

Zhiyi Jiang; Yifu Wang; Xuelian Cheng; Zongyuan Ge

eess.IVNov 27, 2025

ColonAdapter: Geometry Estimation Through Foundation Model Adaptation for Colonoscopy

Authors: Zhiyi Jiang, Yifu Wang, Xuelian Cheng, Zongyuan Ge

Organizations: School of Computer Science and Engineering, Southeast University, China · Vetex Lab, Shanghai, China · Southeast University - Monash University Joint Graduate School, Suzhou 215123, China · Monash Suzhou Research Institute, Monash University, Suzhou 215000, China ·

Abstract

Estimating 3D geometry from monocular colonoscopy images is challenging due to non-Lambertian surfaces, moving light sources, and large textureless regions. While recent 3D geometric foundation models eliminate the need for multi-stage pipelines, their performance deteriorates in clinical scenes. These models are primarily trained on natural scene datasets and struggle with specularity and homogeneous textures typical in colonoscopy, leading to inaccurate geometry estimation. In this paper, we present ColonAdapter, a self-supervised fine-tuning framework that adapts geometric foundation models for colonoscopy geometry estimation. Our method leverages pretrained geometric priors while tailoring them to clinical data. To improve performance in low-texture regions and ensure scale consistency, we introduce a Detail Restoration Module (DRM) and a geometry consistency loss. Furthermore, a confidence-weighted photometric loss enhances training stability in clinical environments. Experiments on both synthetic and real datasets demonstrate that our approach achieves state-of-the-art performance in camera pose estimation, monocular depth prediction, and dense 3D point map reconstruction, without requiring ground-truth intrinsic parameters.

Figures & tables

Fig. 1: Comparison of depth map estimations on colonoscopy images between a 3D geometric foundation model (DUSt3R [ 4 ] ) and our proposed method. The top row shows how extensive textureless regions cause the model to misinterpret distant areas as close (and vice versa). The middle row highlights how shadows caused by moving light sources and complex anatomical structures lead DUSt3R to erroneously label them as distant. In contrast, the bottom row illustrates that our method accurately reconstructs scenes containing textureless surfaces, dynamic shadows, and non-Lambertian regions, where DUSt3R fails.

Fig. 2: The training pipeline of our proposed framework, consisting of an adapted foundation module (a), an affiliation module (b), and loss calculation (c). The adapted foundation module takes two input images and generates corresponding point maps along with their confidence maps. The affiliation module, used only during training, provides image brightness calibration, optical flow, and affiliated pose information. With the generated components from these two modules, we reconstruct image and calculate the losses. During evaluation, we rely solely on the adapted foundation module to generate point maps, which are then used to derive camera information, including poses and intrinsic parameters.

Fig. 3: Architecture of the proposed Detail Restoration Module. The ResNet-18 extracts multi-level features from the input images, which are then fused with ViT features through fusion adapters. The fused features are injected only into the first five layers of the ViT decoder to provide low-level information.

Method	G.I.	ATE $\downarrow$	RPE Trans $\downarrow$	RPE Rot $\downarrow$
AF-SfMLearner	✓	0.0067	0.1479	0.8518
Lite-Mono	✓	0.0169	0.1438	0.5962
Monodepth2	✓	0.0103	0.1459	0.5679
DARES	✓	0.0109	0.1473	0.9073
EndoDAC	$\times$	0.0146	0.1411	0.5787
Ours	$\times$	0.0062	0.0570	0.5573

TABLE I: Quantitative camera pose estimation on various sequences of SimCol3D, evaluated using 10 random frames in accordance with [ 31 ] . ”G.I.” indicates the use of ground-truth intrinsic parameters.

Method	G.I.	$\mathrm{ATE}_{s1}\downarrow$	$\mathrm{ATE}_{s2}\downarrow$	$\mathrm{ATE}_{s3}\downarrow$
AF-SfMLearner	✓	0.2704	0.2710	0.1591
Lite-Mono	✓	0.3064	0.3114	0.1620
Monodepth2	✓	0.2951	0.3143	0.1702
DARES	✓	0.2678	0.2620	0.1594
EndoDAC	$\times$	0.2711	0.2937	0.1596
Ours	$\times$	0.2654	0.2660	0.1580

TABLE II: Quantitative ego-motion comparison on the SimCol3D dataset. The ATE is averaged over all 5-frame snippets following [ 5 ] .

Method	G.I.	SimCol3D					CSD
		Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$	$\delta\uparrow$	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$	$\delta\uparrow$
AF-SfMLearner	✓	0.077	0.474	4.124	0.111	0.951	0.071	0.208	2.565	0.110	0.950
Lite-Mono	✓	0.092	0.708	3.976	0.123	0.937	0.080	0.207	2.428	0.117	0.943
Monodepth2	✓	0.083	0.554	3.861	0.112	0.950	0.100	0.668	3.208	0.129	0.915
DARES	✓	0.077	0.439	4.458	0.121	0.944	0.116	0.616	3.339	0.157	0.881
ManyDepth	✓	0.084	0.526	4.291	0.118	0.944	0.072	0.207	2.347	0.104	0.959

TABLE III: Monocular depth estimation on SimCol3D and CSD datasets. Best results are in bold , second-best are underlined .

Fig. 4: Qualitative depth estimation results on the EndoMapper dataset. The top row highlights our method’s ability, enhanced by the integration of DRM, to capture fine structural details (highlighted with red box) that are missed by other approaches. The bottom row shows that even in the presence of unseen artifacts such as bubble, our method still predicts artifact-free geometry.

Method	G.I.	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$	$\delta\uparrow$
AF-SfMLearner	✓	0.152	1.153	6.321	0.190	0.805
Lite-Mono	✓	0.146	1.406	7.210	0.191	0.812
Monodepth2	✓	0.143	1.248	7.241	0.188	0.820
DARES	✓	0.134	1.096	6.593	0.175	0.817
ManyDepth	✓	0.190	1.960	8.641	0.260	0.716
EndoDAC	$\times$	0.153	1.380	7.280	0.185	0.812

TABLE IV: Monocular depth estimation on C3VD dataset. Best results are in bold , second-best are underlined .

Fig. 5: Qualitative comparison of our predicted 3D point maps and the baseline DUSt3R on real colonoscopy images. In the top row, our method successfully recovers a 3D scene from two images containing textureless and non-Lambertian surfaces, while DUSt3R produces a distorted plane. In the bottom row, our method reconstructs the scene geometry, whereas DUSt3R predicts most of the region as a plane.

Method	G.I.	Acc. $\downarrow$	Comp. $\downarrow$	SqRel $\downarrow$	RMSE log $\downarrow$
AF-SfMLearner	✓	1.779	1.284	0.032	0.116
EndoDAC	$\times$	19.610	8.566	2.463	0.368
DUSt3R	$\times$	41.535	27.028	1.109	0.486
Ours	$\times$	1.742	1.182	0.028	0.111

TABLE V: Point map estimation on SimCol3D

Method	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE log $\downarrow$	$\sigma\uparrow$
Fusion(1)	0.075	0.528	4.666	0.114	0.944
Fusion(2)	0.072	0.459	4.318	0.107	0.952
Fusion(3)	0.062	0.340	3.732	0.094	0.969

TABLE VI: Ablation study results on fusion strategy

$\mathcal{L}_{\text{conf-photo}}$	$\mathcal{L}_{\text{geo}}$	DRM	Abs Rel $\downarrow$	RMSE $\downarrow$	$\sigma\uparrow$
$\times$	$\times$	$\times$	0.204	8.384	0.699
✓	$\times$	$\times$	0.121	5.800	0.881
✓	✓	$\times$	0.116	5.505	0.892
✓	$\times$	✓	0.104	5.300	0.904
$\times$	✓	✓	0.334	11.703	0.437
✓	✓	✓	0.062	3.732	0.969

TABLE VII: Ablation study results

Endoscopic 3D Reconstruction

See all 19 papers

Colon3R: Cross-Domain 3D Reconstruction from Monocular Colonoscopic Video
Sep 21, 2026Zhihao Xing, Yingyu Wang, Liang Zhao +1Domain Adaptation 3D Reconstruction
Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation
Mar 16, 2026Xiaoxian Zhang, Minghai Shi, Lei LiMonocular Depth Estimation Vision Foundation Model Adaptation
Department of Biomedical Engineering, National University of Singapore, Singapore · Department of Radiotherapy, The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an, China
CoGE: Sim-to-Real Online Geometric Estimation for Monocular Colonoscopy
May 13, 2026Liangjing Shao, Beilei Cui, Hongliang RenSynthetic-to-Real Domain Adaptation Endoscopy
Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · Shenzhen Loop Area Institute, China

Sep 21, 2026J/K move · Enter open · S save

Colon3R: Cross-Domain 3D Reconstruction from Monocular Colonoscopic Video

Zhihao Xing, Yingyu Wang, Liang Zhao, Shoudong Huang

Domain Adaptation 3D Reconstruction Monocular 3D Reconstruction Semi-Supervised Learning Endoscopic 3D Reconstruction

Open paper AskPDF arXiv

Monocular colonoscopic 3D reconstruction is important for surgical robotic colonoscopy, but remains challenging due to weak texture, specular reflections, limited view overlap, and non-rigid tissue motion. Conventional multi-view 3D reconstruction methods rely on stable correspondences and approximate rigidity, which are often violated in colonoscopy. Existing endoscopic methods often rely on domain-specific supervision, whereas there are not enough in-vivo labeled data available to adapt geometry foundation models to clinical colonoscopy. We present Colon3R, a cross-domain semi-supervised framework built on pretrained VGGT that transfers coupled camera, depth, and pointmap geometry from labeled phantom and simulated data to unlabeled in-vivo colonoscopy without requiring target-domain geometric annotations. Unlike source-only fine-tuning, which learns only from phantom and simulated data, Colon3R directly exploits unlabeled in-vivo video through teacher-derived cross-view supervision. Our proposed hierarchical quasi-rigid reliability selects reliable supervision at the sequence, directed-pair, and pixel levels, while source-preserving adaptation retains the learned coupled geometry during target-domain adaptation. Extensive experiments demonstrate that our method achieves superior overall performance over state-of-the-art approaches in depth, pointmap, and camera pose estimation. Qualitative comparisons on real in-vivo colonoscopy further show substantially more complete and geometrically consistent reconstructions than competing methods under clinical domain shift. The code is publicly available at https://github.com/vglx/Colon3R.

Abstract

Endoscopic 3D Reconstruction

Explore similar work

Colon3R: Cross-Domain 3D Reconstruction from Monocular Colonoscopic Video

Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation

CoGE: Sim-to-Real Online Geometric Estimation for Monocular Colonoscopy