Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation

Xiaoxian Zhang; Minghai Shi; Lei Li

cs.CVMar 16, 2026

Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation

Authors: Xiaoxian Zhang, Minghai Shi, Lei Li

Organizations: Department of Biomedical Engineering, National University of Singapore, Singapore · Department of Radiotherapy, The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an, China

Abstract

Accurate monocular depth estimation is critical in colonoscopy for lesion localization and navigation. Foundation models trained on natural images fail to generalize directly to colonoscopy. We identify the core issue not as a semantic gap, but as a statistical shift in the frequency domain: colonoscopy images lack the strong high-frequency edge and texture gradients that these models rely on for geometric reasoning. To address this, we propose SpecDepth, a parameter-efficient adaptation framework that preserves the robust geometric representations of the pre-trained models while adapting to the colonoscopy domain. Its key innovation is an adaptive spectral rectification module, which uses a learnable wavelet decomposition to explicitly model and amplify the attenuated high-frequency components in feature maps. Different from conventional fine-tuning that risks distorting high-level semantic features, this targeted, low-level adjustment realigns the input signal with the original inductive bias of the foundational model. On the public C3VD and SimCol3D datasets, SpecDepth achieved state-of-the-art performance with an absolute relative error of 0.022 and 0.027, respectively. Our work demonstrates that directly addressing spectral mismatches is a highly effective strategy for adapting vision foundation models to specialized medical imaging tasks. The code will be released publicly after the manuscript is accepted for publication.

Figures & tables

Figure 1: Statistical shift in the frequency domain between natural and colonoscopy images. (a) Visual comparison of Fourier magnitude spectra. Natural images (left) display prominent star-like anisotropic radiation patterns, reflecting the abundance of straight lines and sharp edges in real-world scenes. Colonoscopy images (right) show centrally concentrated, more isotropic diffusion patterns, indicating a lack of directional geometric structures, with high-frequency energy decaying rapidly but influenced by specular highlights and unstructured noise. (b) Scatter plot of power-law spectral slope ( $\alpha$ ) versus linearity of the fit ( $R^{2}$ ). Natural street scenes (orange clusters) exhibit spectral slopes and linearity that strictly adhere to the characteristic power-law distribution of natural scenes. Conversely, colonoscopy images (blue clusters) significantly deviate from this norm, demonstrating a distinct statistical shift in the frequency domain. This shift is characterized by impulsive high-frequency components derived from mucosal artifacts, which elevate the energy in the spectral tail and result in a slower decay rate. The consistently lower $R^{2}$ values further indicate that biological tissue lacks the strict scale invariance typical of macroscopic natural scenes.

Figure 2: Overview of the proposed SpecDepth framework for the depth estimation of colonoscopy images. It employs a partially frozen DINOv2 encoder for stable feature extraction, followed by the gated wavelet transform to decouple feature representations in the frequency domain and selectively amplify critical edge signals. LN: layer normalization. MHSA: multi-head self-attention. MLP: multi-layer perceptron. BN: batch normalization.

Figure 3: Illustration of wavelet decomposition process used in wavelet transform convolution. Note that here 2× upsampling applied to frequency-domain components for visual alignment with spatial domain.

Metric	Formula
AbsRel, SqRel	$\displaystyle\frac{1}{N}\sum_{n=1}^{N}\frac{\|d_{n}-\hat{d}_{n}\|}{d_{n}}$ , $\displaystyle\frac{1}{N}\sum_{n=1}^{N}\frac{(d_{n}-\hat{d}_{n})^{2}}{d_{n}}$
RMSE, RMSElog	$\displaystyle\sqrt{\frac{1}{N}\sum_{n=1}^{N}(d_{n}-\hat{d}_{n})^{2}}$ , $\displaystyle\sqrt{\frac{1}{N}\sum_{n=1}^{N}(\log d_{n}-\log\hat{d}_{n})^{2}}$
log10	$\displaystyle\frac{1}{N}\sum_{n=1}^{N}\left\|\log_{10}d_{n}-\log_{10}\hat{d}_{n}\right\|$
SILog	$\displaystyle\frac{1}{N}\sum_{n=1}^{N}\Delta_{n}^{2}-\frac{1}{N^{2}}\left(\sum_{n=1}^{N}\Delta_{n}\right)^{2}$ , where $\displaystyle\Delta_{n}=\log d_{n}-\log\hat{d}_{n}$
$\delta_{1/2/3}$	% pixels s.t. $\displaystyle\max\left(\frac{d_{n}}{\hat{d}_{n}},\frac{\hat{d}_{n}}{d_{n}}\right)<1.25/1.25^{2}/1.25^{3}$

Table 1: Depth estimation metrics used in this study.

Method	Params (M)	AbsRel↓	SqRel↓	log10↓	RMSE↓	RMSElog↓	SILog↓	$\delta_{1}\uparrow$	$\delta_{2}\uparrow$	$\delta_{3}\uparrow$
NeWCRFs ( Yuan et al., 2022 )	270.4	0.133	0.557	0.048	3.058	0.151	11.817	0.854	0.973	0.997
NDDepth ( Shao et al., 2024 )	348.4	0.316	2.651	0.111	7.043	0.322	24.255	0.498	0.873	0.956
BTS ( Lee et al., 2019 )	50.3	0.127	0.622	0.055	3.823	0.150	11.400	0.812	0.979	0.998
Col3D-MTL+SSL ( Solano et al., 2025 )	50.3	0.107	0.346	0.042	2.729	0.128	10.072	0.899	0.989	0.999
EcoDepth ( Patni et al., 2024 )	993.48	0.103	0.367	0.042	2.721	0.116	8.539	0.901	0.994	0.998
Marigold ( Ke et al., 2024 )	865.92	0.075	0.228	0.032	2.353	0.115	11.424	0.933	0.988	0.995

Table 2: Summary of quantitative results of different method for depth estimation on the C3VD dataset.

Method	AbsRel↓	SqRel↓	RMSE↓	RMSElog↓	$\delta_{1}\uparrow$
Monodepth2 ( Godard et al., 2019 )	0.076	0.061	0.402	0.106	0.950
FeatDepth ( Shu et al., 2020 )	0.077	0.069	0.374	0.098	0.957
HR-Depth ( Lyu et al., 2021 )	0.072	0.044	0.378	0.100	0.961
DIFFNet ( Zhou et al., 2021 )	0.074	0.053	0.401	0.105	0.957
Endo-SfMLearner ( Ozyoruk et al., 2021 )	0.072	0.042	0.407	0.103	0.950
AF-SfMLearner ( Shao et al., 2022 )	0.071	0.045	0.372	0.099	0.961

Table 3: Summary of quantitative results of different method for depth estimation on the SimCol3D dataset.

Figure 5: Visualization of monocular depth estimation on the SimCol3D dataset of different methods. EcoDepth was heavily corrupted by structured noise, while Marigold, though smoother, tended to produce blob-shaped depth maps that failed to follow the elongated recession of the lumen. Depth Anything V2 captured the overall scene geometry more faithfully but often folded boundaries and transitional regions. SpecDepth produced the most coherent results, accurately rendering the tubular depth gradient along the lumen axis and maintaining crisp fold structures throughout.

Encoder	GWT	MC	AbsRel ↓	SqRel ↓	RMSE ↓	$\delta_{1}\uparrow$
Full Fine-Tuning	$\times$	$\times$	0.125	0.571	5.205	0.866
Full Fine-Tuning	$\checkmark$	$\times$	0.104	0.578	4.461	0.939
Full Fine-Tuning	$\times$	$\checkmark$	0.092	0.562	5.517	0.881
Full Fine-Tuning	$\checkmark$	$\checkmark$	0.077	0.553	3.703	0.960
LoRA	$\times$	$\times$	0.133	0.688	5.862	0.859
LoRA	$\checkmark$	$\times$	0.086	0.567	3.866	0.953

Table 4: Ablation study of the proposed model on the C3VD validation set. GWT: gated-wavelet transform. MC: multi-constraint.

$\lambda_{\mathrm{grad}}$	$\lambda_{\mathrm{smooth}}$	AbsRel $(\downarrow)$	SqRel $(\downarrow)$	RMSE $(\downarrow)$	$\delta_{1}\uparrow$
0	0	0.125	0.571	5.205	0.866
0	0.1	0.115	0.571	5.310	0.876
0	0.2	0.112	0.570	5.308	0.881
0.1	0	0.099	0.566	5.694	0.879
0.1	0.1	0.092	0.562	5.517	0.881
0.1	0.2	0.122	0.572	5.213	0.863

Table 5: Sensitivity analysis of the gradient and smoothness loss weights on the C3VD validation set.

Figure 6: Qualitative results of 3D point cloud reconstruction on the C3VD dataset. The top row displays the original colonoscopy image. The bottom row presents reconstructed point cloud generated by Open3D library.

Figure 7: Visualization of monocular depth estimation on colorectal polyp scenes of different methods, evaluated in a zero-shot setting without any fine-tuning. Images are frames extracted from the Polyp-Size dataset, a high-resolution colonoscopy video collection featuring colorectal polyps of diverse morphologies. While EcoDepth struggled with visible grid-like artifacts and Marigold tended to dissolve polyp boundaries into the surrounding mucosa, Depth Anything V2 recovered a more coherent global layout yet still lost the three-dimensional relief of lesions. SpecDepth preserved the convex profile of each polyp with well-defined boundary transitions and smoothly varying surface gradients, demonstrating more robust generalization across diverse polyp morphologies.

Monocular Depth Estimation

Monocular depth estimation infers scene geometry and distance from a single camera image, enabling 3D perception without stereo or depth sensors.

See all 132 papers

Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation
Feb 25, 2026Juan Yang, Yuyan Zhang, Han Jia +2Monocular Depth Estimation Synthetic-to-Real Domain Adaptation
College of Computer Science, Sichuan University, Chengdu, China · Department of Gastroenterology and Hepatology, West China Hospital, Sichuan University, Chengdu, China
DepthPolyp: Pseudo-Depth Guided Lightweight Segmentation for Real-Time Colonoscopy
May 15, 2026Zhuoyu Wu, Wenhui Ou, Lexi Zhang +5Endoscopy Image Segmentation
CyPhi AI Lab, Monash University, Malaysia Campus, Malaysia · Department of Electronic & Computer Engineering, Hong Kong University of Science & Technology, Hong Kong, P.R. China · Harbin Institute of Technology, Harbin, P.R. China +1
EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images
Aug 25, 2025Xinning Yao, Bo Liu, Bojian Li +3Medical Imaging Monocular Depth Estimation
Image Processing Center, Beihang University, Beijing, 100191, China. · State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology, Beijing 102206, China.

Feb 25, 2026J/K move · Enter open · S save

Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation

Juan Yang, Yuyan Zhang, Han Jia, Bing Hu+1

College of Computer Science, Sichuan University, Chengdu, China · Department of Gastroenterology and Hepatology, West China Hospital, Sichuan University, Chengdu, China

Monocular Depth Estimation Synthetic-to-Real Domain Adaptation Unpaired Image-to-Image Translation

Open paper AskPDF arXiv

Monocular depth estimation (MDE) for colonoscopy is hampered by the domain gap between simulated and real-world images. Existing image-to-image translation methods, which use depth as a posterior constraint, often produce structural distortions and specular highlights by failing to balance realism with structure consistency. To address this, we propose a Structure-to-Image paradigm that transforms the depth map from a passive constraint into an active generative foundation. We are the first to introduce phase congruency to colonoscopic domain adaptation and design a cross-level structure constraint to co-optimize geometric structures and fine-grained details like vascular textures. In zero-shot evaluations conducted on a publicly available phantom dataset, the MDE model that was fine-tuned on our generated data achieved a maximum reduction of 44.18% in RMSE compared to competing methods. Our code is available at https://github.com/YyangJJuan/PC-S2I.git.

Abstract

Monocular Depth Estimation

Explore similar work

Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation

DepthPolyp: Pseudo-Depth Guided Lightweight Segmentation for Real-Time Colonoscopy

EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images