PureCLIP-Depth: Prompt-Free and Decoder-Free Monocular Depth Estimation within CLIP Embedding Space

Ryutaro Miya; Kazuyoshi Fushinobu; Tatsuya Kawaguchi

cs.CVMar 17, 2026

PureCLIP-Depth: Prompt-Free and Decoder-Free Monocular Depth Estimation within CLIP Embedding Space

Authors: Ryutaro Miya, Kazuyoshi Fushinobu, Tatsuya Kawaguchi

Organizations: Institute of Science Tokyo

Abstract

We propose PureCLIP-Depth, a completely prompt-free, decoder-free Monocular Depth Estimation (MDE) model that operates entirely within the Contrastive Language-Image Pre-training (CLIP) embedding space. Unlike recent models that rely heavily on geometric features, we explore a novel approach to MDE driven by conceptual information, performing computations directly within the conceptual CLIP space. The core of our method lies in learning a direct mapping from the RGB domain to the depth domain strictly inside this embedding space. Our approach achieves state-of-the-art performance among CLIP embedding-based models on both indoor and outdoor datasets. The code used in this research is available at: https://github.com/ryutaroLF/PureCLIP-Depth

Figures & tables

Figure 1: Overview of the proposed architecture. Aside from the CLIP text and image encoders, all operations are performed strictly within the conceptual CLIP embedding space.

Model	AbsRel $\downarrow$	RMSE $\downarrow$	$\log_{10}$ $\downarrow$	$\delta<1.25$ $\uparrow$	$\delta<1.25^{2}$ $\uparrow$	$\delta<1.25^{3}$ $\uparrow$
DepthCLIP (Zhang et al. ) [ 16 ]	0.388	1.167	0.156	0.394	0.683	0.851
Hu et al. [ 20 ]	0.347	1.049	0.140	0.428	0.732	0.898
Auty & Mikolajczyk [ 19 ]	0.319	0.970	0.128	0.465	0.776	0.922
PureCLIP-Depth (Ours)	0.201	0.670	0.084	0.671	0.917	0.979

Table 1: Depth estimation results on NYU Depth V2 (Eigen crop).

Model	AbsRel $\downarrow$	RMSE $\downarrow$	$\log_{10}$ $\downarrow$	$\delta<1.25$ $\uparrow$	$\delta<1.25^{2}$ $\uparrow$	$\delta<1.25^{3}$ $\uparrow$
DepthCLIP (Zhang et al. ) [ 16 ]	0.473	12.958	0.680	0.281	0.531	0.696
Hu et al. [ 20 ]	0.384	12.290	0.632	0.312	0.569	0.739
Auty & Mikolajczyk [ 19 ]	0.238	5.756	0.088	0.652	0.877	0.957
PureCLIP-Depth (Ours)	0.172	1.062	0.073	0.739	0.949	0.988

Table 2: Depth estimation results on KITTI (Garg mask).

Figure 2: Analysis of depth estimation performance across different distance ranges. (a) Joint distributions of predicted versus ground-truth depth for the NYU Depth V2 and KITTI datasets. (b) Empirical uncertainty, calculated as the standard deviation ( $\sigma$ ), with respect to the ground-truth depth. (c) Histograms illustrating the distribution of ground-truth depth (total sample count) in each dataset. All plots are generated within the ranges of 1.0 to 9.7 m for NYU Depth V2 and 1.0 to 23.0 m for KITTI; the initial bin for NYU (0.33 m) and far-range bins for KITTI (25.0, 27.0, and 29.0 m) are trimmed due to the absence of sufficient ground-truth samples in those intervals.

Figure 3: Qualitative visualization on NYU Depth V2 test set. Compared to Auty & Mikolajczyk [ 19 ] , our method shows significant improvements in depth quality for (a, b) chair shapes, (c) depth variations across the bed, (d) regions with missing values in the library aisle, (e) missing values in the shop background, and (f) the central desk and the entrance to the back room.

Figure 4: Qualitative visualization on KITTI test set. Compared to Auty & Mikolajczyk [ 19 ] , our method more accurately reflects depth for (a) the shapes of street trees, (b) cyclists, (c) parked cars, and (d) pedestrians in the shade.

Dataset	Setting	AbsRel $\downarrow$	RMSE $\downarrow$	$\log_{10}$ $\downarrow$	$\delta<1.25$ $\uparrow$	$\delta<1.25^{2}$ $\uparrow$	$\delta<1.25^{3}$ $\uparrow$
NYU	w/o MLPs	0.320	1.015	0.134	0.443	0.751	0.909
NYU	with MLPs (Baseline)	0.201	0.670	0.084	0.671	0.917	0.979
KITTI	w/o MLPs	0.371	1.889	0.147	0.405	0.727	0.886
KITTI	with MLPs (Baseline)	0.172	1.062	0.073	0.739	0.949	0.988

Table 3: Ablation study on Rotation MLPs (

\phi,\psi

Dataset	Setting	AbsRel $\downarrow$	RMSE $\downarrow$	$\log_{10}$ $\downarrow$	$\delta<1.25$ $\uparrow$	$\delta<1.25^{2}$ $\uparrow$	$\delta<1.25^{3}$ $\uparrow$
NYU	w/o CLS	0.215	0.698	0.088	0.655	0.907	0.975
NYU	with CLS (Baseline)	0.201	0.670	0.084	0.671	0.917	0.979
KITTI	w/o CLS	0.182	1.111	0.076	0.717	0.940	0.986
KITTI	with CLS (Baseline)	0.172	1.062	0.073	0.739	0.949	0.988

Table 4: Ablation study on the global context token

\mathbf{z}_{\text{CLS}}

Dataset	Loss	AbsRel $\downarrow$	RMSE $\downarrow$	$\log_{10}$ $\downarrow$	$\delta<1.25$ $\uparrow$	$\delta<1.25^{2}$ $\uparrow$	$\delta<1.25^{3}$ $\uparrow$
NYU	InfoNCE	0.216	0.693	0.087	0.658	0.908	0.974
NYU	InfoNCE + alignment	0.210	0.687	0.087	0.654	0.909	0.976
NYU	InfoNCE + alignment + RMSE (Baseline)	0.201	0.670	0.084	0.671	0.917	0.979
KITTI	InfoNCE	0.215	1.145	0.099	0.585	0.867	0.980
KITTI	InfoNCE + alignment	0.216	1.138	0.097	0.588	0.882	0.980
KITTI	InfoNCE + alignment + RMSE (Baseline)	0.172	1.062	0.073	0.739	0.949	0.988

Table 5: Ablation study on loss functions:

\mathcal{L}_{\mathrm{InfoNCE}}

\mathcal{L}_{\mathrm{align}}

, and

\mathcal{L}_{\mathrm{RMSE}}

Depth Estimation

Depth estimation infers scene distance from images and sensors, enabling 3D reconstruction, navigation, and spatial understanding.

See all 196 papers

SPACE-CLIP: Spatial Perception via Adaptive CLIP Embeddings for Monocular Depth Estimation
Jan 25, 2026Taewan Cho, Taeryang Kim, Andrew Jaeyong ChoiMonocular Depth Estimation
School of Computing, Gachon University, Republic of Korea
Lightweight Prompt-Guided CLIP Adaptation for Monocular Depth Estimation
Apr 1, 2026Reyhaneh Ahani Manghotay, Jie LiangPEFT of VLMs Monocular Depth Estimation
Simon Fraser University, Burnaby, Canada · School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada · Eastern Institute of Technology, Ningbo, China +1
SPACE-CLIPv2: Decoding Local Geometry from Frozen CLIP for Monocular Depth Estimation
Oct 4, 2026Hyun Song, Taewan Cho, Kangmin Kim +1Vision-Language Models Vision Foundation Models
School of Computing, iRASC Lab., Gachon University

Jan 25, 2026J/K move · Enter open · S save

SPACE-CLIP: Spatial Perception via Adaptive CLIP Embeddings for Monocular Depth Estimation

Taewan Cho, Taeryang Kim, Andrew Jaeyong Choi

School of Computing, Gachon University, Republic of Korea

Monocular Depth Estimation

Open paper AskPDF arXiv

Robotic and autonomous systems need dense spatial cues, yet adding a dedicated depth estimator can duplicate visual processing already performed by a multimodal model. CLIP-based depth methods offer an alternative, but commonly rely on text-derived conditioning or backbone adaptation. We present SPACE-CLIP, a decoder-only framework for supervised monocular depth estimation with a frozen CLIP vision backbone and no text encoder at inference. A FiLM-conditioned semantic pathway combines global image context with multilevel patch features, while a structural pathway supplies separately processed spatial features to a hierarchical fusion decoder. Indoor and outdoor evaluations demonstrate depth reconstruction with this architecture, and controlled component comparisons support the contribution of the structural pathway. Layer-selection experiments and frequency interventions further characterize the structural pathway's contribution to depth reconstruction. A shared-backbone microbenchmark further illustrates the reduction in duplicated computation. SPACE-CLIP provides a modular approach to adding dense depth prediction to compatible visual perception stacks. Code is available at https://github.com/taewan2002/SPACE-CLIP.

Abstract

Depth Estimation

Explore similar work

SPACE-CLIP: Spatial Perception via Adaptive CLIP Embeddings for Monocular Depth Estimation

Lightweight Prompt-Guided CLIP Adaptation for Monocular Depth Estimation

SPACE-CLIPv2: Decoding Local Geometry from Frozen CLIP for Monocular Depth Estimation