Learning to Discriminate Scene Structures Makes Self-Supervised Depth Learning Scalable

Mengtan Zhang; Yuchuan Cui; Yu Ma; Zizhan Guo; Wei Ye; Rui Fan

cs.CVOct 16, 2026

Learning to Discriminate Scene Structures Makes Self-Supervised Depth Learning Scalable

Authors: Mengtan Zhang, Yuchuan Cui, Yu Ma, Zizhan Guo, Wei Ye, Rui Fan

Organizations: Tongji University

Abstract

Self-supervised monocular depth estimation is appealing for its potential to scale with unlabeled data, yet existing methods often degrade when trained on diverse scenes. This study finds that, under self-supervision, models tend to encode dataset-specific structural biases rather than transferable scene geometry, leading to cross-domain optimization conflicts. To address this issue, this study proposes the MaSS framework, aiming to make self-supervised depth learning scalable to diverse data. MaSS introduces a dedicated structural representation pathway explicitly decoupled from contextual features, which is guided by a novel scene-structure discrimination objective to learn generalized, layout-aware structural representations for depth decoding. Trained and evaluated on four diverse datasets, both individually and jointly, MaSS substantially outperforms prior self-supervised methods and is the first to consistently benefit from mixed-domain training. Zero-shot evaluation on eight additional datasets further demonstrates its remarkable generalizability, particularly on those entirely out-of-distribution scenes. The source code is publicly available at https://mias.group/MaSS.

Figures & tables

Figure 2: Feature activation map comparison among a conventional self-supervised depth model, DINOv2, and the two representations learned by MaSS. (a) The conventional depth model tends to encode dataset-specific biases, whereas (b) DINOv2 lacks sufficient geometric cues. In contrast, MaSS decouples features into contextual and structural representations: (c) contextual ones capture fine-grained object details and foreground–background relations; (d) structural ones encode global scene layouts that are transferable across diverse datasets.

Figure 3: The overall architecture of the proposed MaSS framework. MaSS bridges a conventional depth encoder with a frozen pretrained VFM via a bidirectional interaction module. The interacted VFM features are fed into a pair of anchored–decoupled feature projectors, producing collapse-free contextual and structural representations. The structural projectors learn generalized representations from discriminating scene structures, making the self-supervised depth learning scalable.

Method	Type	Align.	KITTI		DDAD		NYUv2		nuScenes
Method	Type	Align.	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$
SC-DepthV2 [ 56 ] TPAMI’22	SI	Median	0.118	0.866	0.164	0.775	0.138	0.820	–	–
Lite-Mono [ 19 ] CVPR’23	SI	Median	0.101	0.897	0.175	0.799	0.198	0.688	0.419	0.720
Dynamo-Depth [ 15 ] NeurIPS’23	SI	Median	0.112	0.873	0.150	0.798	–	–	0.179	0.787
Gas-Mono [ 2 ] ICCV’23	SI	Median	0.111	0.894	0.178	0.756	0.113	0.871	–	–
SC-DepthV3 [ 3 ] TPAMI’24	SI	Median	0.118	0.864	0.142	0.813	0.123	0.848	0.167	0.784

Table 1: Quantitative results on the training-evaluation datasets. † indicates training under the mixed-dataset setting. For single-dataset training, results are reported at the image resolutions of 192

\times

640, 384

\times

640, 256

\times

320, and 288

\times

512, respectively, while mixed-dataset training results are evaluated at a unified resolution of 384

\times

640. The best results are marked in bold type.

Figure 4: Qualitative results on zero-shot generalization datasets. Compared with representative SSMDE models specialized for either indoor or outdoor scenes and the model scaled with large-scale data and pretrained backbones, MaSS demonstrates consistent superiority across both scenarios.

Method	Type	Align.	Waymo		DrivingStereo		Cityscapes		DIML
Method	Type	Align.	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$
DA V2 [ 37 ] NeurIPS’24	SSI*	LSQ*	0.084	0.912	0.103	0.908	0.088	0.920	–	–
Marigold [ 25 ] CVPR’24	SSI	LSQ	0.128	0.868	0.128	0.858	0.118	0.886	0.123	0.863
Lotus [ 26 ] ICLR’25	SSI	LSQ	0.123	0.881	0.123	0.877	0.121	0.878	0.099	0.908
Dynamo-Depth [ 15 ] NeurIPS’23	SI	Median	0.167	0.784	0.164	0.788	0.233	0.564	0.175	0.748
DCPI-Depth [ 4 ] TIP’25	SI	Median	0.178	0.771	0.161	0.787	0.241	0.549	0.174	0.733

Table 2: Quantitative zero-shot generalization results. Models under the single dataset training are evaluated at the image resolutions of 192

\times

640 (outdoor) and 256

\times

320 (indoor). Mixed-dataset models ( † ) are evaluated for the image resolutions of 384

\times

640. Supervised foundation models are reported at their pretrained resolutions (518

\times

518 for [ 37 ] and 768

\times

768 for [ 25 , 26 ] ). Results on the DIML [ 54 ] dataset are not reported for method [ 37 ] that was pretrained on this dataset. The best results for self-supervised and supervised models are marked in bold type, respectively. SSI* and LSQ* denote the predicted depth and its alignment with the ground-truth in the inverse depth domain.

Method	Marigold [ 25 ]	DA V2 [ 37 ]	Lite-Mono [ 19 ]	DCPI-Depth [ 4 ]	SC-DepthV3 [ 3 ]	Jasmine [ 9 ]	KBR [ 6 ]	KBR $++$ [ 7 ]	MaSS † (Ours)
Params. (M)	1290.0	97.5	3.1	8.8	14.8	1290.0	92.5	266.6	93.2
FLOPs (G)	339.3	89.5	6.2	13.7	9.8	339.3	88.7	381.7	81.6
Training Data	74K	62M	40K	40K	20K	68K	1.9M	2.2M	150K

Table 3: Quantitative comparisons on the models’ overhead and their required training data.

(ID) Method	Training Sets				Zero-Shot
	nuScenes		NYUv2		Waymo		7-Scenes
	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$
MaSS †	0.134	0.852	0.104	0.890	0.123	0.867	0.120	0.855
(a1) w/o Contrastive Learning	0.145	0.850	0.106	0.891	0.163	0.805	0.135	0.820
(a2) w/o Representation Decoupling	0.144	0.840	0.116	0.862	0.148	0.831	0.128	0.823
(a3) w/o Interaction Architecture	0.147	0.837	0.119	0.859	0.190	0.750	0.139	0.805

Table 4: Quantitative results for ablation studies and model design analyses.

Method	KITTI		DDAD		NYUv2		nuScenes
Method	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$
MaSS † (K+D+NY+NU, 150K)	0.092	0.913	0.125	0.862	0.104	0.890	0.134	0.852
MaSS † * (K+D+NY+NU+T, 200K)	0.093	0.912	0.122	0.857	0.097	0.905	0.136	0.850
Method	Waymo		DrivingStereo		Cityscapes		DIML
Method	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}$ $\uparrow$
MaSS † (K+D+NY+NU, 150K)	0.123	0.867	0.100	0.905	0.103	0.896	0.104	0.890

Table 5: Quantitative results for the training scalability analyses.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Quantitative analyses of the proposed scene-structure discrimination objective for structural representation learning.

Figure S2: Quantitative visualization of the scene structure (depth) distributions across all training and evaluation datasets.

Figure S3: Quantitative visualization of the scene appearance (RGB) distributions across all training and evaluation datasets.

Method	Types	Align.	Waymo		DrivingStereo		Cityscapes		DIML
Method	Types	Align.	Abs Rel	$\delta_{1}$	Abs Rel	$\delta_{1}$	Abs Rel	$\delta_{1}$	Abs Rel	$\delta_{1}$
DA V2 [ 37 ]	SSI*	LSQ*	0.084	0.912	0.103	0.908	0.088	0.920	–	–
KBR [ 6 ]	SI	Median	0.156	0.806	0.139	0.825	0.111	0.864	0.105	0.902
KBR [ 6 ]	SI	LSQ*	0.116	0.850	0.115	0.865	0.098	0.896	0.079	0.902
MaSS † (Ours)	SI	Median	0.123	0.867	0.100	0.905	0.103	0.896	0.104	0.890
MaSS † (Ours)	SI	LSQ*	0.088	0.908	0.088	0.916	0.087	0.917	0.089	0.922

Appendix

Table S1: Quantitative zero-shot generalization results. Mixed-dataset models ( † ) are evaluated for the image resolutions of 384

\times

640. Supervised foundation models are reported at their pretrained resolutions (518

\times

518 for [ 37 , 65 ] and 768

\times

768 for [ 25 , 26 ] ). Results on the DIML [ 54 ] dataset are not reported for methods [ 37 , 65 ] that were pretrained on this dataset. The best results for self-supervised and supervised models are marked in bold type, respectively.

Method	Types	Align.	KITTI		nuScenes		DrivingStereo		Cityscapes
Method	Types	Align.	Abs Rel	$\delta_{1}$	Abs Rel	$\delta_{1}$	Abs Rel	$\delta_{1}$	Abs Rel	$\delta_{1}$
DA V2 [ 37 ]	SSI	LSQ	0.095	0.906	0.171	0.780	0.103	0.908	0.088	0.920
DA V2 $+$ Ours	SI	Median	0.085	0.923	0.125	0.891	0.090	0.924	0.085	0.930
VGGT [ 38 ]	SI	Median	0.099	0.885	0.127	0.837	0.094	0.918	0.111	0.862
VGGT $+$ Ours	SI	Median	0.085	0.924	0.123	0.871	0.089	0.932	0.092	0.920

Appendix

Table S2: Quantitative results on adapting depth/geometry foundation models.

Figure S4: Qualitative monocular depth estimation results on in-the-wild, out-of-distribution data.

Figure S5: Qualitative monocular reconstruction estimation results on in-the-wild, out-of-distribution data.

Self-Supervised Monocular Depth Estimation

See all 17 papers

Sep 21, 2026

CMAMBADEPTH: Self-supervised Monocular Depth Estimation with Channel Mamba and Hybrid Attention

Accurate monocular depth estimation serves as a core enabler for single camera scene understanding. However, existing self-supervised monocular depth estimation methods generally suffer from the bottleneck of inefficient cross-scale information interaction and difficulty in balancing local and global spatial modeling. In this paper, we propose CMambaDepth, a self-supervised framework that achieves efficient multi-scale feature fusion and fine-grained contextual modeling via channel-wise selective state propagation. Specifically, Bidirectional Channel Mamba (Bi-CMamba) aligns encoder features across scales and enables bidirectional information exchange among ordered scale groups. Unidirectional Channel Mamba (Uni-CMamba) progressively aggregates decoder features and retains fine-grained scale groups through a group selection mechanism for subsequent fusion. Furthermore, a Hybrid Attention Module (HAM) is introduced to combine large-kernel local context and Manhattan self-attention for complementary spatial modeling. Experimental results demonstrate that our method achieves highly competitive performance. Specifically, our model achieves an AbsRel of 0.094 and an RMSE of 4.156 on KITTI, and an AbsRel of 0.140 on DDAD. In the zero-shot cross-dataset generalization test on NYUv2, it attains an AbsRel of 0.232, outperforming the baseline RA-Depth by 7.2%.

Xuezhi Xiang, Jiayao Liu, Heqi Xiang +3

School of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin 150001, China · Department of Computer Science, University of Toronto, Toronto ON M5S 2E4, Canada +1

Monocular Depth Estimation Multi-Scale Feature Fusion Visual SSMs Self-Supervised Monocular Depth Estimation

Ask PDF

Oct 17, 2026

SAP-Depth: Scene-Adaptive Probability Framework for Self-Supervised Monocular Depth Estimation

Self-supervised monocular depth estimation remains challenging due to the loss of fine-grained details and limited adaptability to varying scene structures. In this paper, we propose SAP-Depth, a novel architecture based on Scene-Adaptive Probabilistic. Unlike traditional methods, we introduce a Dynamic Bins mechanism powered by a Learnable Parameter Predictor, which adaptively estimates scene-specific depth distributions conditioned on the input image context. Furthermore, a Bin-wise Probability Distribution Decoder is designed to effectively fuse multi-scale representations to predict dense depth probability distributions, significantly enhancing boundary sharpness and structural recovery. Through the collaborative coupling of these proposed modules, our model achieves dynamic prediction of scene-specific depth distributions. Extensive experiments on the KITTI dataset show that SAP-Depth outperforms existing state-of-the-art methods in accuracy, particularly in its ability to better preserve complex edge details. Further, tests on the Make3D dataset validate the model's excellent generalization capability and robustness.

Bowen He, Li Zhang, Yujie Chen +1

Hefei University of Technology, Hefei, China.

Depth Estimation Self-Supervised Monocular Depth Estimation 3D Reconstruction

Ask PDF

Jul 1, 2026

Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation

Self-Supervised Monocular Depth Estimation (MDE) has garnered attention in recent years due to its independence from ground truth. However, most existing models are limited to a single scale and exhibit considerable performance degradation in complex driving environments. Networks specifically designed to handle dynamic traffic participants tend to be overly complex, hindering their deployment on resource-constrained automotive edge devices. To address these limitations and move towards robust driving perception, we propose FlexDepth, a scale-driven and flexible family of self-supervised MDE models tailored for challenging road scenarios. FlexDepth employs a two-stage static-dynamic decoupled training strategy, enabling the independent assessment of confidence for both static backgrounds and dynamic road objects. Furthermore, it introduces a meticulously designed Scale-Driven Decoder (SDD) to dynamically select components based on scale size, facilitating efficient feature fusion and the output of high-precision depth maps. Extensive experiments on standard driving benchmarks demonstrate that without any auxiliary information, our model achieves state-of-the-art performance across arbitrary scales with minimal computational overhead. Our smallest model, Flex-Nano, requires only 0.7 GFLOPs and achieves 37.6 FPS on mobile platforms, ensuring reliable real-time perception while maintaining excellent zero-shot generalization. Our source code is avalible: https://github.com/startnew/flexdepth

Zhaowen Zhu, Li Zhang, Yujie Chen +3

Hefei University of Technology, Hefei, China · Chengpin Home Tech, Nanjing, China · Differential Robotics, Hangzhou, China +1

Depth Estimation Monocular Depth Estimation Autonomous Driving Multi-Scale Feature Fusion Self-Supervised Monocular Depth Estimation

Ask PDF

Abstract

Self-Supervised Monocular Depth Estimation

Explore similar work

CMAMBADEPTH: Self-supervised Monocular Depth Estimation with Channel Mamba and Hybrid Attention

SAP-Depth: Scene-Adaptive Probability Framework for Self-Supervised Monocular Depth Estimation

Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation