SAP-Depth: Scene-Adaptive Probability Framework for Self-Supervised Monocular Depth Estimation

Bowen He; Li Zhang; Yujie Chen; Chang Cheng

cs.CVOct 17, 2026

SAP-Depth: Scene-Adaptive Probability Framework for Self-Supervised Monocular Depth Estimation

Authors: Bowen He, Li Zhang, Yujie Chen, Chang Cheng

Organizations: Hefei University of Technology, Hefei, China.

Abstract

Self-supervised monocular depth estimation remains challenging due to the loss of fine-grained details and limited adaptability to varying scene structures. In this paper, we propose SAP-Depth, a novel architecture based on Scene-Adaptive Probabilistic. Unlike traditional methods, we introduce a Dynamic Bins mechanism powered by a Learnable Parameter Predictor, which adaptively estimates scene-specific depth distributions conditioned on the input image context. Furthermore, a Bin-wise Probability Distribution Decoder is designed to effectively fuse multi-scale representations to predict dense depth probability distributions, significantly enhancing boundary sharpness and structural recovery. Through the collaborative coupling of these proposed modules, our model achieves dynamic prediction of scene-specific depth distributions. Extensive experiments on the KITTI dataset show that SAP-Depth outperforms existing state-of-the-art methods in accuracy, particularly in its ability to better preserve complex edge details. Further, tests on the Make3D dataset validate the model's excellent generalization capability and robustness.

Figures & tables

Figure 1: The proposed SAP-Depth generates more accurate depth maps with sharper boundaries compared to Lite-Mono [ 11 ] , R-MSFM6 [ 23 ] . Ours shows better contour details.

Figure 2: Overview of our proposed SAP-Depth framework. The system consists of two main components: a depth network and a pose network. (a) The depth network utilizes a shared encoder to extract multi-scale features, which are then decoupled into two branches: the LP Predictor predicts global adaptive depth bins, while the HAP-FAM estimates pixel-wise bin probabilities. The final high-quality depth map is generated by the linear combination of the predicted bins and probabilities. (b) During training, the pose network takes adjacent frames ( $I_{t},I_{t^{\prime}}$ ) to estimate the 6-DoF relative pose $T_{t\to t^{\prime}}$ , enabling the photometric reconstruction loss for self-supervision.

Method	Mode	RMSE $\downarrow$	RMSE log $\downarrow$	Sq Rel $\downarrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	$\delta<1.25^{2}\uparrow$	$\delta<1.25^{3}\uparrow$
Monodepth2 [ 10 ]	MS	4.750	0.196	0.818	0.106	0.874	0.957	0.979
DepthHints [ 22 ]	MS	4.627	0.189	0.769	0.105	0.875	0.959	0.982
HR-Depth [ 6 ]	MS	4.612	0.185	0.785	0.107	0.887	0.962	0.982
R-MSFM6 [ 23 ]	MS	4.625	0.189	0.787	0.111	0.882	0.961	0.981
Monodepth2 [ 10 ]	M	4.863	0.193	0.903	0.115	0.877	0.959	0.971
HR-Depth [ 6 ]	M	4.632	0.185	0.792	0.109	0.884	0.962	0.983

Table 1: Quantitative comparison on the KITTI Eigen split. Comparison with other self-supervised monocular methods. Methods are grouped by training mode: MS indicates monocular and stereo training, and M indicates monocular training. The best results are highlighted in bold .

Figure 3: Qualitative comparison on the KITTI dataset . We compare our method against Lite-Mono [ 11 ] , R-MSFM6 [ 23 ] , and DIFFNet [ 25 ] . As shown, our model generates more complete depth predictions and preserves sharper object boundaries.

Method	Size	Mode	RMSE $\downarrow$	RMSE ${}_{log}\downarrow$	Sq Rel $\downarrow$	Abs Rel $\downarrow$
Monodepth2 [ 10 ]	$640\times 192$	M	7.418	0.163	3.589	0.322
HR-Depth [ 6 ]	$640\times 192$	M	7.024	0.159	3.208	0.315
CADepth-Net [ 24 ]	$640\times 192$	M	7.066	0.159	3.086	0.312
DIFFNet [ 25 ]	$640\times 192$	M	7.008	0.155	3.313	0.309
Lite-Mono [ 11 ]	$640\times 192$	M	6.981	0.158	3.060	0.305
Zhao et al. [ 29 ]	$640\times 192$	M	7.095	0.158	3.200	0.316

Table 2: Zero-shot generalization results on the Make3D dataset. Models are trained on KITTI and tested on Make3D without fine-tuning. The best results are highlighted in bold .

Table 6

Number of Bins ( $N$ )	RMSE $\downarrow$	RMSE ${}_{log}\downarrow$	Sq Rel $\downarrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$
16 Bins	4.593	0.182	0.816	0.105	0.891
32 Bins	4.524	0.180	0.773	0.103	0.895
64 Bins	4.452	0.179	0.755	0.101	0.897
128 Bins	4.534	0.181	0.769	0.102	0.896
256 Bins (Ours)	4.453	0.178	0.733	0.101	0.898

Table 5: Performance evaluation across different numbers of adaptive bins (

N

). Setting

N=256

achieves the optimal accuracy under the available computational constraints.

Method	Backbone	Params (M) $\downarrow$	FLOPs (G) $\downarrow$	Abs Rel $\downarrow$
Monodepth2 [ 10 ]	ResNet-50	32.50	16.70	0.110
MonoFormer [ 26 ]	ResNet-50	159	112	0.104
Lite-Mono [ 11 ]	Lite-CNN	3.07	5.04	0.107
R-MSFM6 [ 23 ]	ResNet-18	5.24	34.5	0.109
SAP-Depth (Ours)	HRNet-18	10.16	15.92	0.101

Table 6: Comparison of Model Size (Params) and Computational Complexity (FLOPs) evaluated on the

192\times 640

input resolution.

Depth Estimation

Depth estimation infers scene distance from images and sensors, enabling 3D reconstruction, navigation, and spatial understanding.

See all 196 papers

SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation
May 27, 2026Changxuan Li, Nadine Berner, Nassir Navab +2Self-Supervised Learning Camera Pose Estimation
Technical University of Munich. · Munich Center for Machine Learning (MCML). · BMW Group. +2
CMAMBADEPTH: Self-supervised Monocular Depth Estimation with Channel Mamba and Hybrid Attention
Sep 21, 2026Xuezhi Xiang, Jiayao Liu, Heqi Xiang +3Monocular Depth Estimation Multi-Scale Feature Fusion
School of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin 150001, China · Department of Computer Science, University of Toronto, Toronto ON M5S 2E4, Canada +1
Learning to Discriminate Scene Structures Makes Self-Supervised Depth Learning Scalable
Oct 16, 2026Mengtan Zhang, Yuchuan Cui, Yu Ma +3Self-Supervised Monocular Depth Estimation Domain Generalization
Tongji University

May 27, 2026J/K move · Enter open · S save

SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation

Changxuan Li, Nadine Berner, Nassir Navab, Federico Tombari+1

Technical University of Munich. · Munich Center for Machine Learning (MCML). · BMW Group.+2

Self-Supervised Learning Camera Pose Estimation Monocular Depth Estimation Self-Supervised Monocular Depth Estimation

Open paper AskPDF arXiv

Self-supervised depth estimation from monocular sequences relies on the joint learning of a depth and a pose network. Despite abundant research done to improve the depth network, efforts on the pose remain limited. In this context, even when depth is estimated up to scale, we highlight the importance of the alignment between the scene scales estimated by the pose and depth nets. Then, we introduce SA4Depth, an approach to improve this alignment and boost the depth predictions while keeping the inference time unchanged. Our proposed method uses the depth estimated during training to reproject learnable visual features across consecutive frames and refine the pose estimates by reducing feature alignment residuals. With our method, the estimated scene scales by the separate depth and pose networks are aligned, and the prediction scale consistency is improved across different sequences. Our differentiable refinement integrates seamlessly into existing self-supervised pipelines and substantially improves their depth estimates. We demonstrate this with extensive experiments both outdoors and indoors on KITTI, Cityscapes, and NYUv2. Additionally, results on KITTI Odometry confirm the effectiveness of our pose refinement. Our code is available at https://github.com/Runningchauncey/SA4Depth .

Abstract

Depth Estimation

Explore similar work

SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation

CMAMBADEPTH: Self-supervised Monocular Depth Estimation with Channel Mamba and Hybrid Attention

Learning to Discriminate Scene Structures Makes Self-Supervised Depth Learning Scalable