M2Depth: Unifying Monocular Depth Foundation Priors with Multi-View Stereo

Byeonggwon Lee; Sanggi Lee; Siwoo Lee; Khang Truong Giang; Soohwan Song

cs.CVAug 21, 2026

M2Depth: Unifying Monocular Depth Foundation Priors with Multi-View Stereo

Authors: Byeonggwon Lee, Sanggi Lee, Siwoo Lee, Khang Truong Giang, Soohwan Song

Organizations: Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea · 42dot, Seongnam-si, Gyeonggi-do 13449, Republic of Korea

Abstract

Deep learning-based Multi-View Stereo (MVS) has advanced significantly but often generalizes poorly to unseen scenes, particularly in occluded areas or regions with limited view overlap. To mitigate this, recent approaches integrate Depth Foundation Models (DFMs) into MVS pipelines to provide monocular depth priors. However, existing methods typically rely on a static, one-way fusion scheme, which fails to fully exploit the complementary strengths of both modalities. We propose a novel framework that overcomes this limitation by tightly coupling a DFM with a cascade MVS pipeline through a bidirectional mutual refinement strategy. Our method leverages MVS depth to resolve the scale ambiguity in monocular predictions, while the monocular depth, in turn, enhances the structural completeness and fine-grained detail of the MVS estimate. Furthermore, we introduce a prior-guided cost volume refinement mechanism that effectively integrates multi-view and monocular information via attention-based fusion and discretized depth bins, thereby promoting local geometric consistency. Extensive experiments demonstrate that our method outperforms state-of-the-art MVS approaches on standard benchmarks, producing more complete and generalizable depth maps with sharp boundaries. Furthermore, although not explicitly designed for sparse-view settings, our framework generalizes remarkably well, competing favorably with even dedicated sparse-view methods while maintaining a superior accuracy-efficiency trade-off.

Figures & tables

Fig. 2 : Our system follows a cascaded framework. Similar to the baseline [ 13 ] , we construct and regularize a multi-view cost volume $\textbf{CV}_{\text{multi}}$ at each stage, which is then converted into a probability volume PV . The MVS depth map $\textbf{D}^{\text{mvs}}$ is then estimated from PV by selecting the depth hypothesis with the highest probability. Unlike the baseline, our method (a) extracts a monocular depth map $\textbf{D}^{\text{mono}}$ using the DFM [ 12 ] , (b) refines the cost volume in the first stage using $\textbf{D}^{\text{mono}}$ , and (c) performs iterative mutual refinement between the MVS depth and the monocular depth at all stages.

Fig. 3 : Architecture of the Cost-Crafter. This module enhances the multi-view cost volume by incorporating a monocular cost volume and structural bin masks, which serve as spatial priors.

Fig. 4 : Architecture of the mutual depth refinement module. The module employs two symmetric recurrent (GRU) branches that iteratively update the MVS and monocular depth maps along with the MVS confidence map.

Fig. 5 : Qualitative results on DTU. Our method produces highly complete 3D reconstructions with clean depth maps even on challenging Scans 48 and 77.

Methods	Years	Intermediate									Advanced
	Years	Mean	Fam.	Fra.	Hor.	Lig.	M60	Pan.	Pla.	Tra.	Mean	Aud.	Bal.	Cou.	Mus.	Pal.	Tem.
COLMAP [ 8 ]	CVPR’16	42.14	50.41	22.25	25.63	56.43	44.83	46.97	48.53	42.04	27.24	16.02	25.23	34.70	41.51	18.05	27.94
CasMVSNet [ 4 ]	CVPR’20	56.84	76.37	58.45	46.26	55.81	56.11	54.06	58.18	49.51	31.12	19.81	38.46	29.10	43.87	27.36	28.11
WT-MVSNet [ 17 ]	NeurIPS’22	65.34	81.87	67.33	57.76	64.77	65.68	64.61	62.35	58.38	39.91	29.20	44.48	39.55	53.49	34.57	38.15
RA-MVSNet [ 42 ]	CVPR’23	65.72	82.44	66.61	58.40	64.78	67.14	65.60	62.74	58.08	39.93	29.14	46.04	40.30	53.22	34.63	36.28
MVSFormer [ 20 ]	TMLR’23	66.37	82.06	69.34	60.49	68.61	65.67	64.08	61.23	59.53	40.87	28.22	46.75	39.30	52.88	35.16	42.95

TABLE II : Quantitative F-score results on the TNT benchmark. Our method achieves the highest mean F-scores on both test sets.

Model	Years	DTU		TNT		KITTI
Model	Years	rel $\downarrow$	$\tau\uparrow$	rel $\downarrow$	$\tau\uparrow$	rel $\downarrow$	$\tau\uparrow$
Depth Pro [ 43 ]	ICLR’24	5.6	49.6	5.6	57.5	6.1	39.6
DepthAnythingV2 [ 12 ]	NeurIPS’24	2.6	74.7	4.5	57.5	6.6	38.6
MVSFormer++ [ 13 ]	ICLR’24	0.9	95.3	3.2	88.1	4.4	65.7
MonoMVSNet [ 1 ]	ICCV’25	0.9	95.5	2.6	89.0	4.1	66.2
MVSAnywhere [ 2 ]	CVPR’25	1.3	95.0	2.1	90.5	3.2	68.8

TABLE III : Quantitative results on the RobustMVD benchmark, reported using Absolute Relative Error (rel) and Inlier Ratio (

\tau

Fig. 6 : Qualitative sparse-view reconstruction results on DTU under the SparseRecon protocol. Our method recovers more complete and cleaner geometry than competing methods, especially in the highlighted challenging regions.

Methods		Years	Chamfer Distance $\downarrow$						Time $\downarrow$	Mem $\downarrow$
Methods		Years	24	34	110	114	118	Avg. $\downarrow$	Time $\downarrow$	Mem $\downarrow$
General MVS:
	CDS-MVSNet [ 37 ]	ICLR’22	1.98	1.43	1.02	0.86	1.87	1.43	0.64	1.30
	IterMVS [ 38 ]	CVPR’22	2.65	1.36	1.01	0.66	1.58	1.45	0.60	0.27
	Effi-MVS [ 39 ]	CVPR’22	4.35	1.88	3.54	1.67	3.15	2.92	0.62	0.51
	MVSFormer [ 20 ]	TMLR’23	1.34	0.74	0.66	0.61	1.47	0.97	0.71	0.83

TABLE IV : Sparse-view results under the SparseRecon protocol (3 unfavorable views), evaluated using Chamfer Distance (CD) on the five DTU test scenes. All methods are evaluated at an input resolution of

576\times 768

, and runtime (s) and GPU memory (GiB) are reported under the same setting.

Fig. 7 : Qualitative comparison of two-view reconstruction. Compared to MonoMVSNet, which produces outliers, and MVSAnywhere, which suffers from scale ambiguity, our method generates complete and aligned surfaces.

Fig. 8 : Illustration of scale alignment of depth maps. (a) The monocular depth map $\mathbf{D}^{\text{mono}}$ is rescaled to fit within a predefined depth hypothesis range. However, due to the presence of a few outliers, the resulting scale often fails to align accurately with the ground truth. (b) The Median Absolute Deviation (MAD) method removes statistical outliers, resulting in a refined depth map $\mathbf{D}^{\text{mad}}$ . This normalization and rescaling of $\mathbf{D}^{\text{mad}}$ leads to more accurate and consistent scale alignment with the ground-truth depth.

Fig. 9 : Qualitative visualization of component contributions. The “Cost-Crafter only” configuration provides accurate geometric guidance but tends to produce noise in textureless regions. In contrast, the “MDR only” configuration generates smoother and more complete depth maps but suffers from scale inaccuracies due to the lack of precise initialization.

Fig. 10 : Visualization of the progressive refinement of the monocular depth map ( $\text{{D}}^{\text{mono}}$ ) across cascade stages. (a) Ground Truth depth map. (b)–(d) Error maps representing the absolute difference between the estimated monocular depth and the ground truth at the 1st, 3rd, and Final stages, respectively. As the refinement proceeds through the cascade, the error magnitude significantly decreases, demonstrating that our method iteratively corrects scale ambiguity and enhances geometric consistency to match the ground truth.

Fig. 11 : Visual demonstration of robustness against erroneous DFM priors. (a) GT Image. (b) Initial monocular depth map estimated by the DFM, showing structural artifacts (highlighted in red boxes). (c) Final depth map output. Even when the initial monocular prior contains significant errors, our method leverages multi-view geometric consistency to effectively identify and override these inaccuracies, resulting in a geometrically correct final depth estimation.

Method	Num. Views	Acc. $\downarrow$	Comp. $\downarrow$	Overall $\downarrow$	$e_{2}\downarrow$	$e_{4}\downarrow$	GPU[GiB] $\downarrow$	Time(s) $\downarrow$
Baseline	3	0.3308 (2.5% $\uparrow$ )	0.2788 (6.1% $\uparrow$ )	0.3048 (4.1% $\uparrow$ )	19.01 (19.4% $\uparrow$ )	12.13 (21.4% $\uparrow$ )	2.72	0.16
	5	0.3227	0.2628	0.2928	15.92	9.99	2.88	0.24
	7	0.3201 (0.8% $\downarrow$ )	0.2612 (0.6% $\downarrow$ )	0.2909 (0.7% $\downarrow$ )	15.20 (4.7% $\downarrow$ )	9.51 (5.0% $\downarrow$ )	3.00	0.31
	10	0.3033 (6.4% $\downarrow$ )	0.2583 (1.7% $\downarrow$ )	0.2852 (2.6% $\downarrow$ )	13.63 (16.8% $\downarrow$ )	8.73 (14.4% $\downarrow$ )	3.19	0.42
Ours	3	0.3080 (2.2% $\uparrow$ )	0.2544 (5.3% $\uparrow$ )	0.2812 (3.6% $\uparrow$ )	15.38 (11.3% $\uparrow$ )	9.89 (14.7% $\uparrow$ )	2.42	0.34
	5	0.3013	0.2415	0.2714	13.82	8.62	2.53	0.41

TABLE VI : Quantitative results for the baseline and our method with varying numbers of source views

N

on the DTU test set. 3D reconstruction errors are reported using Accuracy, Completeness, and Overall scores (lower is better). Depth errors are measured by the percentage of pixels deviating more than 2mm (

e_{2}

) or 4mm (

e_{4}

). Values in parentheses indicate the relative performance change compared to the default setting (

N=5

). GPU memory usage and inference time are also reported.

$K$	Acc. $\downarrow$	Comp. $\downarrow$	Overall $\downarrow$	Mem. $\downarrow$	Time $\downarrow$
[1,1,1,1]	0.3031	0.2465	0.2748	2.53	0.25
[3,3,3,3]	0.3027	0.2459	0.2743	2.53	0.31
[12,8,8,3]	0.3011	0.2413	0.2712	2.53	0.44
[12,8,5,5]	0.3009	0.2407	0.2708	2.53	0.46
[12,8,5,3]	0.3013	0.2415	0.2714	2.53	0.41

TABLE VII : Effect of the MDR iteration setting

K

on reconstruction accuracy and efficiency on the DTU dataset.

Fig. 12 : Qualitative comparison of sparse-view 3D reconstruction using only two input images. The top row for each scene displays the estimated depth maps, and the bottom row shows the fused point clouds. MonoMVSNet [ 1 ] fails to find reliable correspondences due to limited overlap, resulting in severe noise and missing structures. MVSAnywhere [ 2 ] generates smooth depth maps but suffers from scale ambiguity, leading to misaligned geometries. In contrast, Ours produces highly complete and geometrically accurate reconstructions.

Panel of Figure 17 — Fig. 13 : Qualitative comparison of depth maps across three datasets: (a) DTU, (b) Tanks and Temples, and (c) KITTI from RobustMVD.

Multi-View Depth Estimation

Multi-view depth estimation recovers consistent 3D structure from multiple images using camera geometry, learned priors, and cross-view evidence.

See all 23 papers

Reference-Free Omnidirectional Stereo Matching via Multi-View Consistency Maximization
Mar 16, 2026Lehuai Xu, Weiming Zhang, Yang Li +2Panoramic Depth Estimation Stereo Matching
Nanjing University, Nanjing, China · Nanyang Technological University, Singapore · HKUST(GZ), Guangzhou, China
Marginalized Bundle Adjustment: Multi-View Camera Pose from Monocular Depth Estimates
Feb 21, 2026Shengjie Zhu, Ahmed Abdelkader, Mark J. Matthews +2Camera Pose Estimation Structure from Motion
Google · Michigan State University
LiteMVS: Efficient Multi-View Stereo with Foundation Distillation and Expert Aggregation
Aug 4, 2026Tianbao Zhang, Zeyu Liu, Shuyu Wu +4Multi-View Depth Estimation Depth Estimation
Shanghai Key Laboratory of Intelligent Sensing and Recognition, Shanghai Jiao Tong University · Dim12 AI Inc · Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University

Mar 16, 2026J/K move · Enter open · S save

Reference-Free Omnidirectional Stereo Matching via Multi-View Consistency Maximization

Lehuai Xu, Weiming Zhang, Yang Li, Sidan Du+1

Nanjing University, Nanjing, China · Nanyang Technological University, Singapore · HKUST(GZ), Guangzhou, China

Panoramic Depth Estimation Stereo Matching Multi-View Depth Estimation Depth Estimation Multi-View Consistency

Open paper AskPDF arXiv

Reliable omnidirectional depth estimation from multi-fisheye stereo matching is pivotal to many applications, such as embodied robotics. Existing approaches either rely on spherical sweeping with heuristic fusion strategies to build the cost columns or perform reference-centric stereo matching based on rectified views. However, these methods fail to explicitly exploit geometric relationships between multiple views, rendering them less capable of capturing the global dependencies, visibility, or scale changes. In this paper, we shift to a new perspective and propose a novel reference-free framework, dubbed FreeOmniMVS, via multi-view consistency maximization. The highlight of FreeOmniMVS is that it can aggregate pair-wise correlations into a robust, visibility-aware, and global consensus. As such, it is tolerant to occlusions, partial overlaps, and varying baselines. Specifically, to achieve global coherence, we introduce a novel View-pair Correlation Transformer (VCT) that explicitly models pairwise correlation volumes across all camera view pairs, allowing us to drop unreliable pairs caused by occlusion or out-of-focus observations. To realize scalable and visibility-aware consensus, we propose a lightweight attention mechanism that adaptively fuses the correlation vectors, eliminating the need for a designated reference view and allowing all cameras to contribute equally to the stereo matching process. Extensive experiments on diverse benchmark datasets demonstrate the superiority of our method for globally consistent, visibility-aware, and scale-aware omnidirectional depth estimation.

Abstract

Multi-View Depth Estimation

Explore similar work

Reference-Free Omnidirectional Stereo Matching via Multi-View Consistency Maximization

Marginalized Bundle Adjustment: Multi-View Camera Pose from Monocular Depth Estimates

LiteMVS: Efficient Multi-View Stereo with Foundation Distillation and Expert Aggregation