UniMetric: Universal Metric Scale Recovery for Visual Geometry Foundation Models

Lei Yang; Yuner Zhang; Ziyang Liu; Ziying Song; Kai WU; Yiyang Gao; Jiangtao Su; Yiming Li

cs.CVOct 12, 2026

UniMetric: Universal Metric Scale Recovery for Visual Geometry Foundation Models

Authors: Lei Yang, Yuner Zhang, Ziyang Liu, Ziying Song, Kai WU, Yiyang Gao, Jiangtao Su, Yiming Li

Organizations: Nanyang Technological University · Carnegie Mellon University · The Hong Kong Polytechnic University · Tongji University ·

Abstract

Recent advances in multimodal foundation models, exemplified by GPT-6 Astra, raise the prospect of general-purpose embodied reasoning and planning. Using Internet videos for embodied pretraining calls for reliable physical-scale annotations. Yet visual geometry foundation models often recover scene structure and camera motion without reliable metric scale. We show that uncertain size priors of familiar objects can jointly constrain a reconstruction's shared scale. Building on this insight, we present UniMetric, a universal and training-free metric scale recovery framework for geometry foundation models, without additional sensors or task-specific metric supervision. Specifically, Metric Prior Acquisition elicits object height intervals from a vision-language model. Semantic-Geometric Alignment measures reconstructed object heights along the estimated support-surface normal in each frame and converts the priors into scale intervals. Global Scale Recovery reconciles these intervals through a convex objective, yielding a jointly feasible scale when they overlap and a unique compromise otherwise. The estimated sequence-level scale is applied uniformly to reconstructed points and camera translations. Controlled experiments with real VLM priors show that combining observations improves scale accuracy with exact relative object heights. To evaluate metric accuracy on GFM reconstructions without ground-truth scale alignment, we introduce MetricBench, covering twelve embodied, driving, and roadside subsets with sensor-derived ground truth. Across nine geometry foundation models, UniMetric reduces mean depth Abs Rel, reconstruction Chamfer distance, and trajectory error by 54.2%, 83.8%, and 65.6%, respectively, improving every model on all three tasks at the suite level. We release the UniMetric code and MetricBench data.

Figures & tables

Figure 1: Uncertain object-size priors jointly constrain metric scale. (a) A vision–language model provides plausible height intervals for familiar objects across frames. (b) Combined with reconstructed object heights, these intervals constrain a shared scale. A candidate consistent with one object may violate another’s prior; their joint constraints narrow the admissible range. Here, $\hat{s}$ and $s^{\mathrm{gt}}$ denote the estimated and reference scales. (c) UniMetric improves metric depth accuracy across nine geometry foundation models on MetricBench’s driving-and-infrastructure and embodied-AI suites. Curves compare original (None) and scale-corrected (UniMetric) predictions using $\delta_{1}$ (%, higher is better), averaged equally over six subsets per suite. Evaluation uses no ground-truth scale alignment.

Figure 2: Joint scale estimation with real VLM height priors and exact relative heights. (a) Available frame–object observations per sequence; the dashed line marks the budget of 50 observations. (b) Mean absolute relative scale error as the observation budget increases. (c) Scale-induced distance error computed from the results in (b) at $K=20$ ; the inset shows the uncorrected reference.

Dataset	$K=1$	$K=10$	$K=20$	$K=50$ [95% CI]	Reduction (%)
nuScenes	7.245	2.707	2.152	1.691 [1.178, 2.279]	76.7
JRDB	6.627	2.761	2.275	2.175 [1.762, 2.576]	67.2
3EED	5.435	1.954	1.501	1.097 [0.768, 1.441]	79.8
V2X-Radar-I	4.526	1.717	1.337	1.086 [0.886, 1.310]	76.0

Table 1: Scale recovery with different observation budgets.

K

is the maximum number of frame–object observations per sequence. Mean absolute relative scale error (MARE) is reported in percent (lower is better). Brackets give sequence-bootstrap

95\%

confidence intervals. Reduction is relative to

K=1

Figure 3: Overview of UniMetric. Metric Prior Acquisition obtains object height intervals and masks from RGB images. Semantic–Geometric Alignment measures reconstructed heights along frame-specific support directions and converts the priors into scale intervals. Global Scale Recovery combines these intervals through a weighted convex objective, yielding a shared scale for reconstructed geometry and camera translations.

Dataset	Hardware platform(s)	Reference	Seq.	Frames	Median depth (m)
Driving and infrastructure
KITTI	passenger vehicle	LiDAR	96	5,211	11.755
nuScenes	passenger vehicle	LiDAR	116	4,627	11.585
Waymo	passenger vehicle	LiDAR	149	5,951	19.095
Argoverse	passenger vehicle	LiDAR	70	6,613	21.215
V2X-Radar-I	roadside unit	LiDAR	30	1,092	36.865

Table 2: MetricBench: platforms, sensor references, and physical scale. Counts refer to the evaluated subsets. Reference identifies the sensor source of geometry; trajectory references use dataset metric poses. Median depth is computed from valid projected sensor samples in metres. Seq.: sequences; UGV: unmanned ground vehicle. Subset construction and complete scale statistics are given in Appendix D .

Model	Scale Recovery	Autonomous Driving and Infrastructure						Embodied AI
Model	Scale Recovery	KITTI	nuScenes	Waymo	Argoverse	V2X- Radar-I	DAIR- V2X-I	CEAR	JRDB	Fusion Portable	DROID	Humanoid Everyday	3EED
VGGT	None	0.974	0.983	0.983	0.974	0.943	0.965	0.850	0.805	0.949	0.439	0.359	0.936
VGGT	UniMetric	0.722	0.316	0.311	0.311	0.392	0.866	0.551	0.520	0.676	0.332	0.335	0.534
VGGT- $\Omega$	None	0.971	0.979	0.983	0.972	0.949	0.966	0.786	0.790	0.885	0.294	0.311	0.937
VGGT- $\Omega$	UniMetric	0.347	0.152	0.154	0.244	0.123	0.542	0.538	0.312	0.331	0.284	0.221	0.343
$\pi^{3}$	None	0.190	0.297	0.165	0.282	0.341	0.704	0.328	0.310	0.330	0.315	1.009	0.354

Table 3: Metric depth estimation on MetricBench. Absolute relative depth error (Abs Rel, lower is better) before (None) and after UniMetric correction, without ground-truth scale alignment. Bold marks the better result within each None/UniMetric pair. DA-3 denotes Depth Anything 3.

Model	Scale Recovery	Autonomous Driving and Infrastructure				Embodied AI
Model	Scale Recovery	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	CD (m) $\mathord{\downarrow}$	ATE (m) $\mathord{\downarrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	CD (m) $\mathord{\downarrow}$	ATE (m) $\mathord{\downarrow}$
VGGT	None	0.970	0.0	20.186	42.574	0.723	11.5	5.354	10.302
VGGT	UniMetric	0.486	48.9	4.038	14.878	0.491	35.5	1.189	6.406
VGGT- $\Omega$	None	0.970	0.0	19.167	42.332	0.667	13.8	4.763	9.885
VGGT- $\Omega$	UniMetric	0.260	73.8	1.292	6.746	0.338	55.7	0.628	2.037
$\pi^{3}$	None	0.330	33.0	2.506	8.869	0.441	34.1	0.880	3.298

Table 4: Suite-averaged metric geometry on MetricBench. Metrics are averaged equally over applicable subsets; Average further averages over the nine models.

\delta_{1}

denotes depth accuracy within a factor of 1.25 (%), CD denotes Chamfer distance (m), and ATE denotes mean absolute trajectory error (m). Roadside subsets are excluded from ATE. Bold marks the better result within each None/UniMetric pair.

Model	Scale Recovery	Autonomous Driving and Infrastructure						Embodied AI
Model	Scale Recovery	KITTI	nuScenes	Waymo	Argoverse	V2X- Radar-I	DAIR- V2X-I	CEAR	JRDB	Fusion Portable	DROID	Humanoid Everyday	3EED
VGGT	None	23.094	15.127	20.034	16.584	22.986	23.293	3.216	3.466	18.317	0.224	0.119	6.783
VGGT	UniMetric	6.797	1.850	3.073	1.737	4.089	6.685	1.222	1.418	2.859	0.187	0.117	1.328
VGGT- $\Omega$	None	22.773	14.594	19.060	15.462	21.698	21.415	2.555	3.090	15.848	0.158	0.151	6.778
VGGT- $\Omega$	UniMetric	1.906	0.726	0.630	1.215	1.244	2.033	1.037	0.772	0.653	0.118	0.135	1.053
$\pi^{3}$	None	1.668	1.710	0.848	1.596	3.454	5.761	0.893	0.952	0.971	0.090	0.259	2.116

Table 5: 3D reconstruction across autonomous-driving, infrastructure, and embodied-AI benchmarks. We report CD (Chamfer distance) in metres; lower values are better. Bold marks the better result within each None/UniMetric pair. DA-3 denotes Depth Anything 3. Results use the evaluation subsets described in Sec. D.1 .

Model	Scale Recovery	Autonomous Driving				Embodied AI
Model	Scale Recovery	KITTI	nuScenes	Waymo	Argoverse	CEAR	JRDB	Fusion Portable	DROID	Humanoid Everyday	3EED
VGGT	None	83.455	24.125	34.825	27.892	8.296	2.418	41.462	0.015	0.150	9.471
VGGT	UniMetric	36.194	5.146	12.331	5.840	8.838	2.397	21.360	0.015	0.158	5.666
VGGT- $\Omega$	None	83.272	23.792	34.364	27.901	7.374	2.442	39.863	0.007	0.136	9.485
VGGT- $\Omega$	UniMetric	15.511	2.476	5.090	3.906	2.715	0.921	4.238	0.009	0.101	4.237
$\pi^{3}$	None	15.176	6.738	5.780	7.781	3.054	0.964	10.318	0.018	0.346	5.090

Table 6: Ego pose estimation across autonomous-driving and embodied-AI benchmarks. We report ATE Mean (mean absolute trajectory error): the average camera-position error in metres after rigid alignment of each sequence with scale fixed to 1; lower is better. Bold marks the better result within each None/UniMetric pair; ties at the displayed precision are not bolded. DA-3 denotes Depth Anything 3.

Scale calibration	nuScenes				FusionPortable
Scale calibration	Abs Rel $\downarrow$	$\delta_{1}\uparrow$	CD $\downarrow$	ATE $\downarrow$	Abs Rel $\downarrow$	$\delta_{1}\uparrow$	CD $\downarrow$	ATE $\downarrow$
None	0.979	0.0	14.594	23.792	0.885	0.9	15.848	39.863
Depth Pro	0.212	55.6	1.107	5.134	0.543	22.6	9.378	66.352
UniDepth v2	0.278	53.1	1.638	13.515	0.485	14.8	1.450	23.485
Depth Anything V2	0.347	29.0	1.705	11.423	0.396	47.0	0.702	10.855
ZoeDepth-NK	0.218	65.9	1.044	6.965	0.432	49.4	3.940	56.668

Table 7: Scale calibration of VGGT-

\Omega

predictions. All rows rescale the same GFM outputs by one factor per sequence, estimated from metric depth ( Sec. C.2 ) or by UniMetric. Oracle (gray) fits the factor to ground truth separately for each metric. Bold and underline mark the best and second-best non-oracle results.

Variant	nuScenes				FusionPortable
Variant	Abs Rel $\downarrow$	$\delta_{1}\uparrow$	CD $\downarrow$	ATE $\downarrow$	Abs Rel $\downarrow$	$\delta_{1}\uparrow$	CD $\downarrow$	ATE $\downarrow$
Point priors	0.172	79.1	0.989	5.231	0.452	47.4	1.806	19.430
Uniform weights	0.180	67.5	1.235	5.902	0.469	46.9	1.828	22.636
Single anchor	0.334	45.4	1.983	21.643	0.683	18.0	6.086	48.649
UniMetric (full)	0.152	82.0	0.726	2.476	0.331	52.2	0.653	4.238

Table 8: Design ablations of UniMetric with VGGT-

\Omega

. The variants replace scale intervals with their midpoints (Point priors), assign equal weights to valid observations (Uniform weights), or retain only the lowest-weight observation (Single anchor). Bold and underline mark the best and second-best results.

Figure 4: Qualitative scale recovery with VGGT- $\Omega$ on CEAR (top) and KITTI (bottom). Depth maps and camera trajectories compare predictions before (None) and after (UniMetric) scale correction. Point-cloud views show the highlighted regions and their measured heights alongside LiDAR references. These measurements cover visible regions rather than complete objects. GT denotes ground truth; rel. denotes the original reconstruction unit.

Figure 5: Camera trajectory scale recovery across eight GFMs. Rows show models; columns show selected scenes from FusionPortable, CEAR, nuScenes, Waymo, and KITTI. Trajectories are projected onto the $Z$ – $X$ plane (horizontal–vertical). GT and UniMetric (Ours) are in metres; Model retains its original relative scale. Scene labels give the GT path length. Red dots mark the common starting point; insets magnify the original predictions. GT denotes ground truth.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 14 — Figure S1: Height-only user-prompt template, adapted for height-prior acquisition. {semantic_class} is replaced by the annotated category. The output specifies a height interval in metres and its confidence.

Panel of Figure 15 — Figure S2: General object-anchor prompt template. The task parameter {ANCHOR_GROUP} selects movable objects or static scene elements; the template fixes the selection criteria, the physical-height interval, and the response format without a dataset-specific category inventory.

Panel of Figure 16 — Figure S3: General support-surface prompt template. The VLM returns a semantic label and an image region for the most reliable support surface; its normal is then estimated from the reconstructed geometry ( Sec. 3.2 ).

Component	Setting
VLM	seed-1.8-max
Sampling temperature	0.2
Extended reasoning	Disabled
Segmentation model	SAM2.1 Hiera Large
Segmentation prompt and output	Bounding box; one mask per region
Category reliability weights	0.95 / 0.85 / 0.70 / 0.55

Appendix

Table S1: Shared inference and optimization settings. Optional category metadata is separate from the common prompt; geometric confidence filtering follows the GFM configuration.

Dataset	Median depth (m)	P90 depth (m)	Trajectory / seq. (m)
KITTI	11.755	29.345	167.160
nuScenes	11.585	40.495	94.325
Waymo	19.095	47.595	135.491
Argoverse	21.215	53.165	109.193
V2X-Radar-I	36.865	68.115	–
DAIR-V2X-I	33.655	81.245	–

Appendix

Table S2: Physical scale of each subset. Median and 90th-percentile (P90) projected reference depth and mean metric trajectory length per sequence. Dashes mark static roadside cameras excluded from ego-pose evaluation.

Model	Scale	KITTI		nuScenes		Waymo		Argoverse		V2X-Radar-I		DAIR-V2X-I
Model	Recovery	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$
VGGT	None	0.974	0.0	0.983	0.0	0.983	0.0	0.974	0.0	0.943	0.0	0.965	0.0
VGGT	UniMetric	0.722	46.7	0.316	57.7	0.311	42.9	0.311	51.1	0.392	45.6	0.866	49.3
VGGT- $\Omega$	None	0.971	0.0	0.979	0.0	0.983	0.0	0.972	0.0	0.949	0.0	0.966	0.0
VGGT- $\Omega$	UniMetric	0.347	58.2	0.152	82.0	0.154	77.9	0.244	67.4	0.123	86.5	0.542	71.0
$\pi^{3}$	None	0.190	64.7	0.297	20.3	0.165	68.7	0.282	25.3	0.341	18.2	0.704	0.8

Appendix

Table S3: Quantitative depth estimation on six autonomous driving benchmarks. Abs Rel (absolute relative error; lower is better) measures relative depth error;

\delta_{1}

is the fraction of depths within a factor of 1.25 of reference depth, reported in percent (higher is better).

Model	Scale	CEAR		JRDB		FusionPortable		DROID		Humanoid Everyday		3EED
Model	Recovery	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$	Abs Rel $\mathord{\downarrow}$	$\delta_{1}(\%)\mathord{\uparrow}$
VGGT	None	0.850	0.1	0.805	0.2	0.949	0.2	0.439	33.1	0.359	35.5	0.936	0.1
VGGT	UniMetric	0.551	43.7	0.520	35.1	0.676	18.2	0.332	40.9	0.335	40.4	0.534	34.9
VGGT- $\Omega$	None	0.786	0.4	0.790	0.5	0.885	0.9	0.294	42.6	0.311	38.2	0.937	0.1
VGGT- $\Omega$	UniMetric	0.538	65.3	0.312	54.9	0.331	52.2	0.284	51.5	0.221	60.1	0.343	50.2
$\pi^{3}$	None	0.328	47.4	0.310	39.0	0.330	31.8	0.315	49.5	1.009	4.3	0.354	32.9

Appendix

Table S4: Quantitative depth estimation on six embodied AI benchmarks. Abs Rel (absolute relative error; lower is better) measures relative depth error;

\delta_{1}

is the fraction of depths within a factor of 1.25 of reference depth, reported in percent (higher is better).

Model	Scale	KITTI		nuScenes		Waymo		Argoverse		V2X-Radar-I		DAIR-V2X-I
Model	Recovery	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$
VGGT	None	14.798	17.982	17.519	22.928	23.318	27.824	26.115	31.689	38.772	42.720	39.580	46.124
VGGT	UniMetric	9.622	11.842	5.613	9.099	7.653	10.055	8.387	12.292	15.102	18.630	19.193	24.132
VGGT- $\Omega$	None	14.799	17.988	17.455	22.824	23.142	27.529	26.197	31.729	38.872	42.672	39.962	46.505
VGGT- $\Omega$	UniMetric	5.140	9.442	3.061	6.885	3.726	6.568	6.675	11.954	5.029	8.210	9.063	13.800
$\pi^{3}$	None	3.010	4.899	5.659	8.776	4.085	6.110	7.854	11.170	14.543	17.074	24.687	28.306

Appendix

Table S5: Supplementary depth estimation on six autonomous driving benchmarks. MAE (mean absolute error) summarizes average metric-scale depth error, while RMSE (root mean square error) emphasizes larger errors. Both are in metres; lower is better.

Model	Scale	CEAR		JRDB		FusionPortable		DROID		Humanoid Everyday		3EED
Model	Recovery	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$
VGGT	None	3.633	4.641	6.386	8.146	9.811	12.805	0.508	0.695	0.347	0.472	13.116	16.615
VGGT	UniMetric	2.238	3.136	3.645	5.456	6.345	9.117	0.391	0.542	0.328	0.452	6.138	9.111
VGGT- $\Omega$	None	3.339	4.281	6.253	7.884	9.247	12.108	0.335	0.442	0.314	0.425	13.148	16.624
VGGT- $\Omega$	UniMetric	1.891	2.836	2.329	3.951	2.665	4.607	0.291	0.393	0.256	0.374	4.475	7.481
$\pi^{3}$	None	1.446	2.331	2.664	4.218	3.388	5.125	0.300	0.382	0.825	0.932	5.191	8.218

Appendix

Table S6: Supplementary depth estimation on six embodied AI benchmarks. MAE (mean absolute error) summarizes average metric-scale depth error, while RMSE (root mean square error) emphasizes larger errors. Both are in metres; lower is better. Bold marks the better result within each None/UniMetric pair. DA-3 denotes Depth Anything 3. Evaluation subsets are described in Sec. D.1 .

Model	Scale	KITTI		nuScenes		Waymo		Argoverse		V2X-Radar-I		DAIR-V2X-I
Model	Recovery	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$
VGGT	None	45.367	51.918	29.785	34.824	39.999	46.320	33.105	38.572	40.887	45.105	41.042	47.989
VGGT	UniMetric	22.942	26.723	8.332	11.292	14.638	17.650	9.581	12.927	15.938	19.585	20.931	26.192
VGGT- $\Omega$	None	45.317	51.897	29.458	34.420	38.892	45.109	33.079	38.518	40.955	45.016	41.446	48.391
VGGT- $\Omega$	UniMetric	12.269	16.408	4.489	8.125	6.426	9.430	7.581	12.889	5.498	8.717	9.335	14.245
$\pi^{3}$	None	8.557	10.743	9.214	11.803	7.331	9.445	10.445	13.500	15.238	17.910	25.513	29.335

Appendix

Table S7: Supplementary 3D reconstruction on six autonomous driving benchmarks. MAE (mean absolute error) and RMSE (root mean square error) summarize Euclidean distances between corresponding predicted and reference 3D points after rigid alignment, without estimating any additional scale. RMSE emphasizes larger discrepancies. Both are in metres; lower is better. Bold marks the better result within each None/UniMetric pair. DA-3 denotes Depth Anything 3. Evaluation subsets are described in Sec. D.1 .

Model	Scale	CEAR		JRDB		FusionPortable		DROID		Humanoid Everyday		3EED
Model	Recovery	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$
VGGT	None	6.951	7.776	7.059	8.507	37.490	42.829	0.521	0.672	0.438	0.592	14.534	17.992
VGGT	UniMetric	6.740	7.969	5.254	6.779	19.452	23.547	0.444	0.575	0.432	0.582	8.253	11.274
VGGT- $\Omega$	None	6.416	7.166	6.817	8.212	35.921	41.015	0.379	0.480	0.479	0.614	14.522	17.965
VGGT- $\Omega$	UniMetric	4.934	6.054	3.538	5.095	5.374	6.896	0.307	0.423	0.445	0.583	6.786	9.606
$\pi^{3}$	None	4.273	4.951	3.828	5.323	10.405	12.636	0.257	0.359	0.724	0.866	8.537	11.349

Appendix

Table S8: Supplementary 3D reconstruction on six embodied AI benchmarks. MAE (mean absolute error) and RMSE (root mean square error) summarize Euclidean distances between corresponding predicted and reference 3D points after rigid alignment, without estimating any additional scale. RMSE emphasizes larger discrepancies. Both are in metres; lower is better. Bold marks the better result within each None/UniMetric pair. DA-3 denotes Depth Anything 3.

Figure S4: Metric depth recovery on nuScenes (urban driving; 0–46 m). The scene contains a facade, hedges, and pedestrians at different distances. Before correction, no GFM exceeds 0.3% $\delta_{1}$ . A shared sequence-level scale raises the mean to 88.3%, with $\pi^{3}$ reaching 98.5%, while retaining the spatial structure of each prediction.

Figure S5: Metric depth recovery on Waymo (residential street; 0–63 m). Parked vehicles, houses, and the receding roadway span near and far depths. A shared scale increases the mean frame-level $\delta_{1}$ from 0.9% to 85.2%; VGGT improves from 0.0% to 93.1%.

Figure S6: Metric depth recovery on KITTI (shopping street; 0–44 m). Nearby pedestrians and bicycles introduce occlusions and depth discontinuities. The mean frame-level $\delta_{1}$ increases from 0.0% to 71.3%, and $\pi^{3}$ from 0.4% to 89.3%. Rescaling retains the predicted object boundaries.

Figure S7: Metric depth recovery on Argoverse (urban intersection; 0–70 m). Foreground vehicles and a background facade provide a comparison across distances. The mean frame-level $\delta_{1}$ increases from 0.4% to 63.2%; $\pi^{3}$ and VGGT- $\Omega$ reach 92.8% and 84.8%, respectively.

Figure S8: Metric depth recovery on V2X-Radar-I (elevated roadside view; 0–86 m). Vehicles occupy a small part of a largely weakly textured road scene. The mean frame-level $\delta_{1}$ increases from 0.1% to 58.5%, and VGGT- $\Omega$ from 0.0% to 94.7%, illustrating scale recovery from an elevated viewpoint.

Figure S9: Metric depth recovery on DAIR-V2X-I (roadside view; 0–99 m). Foreground cars and a distant tree-lined avenue span the widest depth range among these examples. The mean frame-level $\delta_{1}$ increases from 0.0% to 71.7%; $\pi^{3}$ and STream3R reach 96.7% and 95.7%, respectively.

Figure S10: Metric depth recovery on CEAR (indoor quadruped view; 0–5.5 m). A low viewpoint captures a robot, stacked boxes, and lab furniture. The mean frame-level $\delta_{1}$ increases from 11.4% to 88.0%, and DA-3 from 0.4% to 90.2%. For $\pi^{3}$ , which is already close to metric, $\delta_{1}$ changes from 90.0% to 91.1%.

Figure S11: Metric depth recovery on JRDB (wheeled robot in a hall; 0–11 m). People at different distances appear among columns and a reflective floor. The mean frame-level $\delta_{1}$ increases from 0.6% to 65.1%, with 4RC and DA-3 reaching 81.6% and 80.1%, respectively.

Figure S12: Metric depth recovery on FusionPortable (handheld office capture; 0–4.9 m). Cabinets, monitors, and equipment appear beneath largely blank walls and ceiling. The mean frame-level $\delta_{1}$ increases from 10.2% to 93.9%, with every displayed GFM reaching at least 88.0%.

Figure S13: Metric depth recovery on DROID (tabletop manipulation; 0–3.7 m). A robot gripper reaches over small objects, with lab equipment in the background. STream3R’s $\delta_{1}$ increases from 0.1% to 86.4%, while $\pi^{3}$ changes from 90.6% to 91.0%. The correction is small where the original prediction is already close to metric.

Figure S14: Metric depth recovery on Humanoid Everyday (close-range manipulation; 0–1.2 m). A gripper holds paper above a waste bin at the closest range among these examples. $\pi^{3}$ improves from 0.1% to 96.2% $\delta_{1}$ , and DVLT from 2.9% to 89.1%, while VGGT and VGGT- $\Omega$ remain near 93% and 97%, respectively.

Figure S15: Metric depth recovery on 3EED (aerial view; 0–52 m). The oblique view covers roads, trees, parked cars, and pedestrians. The mean frame-level $\delta_{1}$ increases from 0.2% to 71.6%, with $\pi^{3}$ and DVLT reaching 94.7% and 91.4%, respectively, from an aerial viewpoint.

3D Reconstruction

3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.

See all 950 papers

Jun 1, 2026

Honey, I Shrunk the Arc de Triomphe!

Metric scale monocular geometry estimation has seen significant progress through large-scale data aggregation, yet current foundation models suffer from a persistent ''scale-collapse'' phenomenon: distant landmarks and vast landscapes are metrically underestimated. We hypothesize that this performance gap stems from a training data bottleneck, where existing metric-scale datasets are hardware-constrained to homogenous vehicle-captured LiDAR or short-range indoor scans, or consist of synthetic data that lacks the semantic complexity of the physical world. To bridge this gap, we curate a new metrically-grounded, in-the-wild dataset that we call MetricScenes, gathered from a variety of sources including Internet photo collections and stereo imagery. We estimate camera poses and initial depth maps for each scene using off-the-shelf methods, and recover absolute scale from geo-tagged metadata as well as known stereo camera baselines. We also improve the quality of depth maps derived from MetricScenes via a new two-stage Poisson completion method. Fine-tuning MoGe-2 on our dataset significantly mitigates scale-collapse and achieves superior metric accuracy in unconstrained, open-domain scenes while maintaining state-of-the-art performance on standard benchmarks.

Yuanbo Xiangli, Hanyu Chen, Xueqing Tsang +1

Cornell University · Shanghai Jiao Tong University

3D Foundation Models Metric Depth Estimation Depth Estimation Depth Completion Monocular Depth Estimation Vision Foundation Model Adaptation

Ask PDF

Jan 29, 2026

MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data. We introduce Metric Anything, a simple and scalable pretraining framework that learns metric depth from noisy, diverse 3D sources without manually engineered prompts, camera-specific modeling, or task-specific architectures. Central to our approach is the Sparse Metric Prompt, created by randomly masking depth maps, which serves as a universal interface that decouples spatial reasoning from sensor and camera biases. Using about 20M image-depth pairs spanning reconstructed, captured, and rendered 3D data across 10000 camera models, we demonstrate-for the first time-a clear scaling trend in the metric depth track. The pretrained model excels at prompt-driven tasks such as depth completion, super-resolution and Radar-camera fusion, while its distilled prompt-free student achieves state-of-the-art results on monocular depth estimation, camera intrinsics recovery, single/multi-view metric 3D reconstruction, and VLA planning. We also show that using pretrained ViT of Metric Anything as a visual encoder significantly boosts Multimodal Large Language Model capabilities in spatial intelligence. These results show that metric depth estimation can benefit from the same scaling laws that drive modern foundation models, establishing a new path toward scalable and efficient real-world metric perception. We open-source MetricAnything at http://metric-anything.github.io/metric-anything-io/ to support community research.

Baorui Ma, Jiahui Yang, Donglin Di +5

Li Auto Inc

Vision Foundation Models Metric Depth Estimation 3D Reconstruction Depth Estimation Depth Completion Monocular Depth Estimation

Ask PDF

Jul 13, 2026

FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry

We present FoundationGeo, a two-stage framework that explicitly bridges relative and metric prediction via spatial calibration and principled data design. Stage 1 learns a high-fidelity, affine-invariant geometry model by initializing with DINOv3 and training on a curated 10.2M-sample multi-domain corpus with complementary local-detail supervision, yielding sharp boundaries and strong cross-domain generalization. Stage 2 moves beyond global scaling by introducing lightweight pixel-wise calibration fields for metric estimation: a scale field for spatially varying metric alignment and a ray-direction correction field that mitigates directional bias in point-map geometry, together producing metrically consistent 3D point maps. Beyond model design, we identify camera intrinsic coverage, especially focal length distribution mismatch between training and test data, as a key bottleneck for zero-shot metric generalization: performance drops sharply when test intrinsics fall outside the training distribution. To address this, we synthesize additional training data across diverse focal lengths using a Blender-based data engine, repairing under-covered focal regimes and improving robustness under intrinsic shift. Extensive zero-shot evaluations across seven benchmarks show that FoundationGeo significantly strengthens cross-domain robustness, staying near the top across diverse domains while avoiding the sharp cross-domain performance drops observed in other methods. This consistency translates into the best overall performance, surpassing heavier baselines by over 5.2% on average.

Muxin Liu, Xiaoyang Lyu, Tianhe Ren +7

The University of Hong Kong · Voyager Research, DiDi Chuxing

Monocular 3D Reconstruction Monocular Depth Estimation

Ask PDF

Abstract

3D Reconstruction

Explore similar work

Honey, I Shrunk the Arc de Triomphe!

MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry