METRIC-GS: Selective Metric-Scale Estimation for 3D Gaussian Splatting
Organizations: Department of Mechanical Engineering, Stony Brook University · Department of Mechanical Engineering, SUNY Korea · Department of Computer Science, SUNY Korea
Abstract
A 3D Gaussian Splatting (3DGS) reconstruction inherits arbitrary units from structure-from-motion, so it cannot be measured or used to place AR content until someone supplies a scale. Both standard recipes are fragile on real captures: on five indoor scenes with tape-measured ceilings, a quantile floor-to-ceiling heuristic errs by 27% and metric-depth alignment by 35-71% across three networks. We present METRIC-GS, a selective scale estimator for real indoor photographs: it reads the scale off the reconstruction's own semantic content and, just as importantly, refuses when the evidence is thin. A frozen 2D teacher labels Gaussians by visibility-aware voting; instance-level size priors for architectural and furniture classes (doors, tables, the floor-to-ceiling gap) are fused by a robust maximum-likelihood step; and an inlier-mass test turns thin evidence into an explicit abstention that doubles as a router to a heavyweight metric model. On 47 ScanNet++ scenes with laser ground truth, with a configuration frozen before public evaluation, the estimator alone cuts the heuristic's mean error from 21.1% to 16.6% (9.3% on the 28 scenes it acts on) at ~11 s per scene. MASt3R remains the more accurate standalone estimator (9.9%), but routing only our refusals to it matches running it everywhere with 60% fewer invocations; this selective hybrid is the configuration we advocate. Visibility-aware voting raises the scenes the estimator can act on from 16 to 28 at unchanged deployed accuracy. Applied unchanged to 13 phone and drone captures of our own, on which MASt3R fails its own registration gate 12 times, the system improves all five captures it acts on and degrades none. We also report that a current feed-forward metric model scores 2.3% on a benchmark inside its training mixture and 28-35% on captures it has never seen: a published metric-scale number is a best case, not a deployment estimate.
Figures & tables
| mean | median | on our 28 | |
| Quantile heuristic | 21.1% | 17.5% | 16.8% |
| DA-3 alignment | 28.8% | 22.5% | 30.2% |
| MoGe-2 alignment ‡ | 2.3% | 1.9% | 2.1% |
| MASt3R (metric) | 9.9% | 7.9% | 10.2% |
| Ours (frozen) | 16.6% | 10.6% | 9.3% |
| Ours (Swin-L teacher) † | 15.7% | 8.7% | 7.4% |
| capture | nom. | quantile | MoGe-2 | ours | mass | cues | decision |
|---|---|---|---|---|---|---|---|
| Apartment A, session 1 | 2.3 | 2.90 (+26) | 1.77 (-23) | 2.65 (+15) | 2.03 | 45 | act |
| Apartment A, session 2 | 2.3 | 3.03 (+32) | 2.40 (+4) | 2.36 (+3) | 1.89 | 57 | act |
| Apartment A, session 3 | 2.3 | 2.75 (+20) | 3.48 (+52) | 2.75 (+20) | 0.73 | 21 | abstain |
| Home B, phone 1 | 2.3 | — | — | — | 0.98 | 33 | abstain |
| Home B, phone 2 | 2.3 | 2.62 (+14) | 2.04 (-11) | 2.62 (+14) | 0.84 | 30 | abstain |
| Home C, stills | 2.3 | 1.91 (-17) | 1.34 (-42) | 1.91 (-17) | 0.52 | 37 | abstain |
Appendix figures & tables44 assets
Supplementary material from the paper’s appendix.
Appendix
| claim (main paper) | evidence |
|---|---|
| Frozen system: 16.6%/10.6% on 47 laser-GT ScanNet++ scenes; acts on 28, abstains on 19 | Tab. 1; per scene supp. Tab. S2 |
| Router parity: routing only abstentions to MASt3R matches MASt3R everywhere ( pp, n.s.; TOST-equivalent at pp) | §4.2; supp. § N , Tabs. S17 – S18 |
| Prior sensitivity: uniform shifts, heterogeneous and regional catalogs, MC worst case | §4.5; supp. § H , Tabs. S9 – S10 |
| Abstention threshold: frozen on the mean plateau; would score 15.2% (reported) | §4.5; supp. Tab. S8 |
| Gate needs no per-teacher recalibration (optimum static under a mIoU teacher) | §4.5; supp. § K , Tab. S14 |
| Visibility gate: one-sided culling; removal or softening costs actionability or accuracy | §4.5; supp. § L , Tab. S16 |
| # | scene | # | scene | # | scene | # | scene |
|---|---|---|---|---|---|---|---|
| 1 | 40aec5fffa | 6 | fb5a96b1a2 | 11 | a24f64f7fb | 16 | 3e8bba0176 |
| 2 | 38d58a7a31 | 7 | 7831862f02 | 12 | 1ada7a0617 | 17 | 5eb31827b7 |
| 3 | bde1e479ad | 8 | acd95847c5 | 13 | f9f95681fd | 18 | cc5237fd77 |
| 4 | 21d970d8de | 9 | c50d2d1d42 | 14 | 5ee7c22ba0 | 19 | 3f15a9266d |
| 5 | c4c04e6d6c | 10 | 5748ce6f01 | 15 | 7b6477cb95 |
| scene | quantile | MASt3R | ours | decision |
|---|---|---|---|---|
| 09c1414f1b | 2.8 | 25.3 | 7.0 | active |
| 0d2ee665be | 9.6 | 9.2 | 24.6 | active |
| 13c3e046d7 | 13.9 | 13.8 | 13.9 | abstain |
| 1ada7a0617 | 15.0 | 12.1 | 5.3 | active |
| 21d970d8de | 28.1 | 3.0 | 9.8 | active |
| 25f3b7a318 | 14.1 | 35.3 | 23.7 | active |
| class | what | |||
|---|---|---|---|---|
| bed | v | 0.55 | 0.15 | 0.30 |
| chair | v | 0.88 | 0.15 | 0.40 |
| coffee table | v | 0.45 | 0.10 | 0.30 |
| counter | v | 0.90 | 0.10 | 0.50 |
| countertop | v | 0.90 | 0.08 | 0.50 |
| desk | v | 0.75 | 0.08 | 0.60 |
| nominal | GT | |||
| PSNR | ||||
| room1 | 26.73 | 27.39 | 27.67 | 27.61 |
| room2 | 28.86 | 29.89 | 29.97 | 29.58 |
| studio1 | 22.56 | 26.70 | 26.33 | 26.45 |
| Gaussians (k) | ||||
| room1 | 376 | 299 | 286 | 287 |
| estimator | mean | median |
|---|---|---|
| room-height prior only | 40.5% | 10.7% |
| gap cue only | 39.6% | 9.4% |
| uniform cue weights | 47.9% | 13.1% |
| full fusion | 16.6% | 10.6% |
| removed | mean | med. | abst. | removed | mean | med. | abst. |
|---|---|---|---|---|---|---|---|
| none (frozen) | 16.6 | 10.6 | 19 | person | 16.8 | 10.6 | 18 |
| floor ceil. gap | 21.1 | 17.5 | 47 | stool | 16.7 | 10.6 | 20 |
| door | 18.5 | 13.8 | 25 | refrigerator | 16.7 | 10.6 | 19 |
| chair | 18.1 | 12.0 | 22 | swivel chair | 16.7 | 10.6 | 22 |
| bed | 17.7 | 11.5 | 20 | toilet | 16.7 | 11.1 | 19 |
| table | 17.6 | 12.4 | 22 | counter | 16.6 | 10.6 | 19 |
| height | quantile | ours | median | active | abstain |
|---|---|---|---|---|---|
| 2.1 m | 21.3% | 13.8% | 9.5% | 7.4% | 23 |
| 2.5 m (frozen) | 21.1% | 16.6% | 10.6% | 9.3% | 19 |
| 3.0 m | 29.5% | 25.3% | 12.3% | 13.1% | 20 |
| mean | median | #abstain | |
|---|---|---|---|
| 0.50 | 17.1% | 8.7% | 2 |
| 0.75 | 17.3% | 8.7% | 4 |
| 1.00 | 15.5% | 8.7% | 7 |
| 1.25 | 15.2% | 9.8% | 13 |
| 1.50 (frozen) | 16.6% | 10.6% | 19 |
| 1.75 | 17.9% | 12.9% | 32 |
| variant | mean | median | #abstain |
|---|---|---|---|
| frozen configuration | 16.6% | 10.6% | 19 |
| all prior means | 17.9% | 13.7% | 22 |
| all prior means | 18.5% | 13.9% | 17 |
| gap prior | 17.3% | 9.8% | 16 |
| gap prior | 16.9% | 10.2% | 17 |
| gap prior | 17.0% | 11.1% | 16 |
| catalog | mean | median | #abstain |
|---|---|---|---|
| frozen configuration | 16.6% | 10.6% | 19 |
| all means (uniform) | 17.9% | 13.7% | 22 |
| all means (uniform) | 18.5% | 13.9% | 17 |
| EU catalog (heterogeneous) | 16.9% | 11.1% | 19 |
| JP catalog (heterogeneous) | 17.3% | 11.9% | 18 |
| signed adversarial example | 17.2% | 12.2% | 17 |
| video | mass | cues | err | mass | cues | err |
|---|---|---|---|---|---|---|
| 41125709 | 0.53 | 27 | 15.5 | 0.53 | 27 | 7.4 |
| 41125718 | 0.36 | 23 | 6.8 | 0.22 | 28 | 7.3 |
| 41125722 | 0.16 | 33 | 6.9 | 0.24 | 28 | 5.4 |
| 41125731 | 0.38 | 26 | 10.5 | 0.05 | 19 | 11.9 |
| 41125756 | 0.28 | 25 | 0.5 | 0.68 | 33 | 0.5 |
| variant | mean | median | CI vs. full (pp) |
|---|---|---|---|
| full | 11.0% | 7.0% | — |
| no in-loop scale | 9.8% | 5.5% | |
| no depth reg. | 12.7% | 9.7% | |
| no metric densif. | 11.2% | 8.1% |
| scene | full | no in-loop | no depth reg. | no metric dens. |
|---|---|---|---|---|
| 1ada7a0617 | 5.3 | 15.0 † | 15.0 † | 5.6 |
| 21d970d8de | 9.8 | 7.8 | 5.3 | 10.0 |
| 38d58a7a31 | 10.6 | 8.4 | 11.6 | 10.3 |
| 3e8bba0176 | 7.0 | 9.2 | 11.4 | 8.1 |
| 3f15a9266d | 2.1 | 2.4 | 1.8 | 1.8 |
| 40aec5fffa | 47.2 † | 47.2 † | 47.2 † | 47.2 † |
| R50 (47.2 mIoU) | Swin-L (57.3 mIoU) | |||
|---|---|---|---|---|
| mean | #abstain | mean | #abstain | |
| 0.50 | 17.1% | 2 | 18.5% | 1 |
| 0.75 | 17.3% | 4 | 17.8% | 3 |
| 1.00 | 15.5% | 7 | 16.7% | 4 |
| 1.25 | 15.2% | 13 | 15.2% | 14 |
| 1.50 (frozen) | 16.6% | 19 | 15.7% | 17 |
| PSNR | SSIM | LPIPS | Gauss. (k) | PSNR | |
|---|---|---|---|---|---|
| full system | 32.62 | 0.9508 | 0.1194 | 320 | — |
| no in-loop scale | 32.55 | 0.9506 | 0.1197 | 322 | |
| no depth reg. | 32.66 | 0.9510 | 0.1191 | 321 | |
| no metric densif. | 32.68 | 0.9514 | 0.1185 | 322 |
| voting variant | mean | median | abstain | active | router @calls |
|---|---|---|---|---|---|
| one-sided (ours) | 16.6% | 10.6% | 19 | 28 | 9.4% @19 |
| contribution-weighted | 17.6% | 11.6% | 22 | 25 | 9.8% @22 |
| symmetric | 18.7% | 13.6% | 27 | 20 | 9.9% @27 |
| no gating | 16.4% | 12.4% | 31 | 16 | 9.0% @31 |
| router statistic | mean @19 calls | best mean | calls at best |
|---|---|---|---|
| door coverage | 15.3% | 9.6% | 37 |
| ceiling coverage | 14.5% | 9.9% | 47 |
| door ceiling coverage | 14.6% | 9.6% | 45 |
| inlier mass (ours) | 9.4% | — | 19 |
| statistic | value [95% CI] |
|---|---|
| quantile mean | 21.1 [16.9, 25.7] |
| quantile median | 17.5 [13.9, 21.4] |
| ours mean | 16.6 [12.2, 21.6] |
| ours median | 10.6 [7.7, 13.9] |
| ours active mean (28) | 9.3 [6.6, 12.4] |
| quantile active mean (28) | 16.8 [13.3, 20.5] |
| clearance rule | mean | median | active | abstain | degraded |
|---|---|---|---|---|---|
| none (frozen) | 16.6% | 10.6% | 9.3% | 19 | 9 |
| drop | 16.1% | 10.3% | 8.7% | 18 | 9 |
| drop | 16.3% | 11.1% | 8.5% | 21 | 7 |
| drop | 17.4% | 12.5% | 9.1% | 26 | 7 |
| drop | 17.8% | 12.9% | 7.2% | 31 | 4 |
| tilt | mean | median | abstain |
|---|---|---|---|
| (selected) | 16.6% | 10.6% | 19 |
| 17.4% | 11.5% | 19 | |
| 16.8% | 10.2% | 18 | |
| 18.0% | 12.4% | 22 | |
| 20.1% | 13.9% | 29 | |
| 25.8% | 16.8% | 29 |
| R50 (47.2 mIoU) | Swin-L (57.3 mIoU) | |
| mean | 16.6% | 15.7% |
| median | 10.6% | 8.7% |
| both-act subset (24) | 9.7% | 6.3% |
| active mean (28 / 30 scenes) | 9.3% | 10.3% |
| worst failure scene | 24.6% | 8.6% |
| scenes acted on | 28 | 30 |
| scene | quantile | ours | MoGe-2 | mass | decision |
|---|---|---|---|---|---|
| delivery_area | 51.3 | 51.3 | 3.7 | 0.5 | abstain |
| kicker | 14.4 | 14.4 | 0.3 | 1.4 | abstain |
| office | 16.5 | 9.1 | 1.9 | 1.9 | active |
| pipes | 9.6 | 9.6 | 16.5 | 1.0 | abstain |
| set | in MoGe-2 training? | MoGe-2 | ours |
|---|---|---|---|
| ScanNet++ (47) | yes (176K fr., 4.6%) | 2.3% | 16.6% |
| ETH3D (4) | no (its eval set) † | 5.6% | 21.1% |
| development captures (5) | no | 34.8% | 7.7% |
| in-the-wild captures (9) ∗ | no | 28.4% | 12.3% |
| scene | quantile | DA-3 | Depth Pro | MoGe-2 | ours |
|---|---|---|---|---|---|
| room1 | -39.1% | -62.4% | -78.1% | -50.8% | -15.1% |
| living_1 | -15.7% | -68.9% | -65.5% | -25.8% | +8.5% |
| living_2 | -15.7% | -60.8% | -65.5% | -25.8% | +6.7% |
| room2 | -46.3% | -74.6% | -77.0% | -42.1% | -5.5% |
| studio1 | -20.5% | -64.6% | -68.5% | -29.4% | +3.0% |
| mean err | 27.4% | 66.3% | 70.9% | 34.8% | 7.7% |
| component | wall clock |
|---|---|
| model + camera loading | 6.2 s |
| teacher 2D inference (127 unique views) | 3.9 s (31 ms/view) |
| rendered-depth visibility ( renders) | 0.2 s |
| vote projection + instance measurement (3 seeds) | 0.6 s |
| robust MLE (2001-point grid) | 0.01 s |
| total | s |
| method | flipped decisions | scenes with any flip |
|---|---|---|
| quantile heuristic | 87/235 (37.0%) | 44/47 |
| DA-3 alignment | 101/235 (43.0%) | 43/47 |
| MoGe-2 alignment ‡ | 11/235 (4.7%) | 9/47 |
| MASt3R (metric) | 67/235 (28.5%) | 38/47 |
| Ours (frozen) | 69/235 (29.4%) | 40/47 |
| Ours + MASt3R router | 58/235 (24.7%) | 36/47 |
| assumed truth (m) | ours | MoGe-2 | ours closer |
|---|---|---|---|
| 2.1 | 19.6% | 26.3% | 4/8 |
| 2.2 | 16.2% | 26.2% | 4/8 |
| 2.3 | 13.3% | 27.1% | 5/8 |
| 2.4 | 11.1% | 28.0% | 6/8 |
| 2.5 | 9.7% | 29.9% | 7/8 |
| 2.6 | 8.4% | 31.6% | 7/8 |
3D Reconstruction
3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.
See all 1,017 papers