We present Kosmo-GFM, a scene reconstruction system that integrates a geometry foundation model (GFM), a pose optimization engine (PoseEngine), and reconstruction-derived supervision from KosmoWorld. GFM predicts camera parameters, depth, cross-view correspondences, and dynamic 3D points from images and videos. PoseEngine combines these predictions with geometric optimization to recover scene-level camera poses and geometry for downstream 3D and 4D Gaussian Splatting. KosmoWorld generates images and geometric supervision from reconstructed real-world scenes to improve GFM's geometric predictions. On the Sparse and Medium subsets of SpatialBench, GFM achieves the lowest absolute relative depth error and the highest relative-pose AUC at 5 degrees among the compared methods. It also achieves the lowest mean endpoint errors for dynamic points and scene flow across the five evaluated datasets. Across 34 scenes completed by all compared reconstruction systems, PoseEngine achieves the highest scene-averaged pose AUC at thresholds of 1, 3, and 5 degrees, with a cumulative runtime of 14.07 hours, approximately 20% of GLUEMAP's 69.67 hours. PoseEngine Flash provides a further accuracy-efficiency trade-off, with a measured mixed-precision runtime of 2.62 hours, approximately 4% of GLUEMAP's runtime. Separate scaling experiments show generally favorable trends with increased data, model computation, and training budgets. These results support the use of learned geometric priors within a scene-level reconstruction pipeline.
Figures & tables
Figure 1: Kosmo-GFM. Top: static scene reconstructions with camera trajectories. Bottom: dynamic 3D points and motion trajectories.
Figure 2: Overview of Kosmo-GFM. GFM supplies geometric priors, and PoseEngine organizes cross-view constraints and refines scene cameras. The resulting cameras and geometry support point-cloud, 3DGS, and 4DGS reconstruction.
Figure 3: Task-specific decoding from shared multi-view features. Camera and depth decoding recover static geometry, STAR matching establishes cross-view correspondences, and source–target queries predict geometry and motion at a selected time.
Figure 4: Co-visibility versus temporal separation. A sampling step near the lower-bound zero crossing favors low-overlap training clips.
Figure 5: Camera-token attention across views under mixed-camera and pinhole-only training. Each panel overlays the attention of every camera token (rows) on the input views (columns). Top: four pinhole input views. Bottom: four fisheye input views. Both models use the same image sequence in each example.
Figure 6: Mixed pinhole–fisheye reconstruction. Left: two pinhole and two fisheye input views. Right: 3D points predicted by GFM from the four views.
Figure 7: Panoramic reconstruction. Left: four equirectangular input views. Right: 3D points predicted by GFM from the four panoramas.
Figure 8: KosmoWorld training data and supervision. The four panels show high-resolution images, depth, multi-camera formats, and semantic and instance annotations. Video panels show representative recorded frames.
Figure 9: KosmoWorld examples from three viewpoints of the same scene. Each row pairs an RGB image (left) with a point-cloud visualization (right).
Figure 10: Effect of KosmoWorld data on SpatialBench Sparse (top) and Medium (bottom). Each setting compares GFM trained with and without KosmoWorld using depth error and camera AUC. Full data-ablation metrics appear in Table 1 .
Depth
Camera pose
Setting / data
AbsRel
SqRel
RMSE
δ1.03
δ1.05
δ1.10
RAcc3
RAcc5
TAcc3
TAcc5
AUC@5
AUC@15
AUC@30
↓
↓
↓
↑
↑
↑
↑
↑
↑
↑
↑
↑
↑
Sparse / Without
0.0854
0.1209
0.6328
0.6396
0.7515
0.8642
0.8169
0.8872
0.6294
0.7121
0.5422
0.7197
0.8104
Sparse / With
0.0630
0.1093
0.6207
0.6696
0.7810
0.8816
0.8358
0.9055
0.6457
0.7394
0.5726
0.7412
0.8226
Medium / Without
0.0740
0.0956
0.5939
0.6417
0.7689
0.8815
0.7907
0.8672
0.6070
0.6984
0.5374
0.7107
0.8021
Medium / With
0.0594
0.0884
0.5822
0.6895
0.8077
0.9025
0.8310
0.8938
0.6468
0.7322
0.5820
0.7450
0.8281
Table 1: KosmoWorld data ablation on SpatialBench. For each of the Sparse and Medium subsets, we compare models trained with and without KosmoWorld data. Accuracies are fractions, and lower errors are better. Bold indicates the better value within each subset.
Depth
Camera pose
Method
Params
AbsRel
SqRel
RMSE
δ1.03
δ1.05
δ1.10
RAcc3
RAcc5
TAcc3
TAcc5
AUC@5
AUC@15
AUC@30
(M)
↓
↓
↓
↑
↑
↑
↑
↑
↑
↑
↑
↑
↑
VGGT
1256.54
0.105
0.105
0.673
0.498
0.627
0.778
0.734
0.833
0.454
0.550
0.405
0.585
0.700
Fast3R
647.55
0.260
0.760
1.576
0.228
0.340
0.508
0.347
0.486
0.156
0.235
0.109
0.264
0.392
FastVGGT
1157.94
0.113
0.098
0.666
0.457
0.590
0.752
0.642
0.773
0.384
0.472
0.314
0.503
0.631
MUSt3R
423.43
0.165
0.276
0.999
0.366
0.502
0.686
0.622
0.726
0.343
0.435
0.273
0.482
0.614
Table 2: SpatialBench Sparse full results. Baseline metrics are taken from the official SpatialBench release. Depth and camera-pose metrics are grouped by column. Accuracy values are fractions, and parameter counts describe model size. Best, second-best, and third-best values are red , blue , and green , with ties sharing a rank. Ranks use unrounded values. Baseline reference for π3 -X: Wang et al., 2026b ; Pi3 Contributors, 2025 .
Depth
Camera pose
Trajectory
Method
Params
AbsRel
SqRel
RMSE
δ1.03
δ1.05
δ1.10
RAcc3
RAcc5
TAcc3
TAcc5
AUC@5
AUC@15
AUC@30
ATE
RPE t
RPE r
(M)
↓
↓
↓
↑
↑
↑
↑
↑
↑
↑
↑
↑
↑
↓
↓
↓
VGGT
1256.54
0.125
0.602
0.688
0.499
0.613
0.747
0.695
0.782
0.484
0.562
0.432
0.586
0.687
0.727
0.216
2.202
Fast3R
647.55
0.255
0.567
1.539
0.241
0.344
0.496
0.296
0.422
0.164
0.248
0.117
0.272
0.386
6.582
2.055
13.650
FastVGGT
1157.94
0.105
0.086
0.611
0.479
0.597
0.737
0.678
0.771
0.448
0.542
0.379
0.554
0.662
0.738
0.259
3.260
MUSt3R
423.43
0.162
0.327
0.966
0.388
0.522
0.696
0.618
0.734
0.361
0.475
0.296
0.507
0.643
3.097
0.613
2.686
Table 3: SpatialBench Medium full results. Baseline metrics are taken from the official SpatialBench release. Depth, camera-pose, and trajectory metrics are grouped by column. ATE and RPE follow the benchmark alignment and units. Accuracy values are fractions, and parameter counts describe model size. Best, second-best, and third-best values are red , blue , and green , with ties sharing a rank. Ranks use unrounded values.
Depth
Camera pose
Model setting
Params
AbsRel
SqRel
RMSE
δ1.03
δ1.05
δ1.10
RAcc3
RAcc5
TAcc3
TAcc5
AUC@5
AUC@15
AUC@30
(M)
↓
↓
↓
↑
↑
↑
↑
↑
↑
↑
↑
↑
↑
RGB-only
1655.29
0.063
0.109
0.621
0.670
0.781
0.882
0.836
0.906
0.646
0.739
0.573
0.741
0.823
Camera-conditioned
1769.82
0.082
0.167
0.757
0.577
0.707
0.842
1.000
1.000
0.997
0.998
0.996
0.998
0.999
Camera-conditioned + per-frame scale
1788.74
0.067
0.131
0.689
0.630
0.753
0.870
1.000
1.000
0.996
0.998
0.995
0.998
0.999
Table 4: Kosmo-GFM on SpatialBench Sparse: RGB-only GFM and Camera-conditioned GFM, with and without per-frame scale prediction. Columns and precision follow Table 2 . Camera inputs provide additional information, so the pose scores are not directly comparable to uncalibrated pose estimation.
Dataset
AbsRel ↓
AUC@30 ↑
DA3-Giant
VGGT- Ω
Kosmo-GFM
DA3-Giant
VGGT- Ω
Kosmo-GFM
7-Scenes ( Shotton et al., 2013 )
0.0656
0.0557
0.0637
0.7848
0.8067
0.7638
ADT ( Pan et al., 2023 )
0.0627
0.0638
0.0408
0.7551
0.8506
0.9574
DROID ( Khazatsky et al., 2024 )
0.1249
0.0907
0.0709
0.6510
0.6194
0.7046
DTU ( Jensen et al., 2014 )
0.0142
0.0135
0.0126
0.9828
0.9885
0.9836
ETH3D ( Schops et al., 2017 )
0.0346
0.0231
0.0349
0.7858
0.8646
0.7992
Table 5: Sparse per-dataset breakdown. Each metric group is ordered DA3-Giant / VGGT- Ω / Kosmo-GFM. DA3-Giant and VGGT- Ω results are from our local evaluations, separate from the official SpatialBench results in Table 2 .
Dataset
AbsRel ↓
AUC@30 ↑
DA3-Giant
VGGT- Ω
Kosmo-GFM
DA3-Giant
VGGT- Ω
Kosmo-GFM
7-Scenes
0.0577
0.0491
0.0586
0.7878
0.8585
0.8130
ADT
0.0564
0.0423
0.0304
0.8473
0.9365
0.9746
DROID
0.0915
0.0584
0.0506
0.5952
0.5324
0.6064
DTU
0.0108
0.0108
0.0148
0.9844
0.9730
0.9602
ETH3D
0.0282
0.0209
0.0263
0.8723
0.9008
0.8857
Table 6: Medium per-dataset breakdown. Each metric group is ordered DA3-Giant / VGGT- Ω / Kosmo-GFM. DA3-Giant and VGGT- Ω results are from our local evaluations, separate from the official SpatialBench results in Table 3 .
Figure 11: Average dynamic-point and scene-flow performance. GFM achieves the lowest mean EPE for both tasks among the compared methods. Threshold-based accuracy and EPE characterize complementary aspects of motion prediction.
Method
Point APD ↑
Point EPE ↓
Flow acc. ↑
Flow EPE ↓
UniQuery4R
72.7
0.610
71.0
0.351
SpaTrackerV2
48.8
1.443
15.5
1.075
St4RTrack
60.9
0.993
41.4
0.796
TraceAnything
53.1
1.896
54.8
0.642
Any4D
63.4
1.287
59.5
0.525
V-DPM
69.7
0.703
69.3
0.418
Table 7: Dynamic-point and scene-flow averages over Panoptic Studio, PointOdyssey, Dynamic Replica, Aria Digital Twin, and SynthVerse. APD and flow accuracy are percentages, and EPE is in meters. Best, second-best, and third-best values are red , blue , and green , with ties sharing a rank. Means are recomputed from Tables 8 and 9 , with rounding applied only for display. Ranks use unrounded means.
Panoptic Studio
PointOdyssey
Dynamic Replica
Aria Digital Twin
SynthVerse
Mean
Method
APD ↑
EPE ↓
APD ↑
EPE ↓
APD ↑
EPE ↓
APD ↑
EPE ↓
APD ↑
EPE ↓
APD ↑
EPE ↓
SpaTrackerV2
49.93
0.4639
53.45
0.5173
54.34
0.4368
69.39
0.2764
16.67
5.5187
48.76
1.4426
St4RTrack
70.80
0.2479
67.38
0.3140
74.24
0.2594
72.90
0.2958
19.24
3.8485
60.91
0.9931
TraceAnything
70.86
0.2780
39.37
1.0462
60.89
0.5642
75.69
0.2515
18.78
7.3385
53.12
1.8957
Any4D
74.72
0.2361
65.99
0.3783
76.72
0.3463
78.09
0.2378
21.67
5.2362
63.44
1.2869
V-DPM
78.95
0.1757
80.82
0.1857
75.29
0.2438
85.47
0.1725
28.01
2.7367
69.71
0.7029
Table 8: Dynamic-point APD (percent) and EPE (meters). All ten methods are shown on all five datasets. Mean is recomputed as the arithmetic mean of the five displayed dataset values. Best, second-best, and third-best values are red , blue , and green , with ties sharing a rank.
Panoptic Studio
PointOdyssey
Dynamic Replica
Aria Digital Twin
SynthVerse
Mean
Method
Acc. ↑
EPE ↓
Acc. ↑
EPE ↓
Acc. ↑
EPE ↓
Acc. ↑
EPE ↓
Acc. ↑
EPE ↓
Acc. ↑
EPE ↓
SpaTrackerV2
8.78
0.4095
15.08
0.3089
13.09
0.2707
34.40
0.2054
6.10
4.1792
15.49
1.0747
St4RTrack
27.45
0.2259
37.24
0.2190
75.69
0.0858
55.76
0.1481
10.77
3.3004
41.38
0.7958
TraceAnything
23.94
0.2456
64.33
0.1173
83.83
0.0495
79.98
0.0839
22.16
2.7136
54.85
0.6420
Any4D
25.08
0.2406
70.43
0.0940
85.94
0.0451
88.25
0.0562
27.93
2.1907
59.53
0.5253
V-DPM
58.99
0.1152
82.91
0.0563
88.98
0.0381
88.75
0.0559
27.09
1.8237
69.34
0.4178
Table 9: Scene-flow accuracy (percent) and EPE (meters). All ten methods are shown on all five datasets. Mean is recomputed as the arithmetic mean of the five displayed dataset values. Best, second-best, and third-best values are red , blue , and green , with ties sharing a rank.
Method
AUC@1 ↑
AUC@3 ↑
AUC@5 ↑
R@1 ↑
T@1 ↑
Focal AbsRel ↓
ATE nRMSE ↓
Registered
Time (h) ↓
COLMAP
75.73
86.33
89.28
93.58
88.45
0.00389
0.05805
11,315
47.08
GLUEMAP
69.13
86.18
90.66
96.24
89.61
0.00230
0.03176
11,336
69.67
VidMap
57.23
71.10
74.29
78.23
76.61
0.00328
0.33359
9,017
20.32
PoseEngine
78.45
91.28
94.26
98.74
96.00
0.00158
0.02640
11,336
14.07
PoseEngine Flash
62.50
83.64
89.28
95.95
89.87
0.00316
0.02823
11,336
2.62
Table 10: Reconstruction, camera calibration, and trajectory consistency on 34 completed scenes with 11,336 images. Metrics are averaged equally across scenes. AUC, R@1, and T@1 are percentages. ATE is Sim(3)-aligned camera-center nRMSE, normalized by the GT center RMS radius. Runtime is cumulative hours. Bold indicates the best value.
Figure 12: Cumulative reconstruction time and pose accuracy on 34 scenes. Time is decomposed by dataset. Flash uses measured mixed-precision runtime. Pose accuracy is evaluated on registered frames.
Dataset
N
MVLoMa
LoMa Large
Δ
AUC@1 ↑
ATE ↓
AUC@1 ↑
ATE ↓
AUC@1 ↑
ATE ↓
Unordered STAR
ETH3D
25
80.29
0.00879
78.00
0.01996
+2.30
-0.01117
MipNeRF360
9
78.81
0.00903
74.35
0.01433
+4.46
-0.00530
Tanks & Temples
6
84.13
0.43660
81.41
0.43793
+2.72
-0.00133
Oxford Spires
4
11.39
2.57710
6.71
2.68618
+4.69
-0.10908
Table 11: MVLoMa versus LoMa Large on 48 scenes for each STAR policy. AUC@1 is a percentage, and Δ is MVLoMa minus LoMa Large: AUC@1 differences are percentage points, and ATE differences use the displayed means in GT coordinate units. ATE is raw Sim(3)-aligned camera-position RMSE in GT coordinate units, averaged equally over scenes within each dataset. Raw ATE is not aggregated across datasets; dashes mark omitted Overall ATE and Δ ATE. Bold marks the better result in each method pair.
Dataset
Initialization
N
PSNR (dB) ↑
SSIM ↑
LPIPS ↓
MipNeRF360
Reference poses
9
28.19
0.8224
0.2570
GLUEMAP
9
28.40
0.8306
0.2510
PoseEngine
9
26.69
0.7714
0.2739
PoseEngine Flash
9
26.34
0.7439
0.2802
Tanks & Temples
Reference poses
6
25.90
0.8472
0.2543
GLUEMAP
6
25.80
0.8486
0.2525
Table 12: 3DGS rendering after camera refinement. All runs use LiteGS, 30,000 iterations and the same image split. Non-reference methods optimize training cameras and refine each test pose against its test image for 20 steps. The fixed-reference baseline uses neither training-pose optimization nor test-pose refinement. Dataset means weight scenes equally.
Figure 13: 3DGS rendering on three examples each from MipNeRF360 and Tanks & Temples. Columns use reference cameras, GLUEMAP, PoseEngine, and PoseEngine Flash. Within each row, all renderings correspond to the same test image, and non-reference poses receive twenty-step image-conditioned refinement.
Encoder initialization
Local AbsRel ↓
Global AbsRel ↓
AUC@30 ↑
AUC@5 ↑
Depth Anything 3-Base (reference)
0.0793
0.4819
0.9606
0.7742
DINOv2-Base
0.1333
1.0499
0.7823
0.3527
DINOv3-Base
0.1218
0.5506
0.8241
0.3883
LingBot-Vision-Base
0.1175
0.5284
0.8441
0.4064
Table 13: Encoder initialization for the scaling model. Base-size reconstruction models are initialized with the Base variant of each pretrained encoder and trained for 100k iterations on Hypersim with 2–24 input views. The geometry-pretrained Depth Anything 3-Base encoder is listed as a reference. Bold marks the best value among the three general-purpose encoders.
Figure 14: Data scaling for the 0.2B model. More training trajectories reduce depth error and improve pose accuracy at each tested budget. The x-axis is logarithmic. Lines connect measured observations only.
Trajectories
8 GPUs
16 GPUs
32 GPUs
AbsRel ↓
AUC@30 ↑
AUC@5 ↑
AbsRel ↓
AUC@30 ↑
AUC@5 ↑
AbsRel ↓
AUC@30 ↑
AUC@5 ↑
1,879
0.65917
0.12169
0.01456
0.66276
0.12468
0.01498
0.71213
0.11167
0.01621
9,394
0.47453
0.28411
0.05513
0.48733
0.28863
0.05477
–
–
–
18,788
0.41348
0.32196
0.06557
0.41262
0.34058
0.06877
0.39743
0.34915
0.07881
93,940
0.32707
0.38433
0.08666
0.32710
0.42081
0.10139
0.31562
0.42707
0.11179
187,880
0.30391
0.39172
0.10103
0.28976
0.43517
0.13044
0.27833
0.44961
0.13702
Table 14: Data-scaling observations for the 0.2B model at 200k steps. AUC values are fractions, and dashes denote an unreported setting.
Figure 15: Model scaling using 8 GPUs for 200k steps. Depth and pose accuracy improve as model scale increases.
Figure 16: Training-compute scaling across four model scales. Each series uses 200k training steps with 8, 16, or 32 GPUs; the horizontal axis reports the corresponding GPU-step budget.
Figure 17: Recurrent-depth scaling using 8 GPUs for 200k steps. The 2.5B, 3.2B, and 4.8B variants reuse the 1.7B model’s weights; their reported scales count parameter uses in the expanded computation graph, including weight reuse, rather than unique trainable parameters. The 0.6B and 1.7B models serve as references. Accuracy continues to improve at larger scales, with diminishing marginal gains.
Figure 18: Camera pose and sparse geometry. Left and center: sparse points of each reconstruction, framed independently to show the main structure. Right: complete registered trajectories with circles/crosses at the start/end, retaining the recorded alignment in a common coordinate system.
Figure 19: Camera-conditioned dense points. Each scene shows a dense-point reconstruction above a small reference input image. We compute the confidence threshold separately for each chunk as the 20th percentile of its point confidence values. For each selected scene, we accumulate point predictions from all available frames.
Figure 20: Feed-Forward 3DGS. Columns show eight input views and rendered views for Kosmo-GFM and InfiniSplat v2 with GT poses. The rendering viewpoints may differ between methods, so these are not matched-pixel quantitative comparisons. Counts describe representation size, not reconstruction accuracy.
Figure 21: 3DGS. Four views along the recorded camera trajectory are shown for each of Scene 1 and Scene 2. All panels are rendered outputs of the reconstructed scenes.
Figure 22: Dynamic-point sequences. Each row shows an input frame and reconstructions at three time points from a fixed viewpoint. Backgrounds, masks, and motion trails use consistent display settings.
Figure 23: 4DGS on VVC 2025 sequences. Top: Scene 1. Bottom: Scene 2. Columns show 1.0, 2.5, and 4.0 seconds from a fixed viewpoint for each sequence.
Figure 24: Qualitative examples. (a) Stairwell input frames and the complete camera trajectory, displayed vertically. (b) Portrait-oriented video frames and the complete camera trajectory. (c) Four input photos of 7 West 54th Street and a point cloud overlaid with GFM cameras (orange) and GT cameras (blue).
East China University of Science and Technology, 130 Meilong Road, Shanghai, 200237, China · Shanghai Open University, 288 Guoshun Road, 200433, Shanghai, China · Shanghai Xiaoyuan Innovation Center, 599 Xingmei Road, 200237, Shanghai, China