We present FLEG, a feed-forward network that reconstructs language-embedded 3D Gaussians from multi-view images. Previous feed-forward language-embedded Gaussian reconstruction methods are restricted to a fixed number of input views and typically attach a language-aligned semantic embedding to each Gaussian, resulting in impractical input settings and semantic redundancy. In contrast, we introduce a geometry-semantic dual-branch distillation framework that supports a variable number of uncalibrated and unposed multi-view images as input. We also propose a novel-view-based distillation strategy during training that mitigates overfitting to input views. In addition, we observe that semantic representations are significantly sparser than geometric ones, and per-Gaussian language embedding is unnecessary. To exploit this sparsity, we design a decoupled language embedding strategy that represents language information with a sparse set of semantic Gaussians, rather than attaching embeddings to every Gaussian. Compared with dense pixel-aligned per-Gaussian embedding schemes, our method uses only 5% of the language embeddings while maintaining comparable semantic fidelity, effectively reducing storage costs. Extensive experiments demonstrate that FLEG outperforms state-of-the-art feed-forward reconstruction and language-embedded Gaussian methods in both reconstruction quality and language-aligned semantic representation.
Figures & tables
Figure 1: FLEG reconstructs language-embedded Gaussians in a single feed-forward pass from a variable number of uncalibrated and unposed images, supporting both sparse and dense views in one model with the compact semantic representation that avoids per-Gaussian language embedding and significantly reduces storage overhead.
Figure 2: Overview of FLEG. We propose a dual-branch distillation to train the network without 3D ground truth and semantic labels. We also propose a Decoupled Gaussian Language Embedding (DGLE) module that produces the compact semantic representation, which significantly reduces semantic redundancy and storage overhead.
ScanNet++ [ 46 ]
ScanNet [ 5 ]
mIoU ↑
Acc. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
mIoU ↑
Acc. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
2 views
2 views
LSeg [ 16 ]
45.28
59.21
-
-
-
42.04
60.49
-
-
-
[ 28 ] +CLIP [ 26 ]
39.58
62.39
-
-
-
34.67
58.52
-
-
-
AnySplat [ 12 ]
-
-
26.81
0.8532
0.1833
-
-
23.03
0.7701
0.2924
LSM [ 7 ]
39.36
58.35
19.87
0.7359
0.4120
39.22
59.05
22.07
0.7606
0.3912
Table 1: Comparison of feed-forward methods on ScanNet++ and ScanNet for novel view synthesis and open-vocabulary query segmentation across different input views. The best and runner-up results are highlighted in red and orange.
ScanNet++ [ 46 ]
ScanNet [ 5 ]
mIoU ↑
Acc. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
mIoU ↑
Acc. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
AnySplat [ 12 ]
-
-
24.08
0.8095
0.2473
-
-
20.91
0.7618
0.3899
LangSplat [ 25 ]
25.81
59.40
24.02
0.7749
0.3029
41.24
61.67
21.56
0.7974
0.3906
LangScene-X [ 22 ]
21.08
47.61
23.13
0.7824
0.3435
27.39
53.33
20.77
0.8412
0.3832
FLEG (Ours)
46.61
67.18
25.14
0.8235
0.2073
60.97
91.67
23.30
0.7898
0.3678
Table 2: Comparison under dense 32 input view settings on ScanNet++ and ScanNet. The best and runner-up results are highlighted in red and orange.
Figure 3: Qualitative comparisons on novel view synthesis.
Figure 4: Qualitative comparison of open-vocabulary query segmentation.
Emb. Num ↓
Storage (MB) ↓
mIoU ↑
Acc. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
Ours
73.89K
18.04
46.87
72.96
26.66
0.8499
0.1707
w/o sparse (voxel)
1.625M
396.7
47.83
73.14
25.51
0.8303
0.1853
w/o sparse (pixel)
2.012M
491.3
46.69
73.71
26.13
0.8415
0.1647
w/o contrast
77.62K
18.95
40.22
68.35
26.17
0.8365
0.1765
w/o novel distill
82.17K
20.06
46.49
72.44
24.77
0.8165
0.1945
w/o geo. distill
1.001M
244.3
18.56
42.12
14.38
0.5966
0.5536
Table 3: Ablation studies on ScanNet++. Results are averaged over 2, 4, 8, and 16 input views.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Impact of semantic sparsity on mIoU and storage overhead. Within the light-blue region, increasing sparsity significantly reduces the number of embeddings and corresponding storage overhead, while preserving semantic representation capability.
AUC@15 ↑
AUC@30 ↑
2 views
VGGT [ 38 ]
0.2733
0.4367
AnySplat [ 12 ]
0.1653
0.3600
FLEG (Ours)
0.2627
0.4407
4 views
VGGT [ 38 ]
0.4316
0.5978
Appendix
Table 4: Comparison of camera pose estimation on ScanNet++ validation set. The best and runner-up results are highlighted in red and orange.
Input Views Depth Estimation
Novel Views Depth Estimation
AbsRel ↓
δ<1.25↑
AbsRel ↓
δ<1.25↑
2 input views
VGGT [ 38 ]
0.0723
0.9424
-
-
AnySplat [ 12 ]
0.0805
0.9311
0.0840
0.9303
FLEG (Ours)
0.0830
0.9317
0.0860
0.9243
4 input views
Appendix
Table 5: Comparison of depth estimation on ScanNet++ validation set. The best and runner-up results are highlighted in red and orange.
mIoU ↑
Acc. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
2 views
LSeg [ 16 ]
28.93
43.52
-
-
-
SAM2 [ 28 ] +CLIP [ 26 ]
27.23
49.67
-
-
-
AnySplat [ 12 ]
-
-
26.23
0.8482
0.1714
LSM [ 7 ]
22.56
40.11
17.38
0.7135
0.4261
Uni3R [ 35 ]
24.33
41.15
23.92
0.8019
0.2877
Appendix
Table 6: Comparison on unseen Replica dataset for evaluating the generalization capability. The best and runner-up results are highlighted in red and orange.
Figure 6: Qualitative comparison of novel view synthesis on the Replica dataset. Our method produces results closer to the ground truth with fewer artifacts.
Figure 7: Qualitative comparison of open-vocabulary query segmentation on the Replica dataset. Our method produces more accurate segmentation results and better aligns with the text queries compared with baselines.
Figure 8: Additional qualitative results of novel view synthesis on the ScanNet++ validation set.
Figure 9: Additional qualitative results of open-vocabulary query segmentation on the ScanNet++ validation set.
Figure 10: Additional qualitative results of novel view synthesis on the ScanNet validation set.
Figure 11: Additional qualitative results of open-vocabulary query segmentation on the ScanNet validation set.
Figure 12: Visualization of 3D Gaussians and 3D Language Fields on the ScanNet++ validation set.
Figure 13: Visualization of 3D Gaussians and 3D Language Fields on the ScanNet validation set.
Tri-Institutional Georgia Institute of Technology/Georgia State University/Emory University Center for Translational Research in Data Science and Neuroimaging (TReNDS) · Georgia State University · Duke University +1