Recent progress in 3D generation has motivated growing interest in compact generative representations of complex 3D scenes. For native 3D Gaussian scenes, however, learning such representations remains challenging due to widely varying primitive counts, nonuniform spatial density, and differences in geometry and appearance even among spatially neighboring primitives. In this work, we introduce TriFold, a native 3D Gaussian VAE that encodes Gaussian scenes into a fixed-layout triplane latent and jointly reconstructs their geometry and appearance. The key insight is to impose structural regularity on the latent space while preserving the spatial adaptivity of the underlying Gaussians. During encoding, we use cell-wise learned queries to aggregate projected Gaussians into multiple slots per cell, reducing the information loss caused by single-feature pooling. During decoding, occupancy prediction identifies active regions, where query-based set prediction and candidate selection reconstruct local Gaussian sets with spatially varying primitive counts. Building on this representation, we train a latent flow-matching model for conditional scene-level 3DGS generation, enabling point-cloud-conditioned synthesis and single-image-conditioned amodal generation. Experiments demonstrate that TriFold achieves high-fidelity reconstruction and generation from a single image that provides only a partial observation of the scene, validating its effectiveness for native 3D Gaussian representation learning. https://fantasy-amap.github.io/TriFold
Figures & tables
Figure 1: Overview of TriFold. (a) TriFold encodes variable-cardinality Gaussian scenes into fixed-layout triplane latents that jointly represent geometry and appearance, and decodes content-adaptive Gaussian sets with variable local densities. (b) The learned representation supports detailed scene reconstruction. (c) Our single-image-conditioned amodal generation preserves observed content while inferring plausible geometry and appearance for unobserved regions.
Figure 2: Overview of TriFold. The triplane encoder maps a variable-cardinality Gaussian set to a fixed-layout variational latent, and the triplane decoder reconstructs spatially adaptive local Gaussian sets. The learned latent further supports image-conditioned scene generation with flow matching.
Figure 3: Qualitative VAE reconstruction comparison. Compared with GaussianGPT, our method better captures scene geometry and appearance.
Figure 4: Adaptive 3DGS allocation. Visualization of the target scene, VAE reconstruction, and retained Gaussian count per occupied voxel.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
CD ↓
Can3Tok
4.53
0.012
0.920
10.96
DiffGS
12.43
0.719
0.374
3.21
GaussianGPT
21.40
0.856
0.157
2.62
TriFold (Ours)
26.55
0.913
0.088
2.23
Table 1: Quantitative comparison with existing 3D Gaussian representation methods. Chamfer Distance (CD) values are multiplied by 103 . Best results are in bold , second-best are underlined .
Observed Appearance
Unobserved Appearance
Unobserved Geometry
Condition
FID ↓
KID ( ×103 ) ↓
FID ↓
KID ( ×103 ) ↓
COV-CD (%) ↑
MMD-CD ( ×103 ) ↓
Full point cloud
54.12
16.76
N/A (complete conditioning)
Single image
60.38
18.20
67.81
21.43
85.24
12.75
Table 2: Conditional generation evaluation. For single-image conditioning, appearance is evaluated separately in regions observed and unobserved from the conditioning view. Full-point-cloud conditioning covers the complete scene and therefore has no unobserved split. Detailed evaluation protocols are provided in Appendix A.6 .
Figure 5: Point-cloud-conditioned generation. Given a full point cloud, the latent flow model generates native 3DGS with coherent geometry and appearance across novel views.
Figure 6: Single-image amodal generation. Given one input image, the latent flow model generates plausible room structure and appearance across novel views.
Table 9
ID
Architecture
Training objective
Rendering quality
Gaussian reconstruction
Triplane Encoder
Triplane Decoder
Phantom
Lpos
Lrender
PSNR ↑
SSIM ↑
LPIPS ↓
Chamfer Distance (CD) ↓
baseline
✓
12.10
0.617
0.491
3.06
1
✓
✓
14.72
0.752
0.378
2.58
2
✓
✓
13.53
0.669
0.330
2.77
3
✓
✓
✓
17.81
0.803
0.231
2.48
4
✓
✓
✓
22.64
0.885
0.134
2.41
Table 5: Component analysis. Each row adds or replaces one model or training component while keeping the remaining evaluation protocol fixed. For the Lpos -only baseline, we use the same correspondence to directly supervise all Gaussian attributes.
Variant
PSNR ↑
SSIM ↑
LPIPS ↓
CD ↓
Encoder Head
Sparse 3D CNN
23.16
0.830
0.139
2.48
PointNet
24.37
0.878
0.117
2.41
PTv3
25.83
0.904
0.096
2.27
Aggregation
Mean pooling
24.60
0.857
0.102
2.39
Table 6: Architectural design choices. Comparison of alternative encoder, projection, and decoding designs under the same evaluation settings.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Variable cardinality and nonuniform spatial density in scene-level 3DGS. (a-c) Representative InteriorGS scene crops with different primitive counts, visualized using Gaussian centers voxelized on a shared 2563 grid. (d) Across 100 random scene crops, the number of primitives ranges from 351,306 to 1,457,726, with a median of 682,845. (e) The complementary cumulative distribution of occupied-voxel density shows a pronounced long tail: 97.8% of occupied voxels contain at most 24 Gaussians, while the maximum density can even reach 389 Gaussians per voxel. These statistics motivate a representation that accommodates both scene-dependent cardinality and non-uniform local Gaussian allocation.
Figure 8: Additional VAE Reconstructions. Under the identical evaluation setup, our VAE preserves more visible layout, appearance, and fine-grained local structures in these examples.
Figure 9: Representative reconstruction failures of the adapted Can3Tok and DiffGS baselines. Both baselines are trained using their officially released implementations on the same scene-level training split and with the same optimization budget used in our comparison. Under this setting, distinct input scenes can be reconstructed as similar low-detail structures. These examples illustrate the observed behavior under our scene-level adaptation and are not intended to characterize the methods outside this setting.
Figure 10: Encoder feature comparison. We visualize features extracted by PTv3, PointNet, and a sparse 3D CNN from the same Gaussian input. PTv3 retains clearer spatial variation around surfaces, objects, and scene boundaries in this example; Table 6 reports the corresponding quantitative comparison.
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
CD ↓
K=8
25.06
0.887
0.112
2.31
K=16
25.61
0.895
0.106
2.33
K=24 (Ours)
25.83
0.904
0.096
2.27
K=32
25.96
0.907
0.092
2.27
Appendix
Table 11: Effect of local Gaussian decoder capacity. K is the maximum number of Gaussian candidates per selected voxel, while rendering supervision always uses the complete target scene. CD values are multiplied by 103 . Best results are in bold , second-best are underlined .
Figure 11: Single-slot removal. This intervention complements the slot-count ablation in Table 10 . We zero one of the Q encoder slots while keeping all remaining latent values fixed. The top row visualizes decoded feature energy on the xz plane, and the bottom row shows the corresponding Gaussian reconstruction. Different removals produce distinct changes, indicating that the slots are not identical copies of the same local summary.
Figure 12: Latent interpolation and perturbation analysis. (a) Linear interpolation produces smooth changes in decoded triplane features. (b) The latent representation remains stable under moderate perturbations, with pronounced degradation under Gaussian noise at σ=1.0 .
Figure 13: Additional point-cloud-conditioned generations. Each panel shows a Gaussian scene sampled from a different point-cloud condition. The examples illustrate the diversity of generated indoor layouts and scene content.
Figure 14: Single-image-conditioned generation. Each row shows the conditioning view followed by top, front, and side renderings of one sampled Gaussian scene. The novel views assess whether input-specific scene cues are preserved within a coherent 3D completion.