GS-Voxel: Fitting-Free Structured Latents for Large-Scale 3DGS Generation

Ming Qian; Zijian Wang; Minchao Sun; Jincheng Xiong; Hang Zhang; Mu Xu; Chi Wang; Baoquan Chen

cs.CVAug 18, 2026

GS-Voxel: Fitting-Free Structured Latents for Large-Scale 3DGS Generation

Authors: Ming Qian, Zijian Wang, Minchao Sun, Jincheng Xiong, Hang Zhang, Mu Xu, Chi Wang, Baoquan Chen

Organizations: Amap, Alibaba · Zhejiang University · Peking University

Abstract

Many scalable latent 3D generators operate on structured tensors, whereas pre-optimized 3D Gaussian Splatting (3DGS) reconstructions are unordered, spatially irregular, and vary widely in primitive count. We present GS-Voxel, a fitting-free structured latent framework, and evaluate it for large-scale aerial 3D Gaussian scene generation. GS-Voxel deterministically converts a compatible pre-optimized 3DGS reconstruction into sparse active voxels without additional per-scene optimization, retaining the sub-voxel positions and rendering attributes of the selected primitives. A GS-specific factorized VAE then separately encodes voxel geometry and local Gaussian attributes into sparse 3D latents whose size grows with the number of occupied voxels rather than being limited by a fixed scene-wide primitive count. We train image-conditioned flow models in the GS-Voxel latent space to generate aerial 3DGS scenes. A key application enabled by GS-Voxel is large-area scene generation: overlap-aware tiled inference extends synthesis beyond a single training crop conditioned on satellite-view images. Our results show that GS-Voxel provides structured latents for pre-optimized aerial 3DGS reconstructions, with latent capacity that grows with the number of occupied voxels.

Figures & tables

Method	Training Data	Condition	Beyond Buildings	Large-Scale	Core Paradigm
Sat2Scene [ 13 ]	Images & Height Map	Layout	✗	✗	Geo. Coloring
Sat2City [ 7 ]	Mesh	Height Map	✗	✓	Building Object Gen.
Sat2Density++ [ 24 ]	Images	Single Satellite	✓	✗	Feedforward
Sat3DGen [ 25 ]	Images	Single Satellite	✓	✗	Feedforward
UrbanWorld [ 29 ]	Mesh + UV	OSM or Layout	✓	✓	Multi-Stage
SynCity [ 3 ]	N/A (training-free)	Text Prompt	✓	✓	Multi-Stage

Table 1: Taxonomic comparison of representative outdoor 3D scene creation systems.

Figure 2: Overview of our structured-latent pipeline. We first convert a compatible pre-optimized 3DGS reconstruction into GS-Voxel, a sparse representation of the retained local Gaussian parameters, and then encode it with a factorized VAE. The Geometry VAE models hierarchical subdivision and occupancy, while the Local Attribute VAE models Gaussian attributes on the decoded support.

Figure 3: Deterministic conversion from a compatible pre-optimized 3DGS reconstruction to GS-Voxel.

Figure 4: Conditional generation pipeline. Given a satellite-view image, a coarse sparse-structure stage predicts scene support; GS-Voxel-specific geometry and attribute stages then generate fine support and Gaussian parameters, yielding a standard 3DGS primitive set.

Layer	$z$	FOV	Pitches (deg)	Views/cell	Views/window
0	$0.0$	$26^{\circ}$	$\{0,30,45,60\}$	$13$	$1{,}300$
1	$0.3$	$18^{\circ}$	$\{15,30,45\}$	$12$	$1{,}200$
2	$0.6$	$14^{\circ}$	$\{0,15,30\}$	$9$	$\phantom{0}900$
3	$0.9$	$14^{\circ}$	$\{15\}$	$4$	$\phantom{0}400$
4	$1.2$	$14^{\circ}$	$\{0\}$	$1$	$\phantom{0}100$
Total				$39$	$3{,}900$

Table 2: Per-layer configuration of the virtual aerial camera rig. Pitch is measured as the deviation from the nadir direction. For

\text{pitch}{=}0^{\circ}

a single yaw is used (top-down view); for

\text{pitch}{>}0^{\circ}

four compass yaws

\{0^{\circ},90^{\circ},180^{\circ},270^{\circ}\}

are sampled. Views/cell counts cameras per

xy

grid cell; Views/window assumes a

10{\times}10

grid.

Figure 5: Examples of retained and rejected aerial renders. Filtering removes unreliable or severely degraded supervision; only the retained views supervise the Local Attribute VAE.

$R$	$K_{\mathrm{in}}$	Avg. Retained Primitives	Avg. Active Voxels	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$
256	1	377K	377K	0.55	18.52	0.425
256	4	971K	377K	0.85	26.82	0.166
256	8	1,340K	377K	0.95	33.26	0.058
256	16	1,602K	377K	0.98	40.04	0.023
256	32	1,737K	377K	0.98	40.94	0.021
512	1	980K	980K	0.87	27.38	0.156

Table 3: Direct GS-Voxel conversion before VAE encoding, evaluated on 10 validation tiles. We report average retained-primitives and active-voxel counts rounded to the nearest thousand (K), together with rendered reconstruction quality. Bold settings denote our default configuration.

Method / Setting	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
$K_{\mathrm{out}}=1$	22.81	0.61	0.350
$K_{\mathrm{out}}=4$ (Ours)	23.09	0.62	0.331
$K_{\mathrm{out}}=8$	22.12	0.58	0.354

Table 4: Choice of

K_{\mathrm{out}}

for the Local Attribute VAE, evaluated using rendered-image reconstruction quality.

Model	Shape IoU $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Single-stage VAE	0.76	21.01	0.54	0.418
Geometry VAE	0.99	–	–	–
Local Attribute VAE	–	23.09	0.62	0.331

Table 5: Separate evaluation of the Geometry VAE and Local Attribute VAE. Shape IoU measures support reconstruction; PSNR, SSIM, and LPIPS measure attribute reconstruction on the target support.

Method	FID	KID
CityDreamer [ 37 ]	97.3	0.096
GaussianCity [ 38 ]	86.9	0.090
EarthCrafter [ 17 ]	69.5	0.061
Ours	28.0	0.020

Table 6: Cross-paper reference for image-level FID/KID. Baselines use different ground-truth sets and camera protocols; values are not directly comparable and no ranking is implied.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Tile-level 3DGS generations conditioned on rendered satellite-view images.

Figure 7: Large-area 3DGS generation (1/2) from constructed satellite-view conditions.

Figure 8: Large-area 3DGS generation (2/2). Overlap-aware tiled inference produces a large-area 3DGS primitive set from the satellite-view condition.

3D Scene Generation

3D scene generation creates structured, explorable environments from text, images, maps, or other inputs.

See all 190 papers

TriFold: Folding irregular 3D Gaussians into structured latents for 3D Scene Generation
Oct 16, 2026Yun Wang, Chiyu Wang, Fan Jiang +53D Scene Generation 3D Representation Learning
City University of Hong Kong · AMAP, Alibaba Group · Hong Kong Polytechnic University
FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation
Jun 23, 2026Haorui Ji, Weizhe Liu, Hongdong Li +1Diffusion Model Alignment 3D Gaussian Splatting
The Australian National University · ByteDance
Fusion-Aware Direct 3D Gaussian Generation with Structured Patch Latent Flows
Aug 18, 2026Yizhao Wang, Jingbo Wang, Guantao Zhang3D Asset Generation 3D Shape Generation
School of Computer Science and Technology, Henan Institute of Science and Technology

Oct 16, 2026J/K move · Enter open · S save

TriFold: Folding irregular 3D Gaussians into structured latents for 3D Scene Generation

Yun Wang, Chiyu Wang, Fan Jiang, Ziyu Zhu+4

City University of Hong Kong · AMAP, Alibaba Group · Hong Kong Polytechnic University

3D Scene Generation 3D Representation Learning 3D Gaussian Splatting Variational Autoencoders

Open paper AskPDF arXiv

Recent progress in 3D generation has motivated growing interest in compact generative representations of complex 3D scenes. For native 3D Gaussian scenes, however, learning such representations remains challenging due to widely varying primitive counts, nonuniform spatial density, and differences in geometry and appearance even among spatially neighboring primitives. In this work, we introduce TriFold, a native 3D Gaussian VAE that encodes Gaussian scenes into a fixed-layout triplane latent and jointly reconstructs their geometry and appearance. The key insight is to impose structural regularity on the latent space while preserving the spatial adaptivity of the underlying Gaussians. During encoding, we use cell-wise learned queries to aggregate projected Gaussians into multiple slots per cell, reducing the information loss caused by single-feature pooling. During decoding, occupancy prediction identifies active regions, where query-based set prediction and candidate selection reconstruct local Gaussian sets with spatially varying primitive counts. Building on this representation, we train a latent flow-matching model for conditional scene-level 3DGS generation, enabling point-cloud-conditioned synthesis and single-image-conditioned amodal generation. Experiments demonstrate that TriFold achieves high-fidelity reconstruction and generation from a single image that provides only a partial observation of the scene, validating its effectiveness for native 3D Gaussian representation learning. https://fantasy-amap.github.io/TriFold

Abstract

3D Scene Generation

Explore similar work

TriFold: Folding irregular 3D Gaussians into structured latents for 3D Scene Generation

FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation

Fusion-Aware Direct 3D Gaussian Generation with Structured Patch Latent Flows