Generating high-fidelity voxels while keeping their lightweight nature remains an open problem in computer vision. Current generators are either high-fidelity but computationally heavy, or lightweight but of lower fidelity. Hierarchical methods keep the lightweight nature of voxels. However, their coarse levels, which decide where all finer cells can appear, are generated with the least information about the shape and are never revisited, so they are often plausible but wrong. Our key insight is to let finer cells take part in deciding coarse cells. To this end, we propose JoVox, a discrete flow matching framework that learns the joint distribution of all levels of the voxel hierarchy, instead of a separate distribution for each level, so that coarse cells can be predicted with finer cells already generated nearby. Within this framework, we further introduce a two-level prediction, which outputs the cells two levels finer at once instead of one level finer, halving the number of levels that the network processes and yielding voxels of higher fidelity. We also introduce an unmasking strategy that revisits early decisions once finer cells are generated and corrects inconsistencies in masked diffusion models. Our framework outperforms the baselines on both unconditional and image-conditioned voxel generation. Compared with the current state-of-the-art TRELLIS.2, we achieve 8x higher training throughput, 1/15 of the FLOPs in a forward pass, train on 4 vs. 32 GPUs, and obtain higher quality with an 11% lower Chamfer distance.
Figures & tables
Method
Hierarchy
No autoencoder
Parallel
Joint levels
TRELLIS, TRELLIS.2
✗
✗
✓
–
LAS-Diffusion
✗
✓
✓
–
DVD
✗
✓
✓
–
XCube
✓
✗
✓
✗
Nexus, OctFusion
✓
✓
✓
✗
Octree Transformer, OAT
✓
✓
✗
✗
Table 1: Comparison of voxel generation methods.
Figure 2: (a) Two-level prediction. For each cell u at level l , the network predicts the occupancy codes of the eight children of u at level l+1 . The occupancy code of the j -th child consists of the values aj,1,…,aj,8 of its eight children at level l+2 , where a value is 1 for an occupied cell, 0 for an empty one and ∅ for a cell whose occupancy is unknown. When this occupancy code is masked, the value aj of the j -th child and aj,1,…,aj,8 are all ∅ . (b) The network. Each cell u at level 0 and each occupied cell u at even levels has one token. The token h embeds the values aj and aj,b , the level l and the position r . A transformer processes the tokens of all levels together, conditioned on c and on the fractions t of unmasked occupancy codes, and outputs for u eight distributions over the 256 values of an occupancy code.
Voxels
Meshes
Method
CD V↓
IoU ↑
LPIPS V↓
DINO V↑
CD shell ( ×10−3 ) ↓
LPIPS N↓
CLIP N↑
Uni3D ↑
TripoSR ∗‡
0.01663
0.3199
0.1837
0.7813
0.781
0.2291
0.7431
0.3414
TripoSG ∗‡
0.01325
0.3722
0.1626
0.8523
0.525
0.1698
0.7592
0.3849
Direct3D-S2 ‡
0.01246
0.3694
0.1709
0.8550
0.539
0.1939
0.7600
0.3824
DVD
0.01973
0.2786
0.2119
0.7914
1.070
0.2412
0.7538
0.3718
TRELLIS
0.01328
0.3595
0.1641
0.8385
0.524
0.1899
0.7583
0.3799
Table 2: Image-conditioned generation on Toys4k ( ∗ voxels from voxelized meshes; ‡ evaluated on their own meshes). The best and second-best values of each column are in bold and underlined, respectively.
Figure 3: An example of image-conditioned generation on Toys4k. The top row shows the generated 643 voxels and the bottom row the meshes obtained from them. Each shape is aligned to its ground truth (GT) for display. More examples are in the appendix.
Method
1-NNA
COV ↑
MMD ↓
FID V↓
FID VD↓
KID VD↓
Four categories
LAS-Diffusion
0.782
0.365
2.08
63.3
82.5
16.7
OctFusion †
0.805
0.290
2.23
46.2
184.5
41.5
OctFusion (mesh)
0.779
0.310
2.23
11.1
43.0
4.99
OctGPT †
0.773
0.388
2.22
43.2
186.6
42.5
OctGPT (mesh)
0.736
0.412
2.25
10.1
61.0
10.4
Table 3: Unconditional generation on ShapeNet, averaged over the four categories, and over chair and airplane for the last two rows. For 1-NNA, closer to 0.5 is better. MMD is in units of 10−3 and KID VD in units of 10−2 ( † 323 occupancy output; “mesh” meshes voxelized at 643 ). The best and second-best values within each block are in bold and underlined, respectively.
Figure 4: Unconditional generation of chairs by our framework on ShapeNet.
Figure 5: Peak GPU memory and training throughput of our network and of the voxel stage of TRELLIS.2, with the same backbone, at each batch size on one A100 GPU with 80 GB. A cross marks the batch size at which TRELLIS.2 runs out of memory.
Variant
CD V↓
IoU ↑
CD shell↓
LPIPS N↓
Uni3D ↑
Single-level
0.02150
0.238
1.280
0.2566
0.3508
Two-level
0.01941
0.264
1.096
0.2428
0.3575
Ours
0.01506
0.334
0.744
0.2019
0.3820
Table 4: Ablation of two-level prediction and joint hierarchical modeling on Toys4k. The first two variants generate the hierarchy level by level. CD shell is in units of 10−3 . The best value of each column is in bold.
Sampler
CD V↓
IoU ↑
MDLM [ 21 ]
0.01162
0.4035
ReMDM [ 25 ]
0.01115
0.4085
ReMDM-conf [ 25 ]
0.01116
0.4139
MaskGIT [ 5 ]
0.01243
0.3987
Ours
0.01107
0.4158
Table 5: Ablation of the unmasking strategy on Toys4k. The best and second-best values of each column are in bold and underlined, respectively.
Figure 6: Image-conditioned generation at 2563 . Each input image is shown in the box at the top left of the generated 2563 voxels, which are colored by the texture stage of TRELLIS.2.
Ours
TRELLIS.2
Batch
Time (s)
Memory (GB)
Time (s)
Memory (GB)
8
2.02
6.1
2.30
5.7
16
1.77
7.9
2.24
11.2
32
1.80
11.3
2.21
22.2
64
1.67
17.8
2.20
44.2
128
1.62
28.9
OOM
Table 6: Time per shape and peak GPU memory of generating the voxels for image-conditioned generation on one A100 GPU with 80 GB, for our framework and the voxel stage of TRELLIS.2 (OOM, out of memory). The time includes image encoding, with 80 sampling steps for our framework and the default 12 for TRELLIS.2.
Chair
Airplane
Method
1-NNA
COV ↑
MMD ↓
FID V↓
FID VD↓
KID VD↓
1-NNA
COV ↑
MMD ↓
FID V↓
FID VD↓
KID VD↓
LAS-Diffusion
0.697
0.432
3.40
30.67
57.6
8.77
0.881
0.357
1.06
73.07
130.3
28.54
OctFusion †
0.689
0.406
3.59
22.49
106.8
15.63
0.902
0.261
1.17
51.04
305.2
67.86
OctFusion (mesh)
0.671
0.412
3.68
5.01
29.5
2.70
0.863
0.268
1.00
12.64
51.2
6.41
OctGPT †
0.715
0.397
3.74
25.38
139.6
26.30
0.872
0.365
1.09
47.83
312.3
68.88
OctGPT (mesh)
0.697
0.401
3.90
9.36
77.8
14.74
0.821
0.361
0.93
12.78
27.9
2.21
Table 7: Unconditional generation on each ShapeNet category (test splits). For 1-NNA, closer to 0.5 is better. MMD is in units of 10−3 and KID VD in units of 10−2 . Rows marked † use the 323 occupancy the method outputs, and rows marked “mesh” use the method’s meshes voxelized at 643 . XCube releases models only for chair and airplane. The best and second-best values of each column within each category are in bold and underlined, respectively.
Aug.
CD V↓
IoU ↑
CD shell↓
LPIPS N↓
CLIP N↑
Uni3D ↑
0.01280
0.3923
0.568
0.1858
0.7588
0.3844
✓
0.01271
0.3790
0.524
0.1797
0.7608
0.3847
Table 8: Training with and without augmentation, compared after the same number of training steps on the first 200 objects of Toys4k. CD shell is in units of 10−3 . The better value of each column is in bold.
Voxels
Meshes
Method
CD V↓
IoU ↑
LPIPS V↓
DINO V↑
CD shell ( ×10−3 ) ↓
LPIPS N↓
CLIP N↑
Uni3D ↑
TRELLIS
0.01328
0.3595
0.1641
0.8385
0.524
0.1899
0.7583
0.3799
Ours, about two days ( 80 K steps)
0.01221
0.3893
0.1485
0.8614
0.497
0.1737
0.7600
0.3851
TRELLIS.2
0.01250
0.4017
0.1524
0.8513
0.524
0.1666
0.7613
0.3865
Ours, about two and a half days ( 104 K steps)
0.01137
0.4035
0.1424
0.8692
0.421
0.1668
0.7612
0.3859
Table 9: Our framework after about two days and two and a half days of training on four H100 GPUs, compared with TRELLIS and TRELLIS.2 on Toys4k. Each of our rows is compared with the baseline above it. Steps are counted in batches of 256 shapes.
Figure 7: Image-conditioned generation on Toys4k. For each input image, the top row shows the generated 643 voxels and the bottom row the meshes obtained from them as in Section 5.1 . The ground truth (GT) shows the voxels and the mesh of the shape. Each shape is aligned to its ground truth for display.
Figure 8: Image-conditioned generation on Toys4k, continued from Figure 7 .
Figure 9: Image-conditioned generation on Toys4k, continued from Figure 8 .
We introduce Discrete Voxel Diffusion (DVD), a discrete diffusion framework to generate, assess, and edit sparse voxels for SLat (Structured LATent) based 3D generative pipelines. Although discrete diffusion has not generally displaced continuous diffusion in image-like generation, we show that it can be an effective first-stage prior for sparse voxel scaffolds. By treating voxel occupancy as a native discrete variable, DVD avoids continuous-to-discrete thresholding and provides a simple framework for voxel generation, uncertainty estimation, and editing. Beyond quality gains, DVD provides more interpretable generation dynamics through explicit categorical modeling. Furthermore, we leverage the predictive entropy as a robust uncertainty metric to identify ambiguous voxel regions and complicated samples, facilitating tasks such as data filtering and quality assessment. Finally, we propose a lightweight fine-tuning strategy using block-structured perturbation patterns. This approach empowers the model to inpaint and edit voxels within a single sampling round, requiring negligible auxiliary computation and no additional model evaluations. Code is available at https://github.com/TeCai/DVD.
Zhengrui Xiang, Jiaqi Wu, Fupeng Sun +2
Imperial College London · Math Magic, Hitem3D · Math Magic
We propose Perceptual Flow Matching (PFM), a simple yet effective framework for few-step generation in flow-matching models. Rather than performing velocity regression in the conventional VAE latent space, PFM supervises flow matching in a perceptual feature space using pretrained perceptual models. This simple change substantially improves the few-step generation capability of flow-matching models, reducing the number of sampling steps from 35-50 to 4-8 while preserving generation quality. Unlike existing acceleration and distillation approaches, PFM requires neither teacher models nor auxiliary score networks and can be integrated into standard flow-matching training pipelines with minimal modifications. Extensive experiments on image generation, video generation, and image editing tasks demonstrate that PFM consistently produces high-quality results while producing fewer artifacts than existing distillation-based methods. We further show that perceptual supervision shifts the regression minimizer from mean-seeking to mode-seeking, biasing predictions toward on-manifold modes that remain accurate under coarse few-step integration. Our results reveal that standard flow-matching training can naturally yield high-quality few-step generators when supervised in an appropriate representation space. We hope this insight inspires future research into representation-aware objectives for efficient generative modeling.
Chuyang Zhao, Yifei Song, Hongfa Wang +7
1Joy Future Academy · 2Fudan University · 3Tsinghua University +1
In this paper, we introduce LATO, a novel topology-preserving latent representation that enables scalable, flow matching-based synthesis of explicit 3D meshes. LATO represents a mesh as a Vertex Displacement Field (VDF) anchored on surface, incorporating a sparse voxel Variational Autoencoder (VAE) to compress this explicit signal into a structured, topology-aware voxel latent. To decapsulate the mesh, the VAE decoder progressively subdivides and prunes latent voxels to instantiate precise vertex locations. In the end, a dedicated connection head queries the voxel latent to predict edge connectivity between vertex pairs directly, allowing mesh topology to be recovered without isosurface extraction or heuristic meshing. For generative modeling, LATO adopts a two-stage flow matching process, first synthesizing the structure voxels and subsequently refining the voxel-wise topology features. Compared to prior isosurface/triangle-based diffusion models and autoregressive generation approaches, LATO generates meshes with complex geometry, well-formed topology while being highly efficient in inference.
Tianhao Zhao, Youjia Zhang, Hang Long +7
Huazhong University of Science and Technology · Peking University · Independent Researcher +1