cs.CVNov 19, 2026

Direct3D-V3: Latent-Free Diffusion Transformers in the Native Voxel Domain

Authors: Shuang Wu, Jiachen Qian, Youtian Lin, Feihu Zhang, Philip Torr, Yao Yao, Xun Cao

Organizations: Nanjing University, Nanjing, China · DreamTech, Shenzhen, China · The University of Hong Kong, Hong Kong, Hong Kong · Nanjing University, Suzhou, China · 

Abstract

Native 3D generation has largely been built around latent diffusion, with recent advances concentrating on stronger variational autoencoders (VAE) and increasingly elaborate latent representations, from vecset and triplanes to structured sparse voxels. Despite their architectural differences, these methods all share the same premise: geometry is first compressed into a latent space and only then modeled by the generator. We take a different route and propose Direct3D-V3, a latent-free diffusion transformer (DiT) framework that discards the VAE altogether and operates in the native voxel domain, directly modeling raw SDF voxel grids during both training and sampling. To make this direct voxel formulation geometrically reliable, we introduce an implicit manifold regularization over neighboring voxels to suppress non-watertight surfaces, and convert predicted SDF grids into meshes via Marching Cubes so that rendered normal losses can supervise the model directly from its predicted SDF. To improve alignment between the conditioning image and generated geometry, Direct3D-V3 also predicts the camera parameters of the conditioning image as an auxiliary task, encouraging the DiT to learn the mapping from image observations to 3D shape. This latent-free design removes the reconstruction bottleneck, brings training under an objective in the native voxel domain, and yields meshes with substantially richer local detail. The predicted camera and native SDF voxel output further make Direct3D-V3 extensible to generative reconstruction, where diffusion is interleaved with normal-guided refinement to improve alignment with the observed view while the generative prior completes unseen regions. Project page: https://dreamtechai.github.io/Direct3D-V3.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. DVD: Discrete Voxel Diffusion for 3D Generation and Editing

    May 8, 2026Zhengrui Xiang, Jiaqi Wu, Fupeng Sun +23D Shape Editing3D Shape Generation

  2. Channel Match: Calibrating Shape Latents for Few-Step 3D Generation

    Oct 11, 2026Di Yang, Zhihao Li, Yanhai Xiong +13D Texture Generation3D Asset Generation

  3. LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtents

    Mar 6, 2026Tianhao Zhao, Youjia Zhang, Hang Long +73D Mesh Generation3D Representation Learning