3D generation, editing, and completion are typically treated as regenerating the complete 3D asset under task-specific constraints, even when only a local region needs modification. This leads to task isolation, inconsistency in content that should remain unchanged, and dependence on task-specific paired data. We address these issues with a key observation: the three tasks all synthesize a designated region while keeping the rest intact---the full region for generation, the target region for editing, and the missing region for completion. We term this paradigm Selective Modeling and build SeMo3D upon it; it naturally unifies the three tasks, preserves unchanged content by construction, and enables training without paired editing or completion data, since any 3D asset can be partitioned into a selected region and its context. For training, we propose region-selective denoising, which randomly selects regions of varying sizes on standard 3D assets and applies noise and loss only to them, jointly learning all three tasks. For architecture, we propose Omni3DiT, a multi-condition 3D diffusion transformer that distinguishes regions to create from those to retain and fuses image, text, and point guidance for fine-grained control. Experiments across three benchmarks show SeMo3D surpasses state-of-the-art methods on all tasks, delivering multimodal controllability in generation and strong consistency and accuracy in editing and completion. SeMo3D further serves as a unified modeling tool for agents, enabling efficient 3D creation through natural-language interaction. Project page: https://xiaokunsun.github.io/SeMo3D.github.io/.
Figures & tables
Figure 2: (a) Existing holistic modeling regenerates the complete asset even for local modification, requiring per-task models and paired editing/completion data, while regions that should remain unchanged are often altered (black boxes). (b) Our selective modeling synthesizes only the designated region, unifying the three tasks, preserving the rest by construction, and needing no task-specific paired data.
Figure 3: Overview of SeMo3D. (a) Region-selective denoising. Training applies noise and loss only to a randomly selected set S while keeping the context set C clean; as S ranges from a local box to the full region, one procedure covers editing, completion, and generation. Inference denoises S from pure noise with a task-defined partition while reusing clean C at every step. (b) Omni3DiT jointly processes clean context and noisy selected tokens, fusing image guidance with optional text and point conditions for fine-grained controllable synthesis.
Figure 4: Agentic 3D modeling with SeMo3D. Agent prepares input images, determines edit regions, and invokes the appropriate SeMo3D mode in response to requests. Left: generation then editing. Right: completion then editing. Refer to Supp. Mat. for video results.
Figure 5: Qualitative comparison of 3D generation on Toys4K [ 63 ] . Only top-performing baselines are shown for brevity. HY3D-Omni [ 29 ] textures are from HY3D-2.1 [ 28 ] . With three complementary modalities, SeMo3D achieves more controllable and faithful generation.
Method
Cond.
LPIPS ↓
FID ↓
KID ( × 100) ↓
DINO ↑
CD ↓
Trellis [ 78 ]
I
0.133
6.65
0.121
0.888
0.0413
Trellis [ 78 ]
T
0.183
15.48
0.202
0.795
0.0736
ShapeLLM-Omni [ 90 ]
I
0.167
27.15
1.167
0.751
0.0820
ShapeLLM-Omni [ 90 ]
T
0.185
27.30
0.836
0.760
0.0666
HY3D-2.1 [ 28 ]
I
0.143
5.68
0.062
0.869
0.0638
Trellis2 [ 77 ]
I
0.111
5.05
0.057
0.908
0.0330
Table 1: Quantitative results on 3D generation. Best and second-best in bold and underlined . Cond.: I = Image, T = Text, P = Point.
Figure 6: Qualitative comparison of 3D editing on Edit3D-Bench [ 34 ] . Translucent volumes mark edit regions; points are omitted as Edit3D-Bench lacks them. Without paired editing data, SeMo3D precisely modifies the target region while preserving the rest.
Figure 7: Qualitative comparison of 3D completion on Omni-Comp [ 82 ] . Without any paired completion data, SeMo3D completes missing regions with coherent geometry that blends seamlessly with the preserved structure. † With additional sparse point guidance.
Method
Cond.
Consistency
Quality
Accuracy
CD ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
FVD ↓
DINO ↑
CLIP ↑
VoxHammer [ 34 ]
I
0.0111
31.21
0.973
0.038
26.35
586.1
0.869
0.281
VoxHammer [ 34 ]
T
0.0092
31.55
0.976
0.033
23.69
339.2
0.833
0.271
Nano3D [ 91 ]
I
0.0101
31.42
0.975
0.035
24.12
388.6
0.861
0.280
3DEditFormer [ 76 ]
I
0.0213
27.44
0.948
0.061
28.03
572.4
0.858
0.278
PartFlow [ 72 ]
I
0.0153
30.13
0.967
0.044
25.33
437.7
0.858
0.277
Table 2: Quantitative results on 3D editing. Cond. as in Tab. 1 . Edit3D-Bench [ 34 ] does not provide point conditions.
Method
Cond.
CD ↓
EMD ↓
F-Score ↑
LaS-Comp [ 82 ]
I
0.0226
0.0375
0.359
LaS-Comp [ 82 ]
T
0.0313
0.0489
0.335
HY3D-Omni [ 29 ]
I, P
0.0360
0.0559
0.347
HY3D-Omni † [ 29 ]
I, P
0.0270
0.0464
0.412
SeMo3D (Ours)
I, T
0.0208
0.0346
0.468
SeMo3D † (Ours)
I, T, P
0.0133
0.0223
0.541
Table 3: Quantitative results on 3D completion. Cond. as in Tab. 1 . † With additional sparse point guidance.
Configuration
Cond.
Generation
Editing
Completion
LPIPS ↓
FID ↓
KID ( × 100) ↓
DINO ↑
CD ↓
FID ↓
FVD ↓
DINO ↑
CLIP ↑
CD ↓
EMD ↓
F-Score ↑
(i)
SeMo3D (Ours)
I
0.113
5.49
0.074
0.904
0.0334
22.53
118.2
0.873
0.284
0.0207
0.0346
0.471
(ii)
SeMo3D (Ours)
I, T
0.112
5.34
0.069
0.911
0.0326
22.46
113.6
0.875
0.286
0.0208
0.0346
0.468
(iii)
SeMo3D (Ours)
I, P
0.082
5.06
0.065
0.926
0.0116
–
–
–
–
0.0134
0.0222
0.542
(iv)
SeMo3D (Ours)
I, T, P
0.079
4.99
0.059
0.926
0.0118
–
–
–
–
0.0133
0.0223
0.541
(v)
w/o Role Emb.
I, T, P
0.078
5.01
0.057
0.925
0.0118
22.52
114.2
0.866
0.278
0.0156
0.0249
0.528
Table 4: Quantitative ablation. (i)–(iv): condition combinations; (v)–(vii): full-model component ablations. Editing consistency metrics are omitted given identical context tokens across variants. Edit3D-Bench lacks points, so (iii)–(iv) omit editing and (v)–(vii) edit with I, T.
Figure 8: (a) Qualitative ablation. (b) Multimodal control: with a fixed image, varying text and points yields diverse yet faithful results.
Figure 9: Agentic 3D modeling with GPT-6 Astra [ 48 ] and Codex: (a) direct Blender coding [ 17 ] ; (b) SeMo3D integration, yielding higher modeling fidelity, faster execution, and lower agent cost.
Figure 10: Failure cases: coarse fronds and missing fruits.
SS
SLat (shape)
SLat (texture)
Grid resolution
163
323
323
Width / blocks / heads
1536 / 30 / 12
Conditions
I, T, P
I, T
I, T
Training steps
200K
100K
100K
Batch size
32
32
32
Box side length (voxels)
4–16
8–32
8–32
Table 5: Per-stage training settings.
Figure 11: ULIP and Uni3D scores disagree with visual quality: Trellis2 is closer to the ground truth than Trellis-I but scores lower.
Figure 12: Robustness to mask size. SeMo3D yields consistent edits as the edit region (translucent) grows by up to 40%.
Figure 13: Robustness to the number of condition points. With the image and text fixed, SeMo3D preserves the overall geometry across 16, 32, and 48 points, despite being trained with at most 32.
Figure 14: Robustness to the spatial distribution of condition points. With the image and text fixed, SeMo3D produces consistent overall structure across three point distributions, with variations in local details.
Figure 15: Additional agentic interactive 3D modeling: generation followed by two edits.
Figure 16: Additional agentic interactive 3D modeling: generation followed by two edits.
Figure 17: Additional agentic interactive 3D modeling: completion followed by two edits.
Figure 18: Additional agentic interactive 3D modeling: completion followed by two edits.
Figure 19: Additional qualitative comparison of 3D generation on Toys4K [ 63 ] .
Figure 20: Additional qualitative comparison of 3D editing on Edit3D-Bench [ 34 ] . Translucent volumes mark the edit regions.
Figure 21: Additional qualitative comparison of 3D completion on Omni-Comp [ 82 ] .
Recent advances in image generation have demonstrated the potential of unified multimodal models that integrate understanding, generation, and editing. However, unified 3D modeling remains constrained by scarce multimodal data, particularly the lack of large-scale and geometrically consistent editing data. To address this limitation, we propose Hunyuan3D-Buffalo 1.0, a unified framework supporting 3D understanding, text-to-3D generation, instruction-guided 3D editing, and text-grounded part generation within a single architecture. To enable scalable training, we construct an 87M-scale 3D multimodal corpus, comprising 25M understanding samples, 50M text-to-3D pairs, and 12M editing pairs generated using Nano3D-v2. Architecturally, the framework combines Hunyuan3D-VLM for semantic, structural, and spatial understanding with Hunyuan3D DiT for high-fidelity 3D synthesis. The VLM provides multimodal semantic conditions for generation, while editing and part generation additionally condition the diffusion process on the source object representation to preserve its overall structure and unedited regions. Extensive experiments show that Hunyuan3D-Buffalo 1.0 achieves state-of-the-art or leading performance on text-to-3D generation and 3D editing benchmarks, while exhibiting strong understanding and part-generation capabilities. Our analysis further shows that both generation and understanding improve editing, demonstrating the effectiveness of unified 3D multimodal training. Project Page: https://tencent-hunyuan.github.io/Hunyuan3D-Buffalo1.0/
Recent multimodal large language models have achieved strong performance in unified text and image understanding and generation, yet extending such native capability to 3D remains challenging due to limited data. Compared to abundant 2D imagery, high-quality 3D assets are scarce, making 3D synthesis under-constrained. Existing methods often rely on indirect pipelines that edit in 2D and lift results into 3D via optimization, sacrificing geometric consistency. We present Omni123, a 3D-native foundation model that unifies text-to-2D and text-to-3D generation within a single autoregressive framework. Our key insight is that cross-modal consistency between images and 3D can serve as an implicit structural constraint. By representing text, images, and 3D as discrete tokens in a shared sequence space, the model leverages abundant 2D data as a geometric prior to improve 3D representations. We introduce an interleaved X-to-X training paradigm that coordinates diverse cross-modal tasks over heterogeneous paired datasets without requiring fully aligned text-image-3D triplets. By traversing semantic-visual-geometric cycles (e.g., text to image to 3D to image) within autoregressive sequences, the model jointly enforces semantic alignment, appearance fidelity, and multi-view geometric consistency. Experiments show that Omni123 significantly improves text-guided 3D generation and editing, demonstrating a scalable path toward multimodal 3D world models.
Recent 3D generative models produce high-quality geometry from a single image using large-scale priors and diffusion architectures. However, they assume complete visibility and single-view inputs, limiting applicability in multi-view, occluded, or editing scenarios. Although prior works address these challenges individually, they lack a unified framework for controllable 3D completion under diverse conditioning signals. We present Axolotl3D, a multi-modal and occlusion-aware 3D generation model that jointly conditions on images, visibility masks, camera parameters, and a partial point cloud. The point cloud serves as a geometric anchor promoting faithful shape completion, while camera parameters ensure consistent multi-view alignment in a shared 3D coordinate system. A unified training strategy synthesizes diverse conditioning regimes from large-scale 3D data, enabling robust cross-modal reasoning. Experiments on Toys4K and OmniObject3D demonstrate state-of-the-art performance under both clean and occluded settings, as well as strong results in real-world reconstruction and geometry-consistent editing.