cs.CVMar 15, 2025

DecompDreamer: A Composition-Aware Curriculum for Structured 3D Asset Generation

Authors: Utkarsh Nath, Rajeev Goel, Rahul Khurana, Kyle Min, Mark Ollila, Pavan Turaga, Varun Jampani, Tejaswi Gowda

Organizations: Arizona State University · Intel Labs · Stability AI

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Current text-to-3D methods excel at generating single objects but falter on compositional prompts. We argue this failure is fundamental to their optimization schedules, as simultaneous or iterative heuristics predictably collapse under a combinatorial explosion of conflicting gradients, leading to entangled geometry or catastrophic divergence. In this paper, we reframe the core challenge of compositional generation as one of optimization scheduling. We introduce DecompDreamer, a framework built on a novel staged optimization strategy that functions as an implicit curriculum. Our method first establishes a coherent structural scaffold by prioritizing inter-object relationships before shifting to the high-fidelity refinement of individual components. This temporal decoupling of competing objectives provides a robust solution to gradient conflict. Qualitative and quantitative evaluations on diverse compositional prompts demonstrate that DecompDreamer outperforms state-of-the-art methods in fidelity, disentanglement, and spatial coherence.

Figures & tables

Figure 1: Illustration of high-quality compositional 3D assets generated by DecompDreamer for complex text prompts. Existing methods often miss objects or distort spatial relationships, while DecompDreamer accurately captures geometry and preserves inter-object layout.
Figure 2: A visual taxonomy of optimization heuristics. (a) Holistic uses a single global loss. (b) Simultaneous applies all losses concurrently. (c) Iterative applies losses in a sequential loop. (d) Our staged curriculum temporally decouples relational and object losses
Figure 3: Overview of the DecompDreamer pipeline. Given a text prompt, a VLM generates a scene graph to guide a coarse initialization. The core of our method is a composition-aware optimization curriculum that first models joint relationships to build a coherent structure, then refines individual objects to produce high-fidelity, disentangled 3D assets.
Figure 4: Qualitative comparison between the proposed DecompDreamer and state-of-the-art text-to-3D generators. More results can be found in Section A.12 of the appendix.
Figure 5: Empirical validation of optimization heuristics. The plots show the average object, scene, and edge losses for a complex 4-object text prompt ("A knight in shining armor…"). (a) GALA3D converges suboptimally, (b) GraphDreamer diverges, while (c) our staged approach (DecompDreamer) converges stably.
Figure 6: Qualitative comparison of object disentanglement in GALA3D, GraphDreamer, and DecompDreamer. Each method displays the full 3D object (left) and its components (right).
Figure 7: Results of an ablation study on DecompDreamer with the text prompt: “A knight in full shining armor galloping on a brown horse, wearing a blue hat, and holding a sword". Additional ablation studies can be found in Figure 11 of the appendix.
Figure 8: Qualitative comparison of DecompDreamer with different coarse intializers: (a) TripoSR initialization, (b) Point-E initialization.
Figure 9
Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Additional empirical comparison of optimization heuristics across three complex text prompts (rows). We plot individual loss components (log scale) over iterations. The columns illustrate that GALA3D (Heuristic 2) converges suboptimally with high remaining loss, GraphDreamer (Heuristic 3) exhibits significant instability and divergence, while our staged approach, DecompDreamer (Heuristic 4), achieves stable convergence to lower loss minima.
NotationDescription
tgt^{g}Compositional text prompt
VLMVision-Language Model
TmaxT_{\text{max}}Total number of optimization iterations
I\mathcal{I}Generated image representation
GGScene graph extracted from VLM
NNNumber of objects in the scene
Appendix
Table 4: Summary of Notations Used in the Algorithm
Figure 11: Results of an ablation study on DecompDreamer under different training configurations.
Figure 12: Results of an ablation study on DecompDreamer with (right) and without (left) the azimuth adjustment for object-view dependent text prompts.
Figure 13: Results of an ablation study on DecompDreamer with (right) and without (left) the negative prompts for the objects
Figure 14: Results of an ablation study on DecompDreamer with (right) and without (left) spatial error correction.
Figure 15: More qualitative comparison of object disentanglement in GALA3D, GraphDreamer, and DecompDreamer. Each method displays the full 3D object (left) and its individual components (right).
Figure 16: Qualitative comparison on the prompt: ’A lego man riding a horse, the horse is wearing a blue hat.’ Layout-your-3D merges the hat with the horse’s head and entangles the rider within the body. In contrast, DecompDreamer successfully resolves these physical interactions.
Figure 17: Qualitative comparison with between feed-forward 3D object generation methods such as PartCrafter, Step1X-3D, and Hunyaun3d-2.0, against DecompDreamer.
Figure 18: Example usage of ChatGPT in DecompDreamer for generating compositional 3D objects.
Prompt #Prompt Text
1A well-furnished bedroom with a Baroque-style bed with a red pillow on it, two wooden nightstands with pots on top of them, a square wooden table holding a decorative table lamp, a wooden wardrobe and a kid infront of the bed and a pet dog standing next to the bed between nightstand and wardrobe.
2A well-furnished bedroom with a Baroque-style bed with a red pillow on it, two wooden nightstands, a square wooden table holding a decorative table lamp, a wooden wardrobe.
3Three macaw parrots sharing a milkshake with three straws
4A bedroom with a Baroque bed, two wooden nightstands, a square wooden table with a potting on it, and a wooden wardrobe
5A knight in shining armor gallops on a brown horse, wearing a blue hat, holding a sword in his right hand. A princess sits beside him on the same horse, wearing a flowing gown and a jeweled crown
6A bedroom with a bed, two wooden nightstands, a square wooden table with a table lamp on it, and a wooden wardrobe
Appendix
Table 7: Full Evaluation Benchmark Prompts
Figure 19: Qualitative comparison of the "Wizard" prompt utilizing proprietary GPT-4o (left) versus open-source Qwen-VL (right) initialization priors. Although the initial spatial estimations exhibit minor numerical deviations (as quantified in Table 8 ), ours spatial correction module successfully absorbs this variance. The final optimized geometry maintains strict relational fidelity, confirming the framework’s robustness without reliance on proprietary VLM supervision.
ObjectΔ\Delta Spatial CoordsΔ\Delta AnglesΔ\Delta ScaleΔ\Delta Azimuth
Prompt 1: A knight in shining armor gallops on a brown horse…
1. Knight5%0%8.3%0%
2. Horse5%0%11.1%0%
3. Cap5.4%0%0%0%
4. Sword5.12%11%10.0%0%
5. Crown5.1%0%0%0%
Appendix
Table 8: Relative Deviation of Initialization Priors (GPT-4o vs. Qwen-VL). Values show the overall vector percentage change (Euclidean deviation) from GPT-4o to Qwen-VL.
Figure 20: Examples of failure cases. (a) Difficulty in decomposing a single entity into fine-grained subcomponents—such as segmenting human body parts. (b) A case where DecompDreamer fails to model complex actions, such as a bag with its zipper open .
Figure 21: Qualitative comparison of GALA3D and GraphDreamer against DecompDreamer under the same Point-E initialization. Despite using the same low-fidelity coarse initializer, DecompDreamer produces more coherent object geometry and better preserves inter-object relationships in complex compositional prompts, while requiring comparable or lower runtime.
Figure 22: More qualitative comparison of GALA3D, GraphDreamer, MVDream, LucidDreamer, Magic3D and HiFA against DecompDreamer with SD2.1 and SD3.5 as backbones.
Figure 23: More qualitative comparison between the proposed DecompDreamer and state-of-the-art generators, including GALA3D Zhou et al. (2024) , GraphDreamer Gao et al. (2024) , Trellis Xiang et al. (2025) , MVDream Shi et al. (2024) , LucidDreamer Liang et al. (2024) , Magic3D Lin et al. (2023) and HiFA Zhu et al. (2024) .
Figure 24: More qualitative comparison between the proposed DecompDreamer and state-of-the-art generators, including GALA3D Zhou et al. (2024) , GraphDreamer Gao et al. (2024) , Trellis Xiang et al. (2025) , MVDream Shi et al. (2024) , LucidDreamer Liang et al. (2024) , Magic3D Lin et al. (2023) and HiFA Zhu et al. (2024) .
Figure 25: More qualitative comparison between the proposed DecompDreamer and state-of-the-art generators, including GALA3D Zhou et al. (2024) , GraphDreamer Gao et al. (2024) , Trellis Xiang et al. (2025) , MVDream Shi et al. (2024) , LucidDreamer Liang et al. (2024) , Magic3D Lin et al. (2023) and HiFA Zhu et al. (2024) .
More in this topic

3D Asset Generation

3D asset generation creates and edits objects, characters, materials, and scenes from text, images, video, or geometric constraints.

See all 167 papers

Explore similar work

CardsList
Mar 19, 2026

DreamPartGen: Semantically Grounded Part-Level 3D Generation via Collaborative Latent Denoising

Understanding and generating 3D objects as compositions of meaningful parts is fundamental to human perception and reasoning. However, most text-to-3D methods overlook the semantic and functional structure of parts. While recent part-aware approaches introduce decomposition, they remain largely geometry-focused, lacking semantic grounding and failing to model how parts align with textual descriptions or their inter-part relations. We propose DreamPartGen, a framework for semantically grounded, part-aware text-to-3D generation. DreamPartGen introduces Duplex Part Latents (DPLs) that jointly model each part's geometry and appearance, and Relational Semantic Latents (RSLs) that capture inter-part dependencies derived from language. A synchronized co-denoising process enforces mutual geometric and semantic consistency, enabling coherent, interpretable, and text-aligned 3D synthesis. Across multiple benchmarks, DreamPartGen delivers state-of-the-art performance in geometric fidelity and text-shape alignment.

Tianjiao Yu, Xinzhuo Li, Muntasir Wahed +4

University of Illinois Urbana-Champaign

Part-Level 3D Generation3D Shape GenerationText-to-3D GenerationDiffusion Models
AskPDF
Jun 23, 2026

Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generation

Recent breakthroughs in 3D generation have advanced notably with the development of text-to-image diffusion model. However, existing methods remain two practical challenges: (1) They primarily generate single 3D object, but struggle to generate multi-object compositional 3D assets due to the lack of the modeling for Gaussian primitives in reasonable interactions. (2) They often suffer from cross-view inconsistency during 3D optimization, as Score Distillation Sampling inherently performs on each single view, inevitably resulting in cross-view hallucinations. To solve above issues, we propose I2C-3D, a novel optimization-based method to generate multi-view consistent compositional 3D assets with reasonable interactions. Specifically, we propose an Inclusive Interactive Collisions strategy to guide Gaussian primitives appearing in reasonable interaction regions naturally, thereby ensuring objects in the compositional scene interact in a physically plausible and visually coherent way. Additionally, to enhance multi-view consistency, Multi-View Adaptive Score Distillation Sampling is devised to distill multi-view consistency prior and layout prior from pre-trained diffusion model by modulating attention map of instance token and spatial token across viewpoints. Benefiting from above elaborate designs, I2C-3D not only generates high-fidelity multi-view consistent compositional 3D assets but also supports 3D editing flexibly, facilitating complex scene generation. Extensive experiments demonstrate our I2C-3D outperforms existing methods in generation quality and multi-view consistency.

Chang Liu, Mingwen Shao, Xiang Lv +5

School of Computer Science and Technology, China University of Petroleum (East China), Qingdao, 266580, China · Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology, Shenzhen, 518107, China · College of Computer Science, Northwest University, Xi’an, 710127, China

Multi-View Consistency3D Gaussian Splatting3D Asset Generation3D Scene GenerationScore Distillation Sampling
AskPDF
Mar 17, 2026

Interact3D: Compositional 3D Generation of Interactive Objects

Recent breakthroughs in 3D generation have enabled the synthesis of high-fidelity individual assets. However, generating 3D compositional objects from single images--particularly under occlusions--remains challenging. Existing methods often degrade geometric details in hidden regions and fail to preserve the underlying object-object spatial relationships (OOR). We present a novel framework Interact3D designed to generate physically plausible interacting 3D compositional objects. Our approach first leverages advanced generative priors to curate high-quality individual assets with a unified 3D guidance scene. To physically compose these assets, we then introduce a robust two-stage composition pipeline. Based on the 3D guidance scene, the primary object is anchored through precise global-to-local geometric alignment (registration), while subsequent geometries are integrated using a differentiable Signed Distance Field (SDF)-based optimization that explicitly penalizes geometry intersections. To reduce challenging collisions, we further deploy a closed-loop, agentic refinement strategy. A Vision-Language Model (VLM) autonomously analyzes multi-view renderings of the composed scene, formulates targeted corrective prompts, and guides an image editing module to iteratively self-correct the generation pipeline. Extensive experiments demonstrate that Interact3D successfully produces promising collsion-aware compositions with improved geometric fidelity and consistent spatial relationships.

Hui Shan, Keyang Luo, Ming Li +4

Zhejiang University, China · Shanghai Innovation Institute, China · Westlake University, China +4

3D Asset Generation3D Scene Generation3D Shape Generation
AskPDF