SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions

Jie Feng; Jiawei Shen; Junjia Huang; Junpeng Zhang; Mingtao Feng; Weisheng Dong; Guanbin Li

cs.CVApr 2, 2026

SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions

Authors: Jie Feng, Jiawei Shen, Junjia Huang, Junpeng Zhang, Mingtao Feng, Weisheng Dong, Guanbin Li

Organizations: Xidian University China · Sun Yat-sen University China

Abstract

3D indoor scene generation conditioned on short textual descriptions provides a promising avenue for interactive 3D environment construction without the need for labor-intensive layout specification. Despite recent progress in text-conditioned 3D scene generation, existing works suffer from poor physical plausibility and insufficient detail richness in such semantic condensation cases, largely due to their reliance on explicit semantic cues about compositional objects and their spatial relationships. This limitation highlights the need for enhanced 3D reasoning capabilities, particularly in terms of prior integration and spatial anchoring. Motivated by this, we propose SDesc3D, a short-text conditioned 3D indoor scene generation framework, that leverages multi-view structural priors and regional functionality implications to enable 3D layout reasoning under sparse textual guidance. Specifically, we introduce a Multi-view scene prior augmentation that enriches underspecified textual inputs with aggregated multi-view structural knowledge, shifting from inaccessible semantic relation cues to multi-view relational prior aggregation. Building on this, we design a Functionality-aware layout grounding, employing regional functionality grounding for implicit spatial anchors and conducting hierarchical layout reasoning to enhance scene organization and semantic plausibility. Furthermore, an Iterative reflection-rectification scheme is employed for progressive structural plausibility refinement via self-rectification. Extensive experiments show that our method outperforms existing approaches on short-text conditioned 3D indoor scene generation. Code will be publicly available.

Figures & tables

Figure 1. Short descriptions condense semantics, making 3D indoor scene generation challenging in terms of physical plausibility and detail preservation. To address this, SDesc3D leverages object coherence priors and functionality-based spatial anchoring, achieving superior performance under such a challenging condition.

Figure 2. Overview of our SDesc3D Framework. Given a short user descriptions, SDesc3D first performs Multi-view Scene Prior Augmentation to retrieve scene priors for sparse semantic completion. Next, Functionality-aware Layout Grounding leverages regional functionality implications to reason about a hierarchical layout in a coarse-fine manner. Finally, Iterative reflection-rectification is adopted to iteratively suppress residual physical and structural errors. Overview of Our Framework.

Method	Collision (%) $\downarrow$	OOB (%) $\downarrow$	OP $\uparrow$		AO $\uparrow$		DR $\uparrow$		ZO $\uparrow$		CR $\uparrow$		FC $\uparrow$		AI-Avg $\uparrow$
Method	Collision (%) $\downarrow$	OOB (%) $\downarrow$	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4
HSM (3DV 2026) ( Pun et al., 2026 )	9.36%	8.67%	9.31	8.83	9.34	8.96	7.83	7.47	8.32	8.61	7.73	7.48	8.51	7.96	8.51	8.22
Reason3D (AAAI 2026) ( Berdoz et al., 2026 )	18.62%	12.55%	8.87	8.32	9.32	8.99	7.87	7.94	8.81	8.63	7.69	7.43	8.07	7.86	8.44	8.20
Ours	5.36%	7.70%	9.15	8.89	9.26	8.71	8.21	8.24	9.67	9.04	9.02	8.56	9.03	9.01	9.06	8.74

Table 1. Quantitative comparison under the short-text 3D indoor scene generation setting. Compared with HSM and Reason3D, our method achieves improved overall performance.

Figure 3. Qualitive comparison on the scenes generated on five different short descriptions. Our method achieves better overall scene quality than the compared approaches in terms of physical plausibility and detail richness.

Method	OP $\uparrow$	AO $\uparrow$	DR $\uparrow$	ZO $\uparrow$	CR $\uparrow$	FC $\uparrow$	User-Avg $\uparrow$
HSM (3DV 2026) ( Pun et al., 2026 )	7.26	7.72	7.46	7.40	7.52	7.49	7.48
Reason3D (AAAI 2026) ( Berdoz et al., 2026 )	7.15	7.40	7.36	7.00	6.94	7.44	7.22
Ours	8.64	8.85	8.69	8.74	8.77	8.56	8.71

Table 2. User study of different methods under the short-text 3D indoor scene generation setting. Our method achieves the best results across all evaluated metrics.

MSPA	FLG	IRR	Collision (%) $\downarrow$	OOB (%) $\downarrow$	DR $\uparrow$		OP $\uparrow$		ZO $\uparrow$		CR $\uparrow$		FC $\uparrow$		AI-Avg $\uparrow$
MSPA	FLG	IRR	Collision (%) $\downarrow$	OOB (%) $\downarrow$	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4	Gemini 3-flash	GPT 5.4
			38.66%	26.72%	7.43	7.26	7.41	7.02	6.13	6.05	6.05	5.88	4.78	4.56	6.36	6.15
$\checkmark$			19.56%	13.67%	8.48	8.30	7.85	7.42	8.12	7.76	8.14	7.68	8.31	7.53	8.18	7.74
$\checkmark$	$\checkmark$		14.28%	8.33%	9.05	8.89	8.04	7.65	9.16	8.71	9.44	8.36	9.52	8.62	9.04	8.45
$\checkmark$	$\checkmark$	$\checkmark$	5.44%	6.73%	9.21	9.03	9.25	8.98	9.54	9.21	9.53	8.88	9.88	8.76	9.48	8.97

Table 3. Ablation study of our SDesc3D framework under the short-text 3D indoor scene generation setting. MSPA, FLG, and IRR are added progressively to analyze their individual and joint contributions. Each added component consistently improves the results, with the complete model performing best across nearly all metrics.

Metric	User-AI Score Pair 1	User-AI Score Pair 2
Pearson $r$	0.81	0.76
Spearman $\rho$	0.73	0.69

Table 4. Correlation Coefficient between User and AI Scores. The results suggest a positive correlation between user judgments and AI-based evaluations.

Model	GPT5.4	Gemini3-flash
GPT5.4	8.72	8.89
Gemini3-flash	8.74	9.06
Qwen3	8.67	8.94
Claude-sonnet-4-6	8.70	9.03

Table 5. Model comparison results. The results indicate that our SDesc3D framework exhibits strong robustness across different foundation models.

Figure 4. Qualitative comparison of HSM, Reason3D, and our method under the long-text setting.

Figure 5. Examples of scene editing results of object addition, deletion and relocation by SDesc3D. Without additional treatments, SDesc3D handles these editing actions with plausible results. Editing Task.

Indoor Scene Generation

Indoor scene generation creates and edits furnished spaces from text, images, floor plans, or functional requirements.

See all 65 papers

Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation
Mar 14, 2026Stefan Ainetter, Thomas Deixelberger, Edoardo A. Dominici +3Diffusion Model Guidance 3D Scene Generation
Huawei Technologies, Austria · Graz University of Technology, Austria · Huawei Technologies, Switzerland
ScenePilot: Grow-and-Repair Policy for Text-Driven 3D Indoor Scene Generation
Aug 31, 2026Jiawei Zhang, Hongsong Wang, Pan Zhou3D Scene Editing Functional 3D Scene Generation
Southeast University · Singapore Management University
ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoning
Jul 15, 2026Yuan Xiao, Can Wang, Xiangyu Kong +13D Scene Editing 3D Scene Generation
Department of Computer Science, City University of Hong Kong, Hong Kong · School of Computing and Data Science, University of Hong Kong · School of Computer Science, Beijing Information Science and Technology University

Mar 14, 2026J/K move · Enter open · S save

Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation

Stefan Ainetter, Thomas Deixelberger, Edoardo A. Dominici, Philipp Drescher+2

Huawei Technologies, Austria · Graz University of Technology, Austria · Huawei Technologies, Switzerland

Diffusion Model Guidance 3D Scene Generation 3D Scene Layout Generation Text-to-3D Generation Indoor Scene Generation

Open paper AskPDF arXiv

We present GuidedSceneGen, a text-to-3D generation framework that produces metrically accurate, globally consistent, and semantically interpretable indoor scenes. Unlike prior text-driven methods that often suffer from geometric drift or scale ambiguity, our approach maintains an absolute world coordinate frame throughout the entire generation process. Starting from a textual scene description, we predict a global 3D layout encoding both semantic and geometric structure, which serves as a guiding proxy for downstream stages. A semantics- and depth-conditioned panoramic diffusion model then synthesizes 360° imagery aligned with the global layout, substantially improving spatial coherence. To explore unobserved regions, we employ a video diffusion model guided by optimized camera trajectories that balances coverage and collision avoidance, achieving up to 10x faster sampling compared to exhaustive path exploration. The generated views are fused using 3D Gaussian Splatting, yielding a consistent and fully navigable 3D scene in absolute scale. GuidedSceneGen enables accurate transfer of object poses and semantic labels from layout to reconstruction, and supports progressive scene expansion without re-alignment. Quantitative results and a user study demonstrate greater 3D consistency and layout plausibility compared to recent panoramic text-to-3D baselines.

Abstract

Indoor Scene Generation

Explore similar work

Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation

ScenePilot: Grow-and-Repair Policy for Text-Driven 3D Indoor Scene Generation

ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoning