SceneExpander: Text-Guided 3D Scene Expansion via Free-Form View Insertion

Zijian He; Renjie Liu; Yihao Wang; Weizhi Zhong; Huan Yuan; Kun Gai; Guangrun Wang; Guanbin Li

cs.CVMar 28, 2026

SceneExpander: Text-Guided 3D Scene Expansion via Free-Form View Insertion

Authors: Zijian He, Renjie Liu, Yihao Wang, Weizhi Zhong, Huan Yuan, Kun Gai, Guangrun Wang, Guanbin Li

Organizations: Sun Yat-sen University · KlingAI Research, Kuaishou Technology

Abstract

World building with 3D scene representations is increasingly important for content creation, simulation, and interactive experiences, yet real workflows are inherently iterative: creators repeatedly extend existing scenes under user control. Motivated by this gap, we study text-guided 3D scene expansion via free-form view insertion. Starting from a real scene captured by multi-view images, a user specifies a text expansion intent, which a generative model materializes as an inserted view extending the scene coverage. Unlike simple object editing or style transfer within a fixed scene, the inserted view may be 3D-misaligned with the original reconstruction, introducing geometric shifts, hallucinated content, or view-dependent artifacts that disrupt global multi-view consistency. To address this challenge, we propose SceneExpander, which applies test-time adaptation to a parametric feed-forward 3D reconstruction model with two complementary distillation signals: anchor distillation stabilizes the captured scene using geometric cues from the captured views, while inserted-view self-distillation retains insertion-supported predictions to accommodate the misaligned view. Experiments on ETH scenes and online data demonstrate improved expansion behavior and reconstruction quality under misalignment.

Figures & tables

Figure 2: Challenges in text-guided 3D scene expansion via free-form view insertion. (1) A 3D-misaligned inserted view can corrupt reconstruction quality in the captured region. (2) Newly introduced content must remain consistent under novel viewpoints. Red boxes highlight ghosting artifacts caused by conflicting 3D constraints from the inserted view.

Figure 3: Overview of SceneExpander. (1) A user provides a text expansion instruction, which an image generative model materializes as a free-form inserted view $I_{g}$ ; a feed-forward reconstructor $f_{\theta}$ predicts an initial 3D scene from captured views. (2) To accommodate the text-guided inserted view that may be 3D-misaligned, we adapt $f_{\theta}$ at test time using two distillation losses: anchor distillation on captured views and inserted-view self-distillation on $I_{g}$ . (3) The adapted model produces an expanded 3D scene consistent with the captured region while incorporating the inserted content.

Figure 4: Geometry-perturbation augmentation. We show four augmentation modes: identity (original), global affine warp, blockwise piecewise-affine warp (grid $3{\times}3$ ), and their combination (global+block), designed to mimic pose drift and local geometric inconsistencies.

Figure 5: Qualitative comparison of scene expansion under misaligned insertion. We show an indoor scene from the Online collection (left) and an outdoor scene from the ETH dataset (right). The first row presents the generated inserted view (purple box) together with a subset of captured multi-view images from the original scene (blue box). Each subsequent row shows renderings at novel camera poses for different methods, illustrating both fidelity to the captured region and insertion satisfaction. Please refer to the media supplement for video visual results.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	GPT-IS $\uparrow$	GPT-RQ $\uparrow$	H-IS $\uparrow$	H-RQ $\uparrow$
OminVGGT ( Peng et al. 2025 )	14.969	0.3647	0.370	3.16	2.66	2.26	2.40
TTT3R ( Chen et al. 2025b )	13.483	0.2747	0.457	2.94	2.38	1.72	1.71
WorldMirror ( Liu et al. 2025 )	19.295	0.614	0.357	3.50	2.61	4.04	4.07
3DGS ( Kerbl et al. 2023 )	14.046	0.3045	0.418	3.16	2.61	2.40	2.32
SceneExpander (Ours)	21.206	0.776	0.276	3.72	3.05	4.56	4.53

Table 1: Quantitative comparison. We report preservation on the captured region using image-space fidelity metrics (PSNR/SSIM/LPIPS) and expansion quality using LLM-based and human evaluation. GPT-IS/H-IS measure insertion satisfaction, and GPT-RQ/H-RQ measure overall rendering quality.

\uparrow

higher is better,

\downarrow

lower is better.

Variant	PSNR	SSIM	LPIPS	GPT-IS	GPT-RQ
Baseline	19.295	0.614	0.357	3.50	2.61
+ Anchor distill.	20.763	0.714	0.329	3.52	2.96
+ Subset samp. & aug.	20.892	0.720	0.322	3.56	3.24
+ Self-distill. ( $p{=}0.5$ )	21.124	0.757	0.294	3.70	3.08
+ Self-distill. ( $p{=}1.0$ )	20.856	0.704	0.318	3.84	2.88
+ Stoch. rest. (Full)	21.206	0.776	0.276	3.72	3.05

Table 2: Ablation study. Each row adds one component to SceneExpander. “distill.” denotes distillation, “samp.” denotes sampling, “aug.” denotes augmentation, and “stoch. rest.” denotes stochastic restoration.

Figure 6: Multi-round expansion. Starting from an initial captured scene, users can repeatedly insert new views to expand the reconstruction. Across consecutive rounds, SceneExpander progressively enlarges the 3D scene while preserving previously reconstructed regions.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: More cases of SceneExpander. Left of the dashed line: input images, where the purple dashed box indicates the generated inserted view and the remaining images are captured views. The second and third cases show a representative subset of captured views for brevity. Right of the dashed line: novel views and corresponding depth maps rendered by camera navigation in the expanded scene.

Figure 8: User study interface. Participants rank videos A–E by insertion satisfaction (H-IS) based on consistency with the edit goal, with the edited region highlighted in red. Options A–E include our method and four baselines, shown in a randomized order. The H-RQ interface follows the same format.

Figure 9: A failure case of SceneExpander. The purple box shows the generated inserted view, and the red boxes highlight artifacts. SceneExpander alleviates misalignment on the two side pillars and the left stone pedestal, but still produces errors on the top crossbeam.

Panel of Figure 10 — Figure 10: Prompt for VLM-based evaluation on insertion satisfaction.

Panel of Figure 11 — Figure 11: Prompt for VLM-based evaluation on rendering quality.

Explore similar work

Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation
Mar 14, 2026Stefan Ainetter, Thomas Deixelberger, Edoardo A. Dominici +3Diffusion Model Guidance 3D Scene Generation
Huawei Technologies, Austria · Graz University of Technology, Austria · Huawei Technologies, Switzerland
Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas
Mar 30, 2026Felix Wimbauer, Fabian Manhardt, Michael Oechsle +43D Scene Generation Text-to-3D Generation
Google · MCML · Technical University of Munich +1
One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Image
Feb 23, 2026Pengfei Wang, Liyi Chen, Zhiyuan Ma +33D Scene Generation Single-View 3D Reconstruction
The Hong Kong Polytechnic University

Mar 14, 2026J/K move · Enter open · S save

Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation

Stefan Ainetter, Thomas Deixelberger, Edoardo A. Dominici, Philipp Drescher+2

Huawei Technologies, Austria · Graz University of Technology, Austria · Huawei Technologies, Switzerland

Diffusion Model Guidance 3D Scene Generation 3D Scene Layout Generation Text-to-3D Generation Indoor Scene Generation

Open paper AskPDF arXiv

We present GuidedSceneGen, a text-to-3D generation framework that produces metrically accurate, globally consistent, and semantically interpretable indoor scenes. Unlike prior text-driven methods that often suffer from geometric drift or scale ambiguity, our approach maintains an absolute world coordinate frame throughout the entire generation process. Starting from a textual scene description, we predict a global 3D layout encoding both semantic and geometric structure, which serves as a guiding proxy for downstream stages. A semantics- and depth-conditioned panoramic diffusion model then synthesizes 360° imagery aligned with the global layout, substantially improving spatial coherence. To explore unobserved regions, we employ a video diffusion model guided by optimized camera trajectories that balances coverage and collision avoidance, achieving up to 10x faster sampling compared to exhaustive path exploration. The generated views are fused using 3D Gaussian Splatting, yielding a consistent and fully navigable 3D scene in absolute scale. GuidedSceneGen enables accurate transfer of object poses and semantic labels from layout to reconstruction, and supports progressive scene expansion without re-alignment. Quantitative results and a user study demonstrate greater 3D consistency and layout plausibility compared to recent panoramic text-to-3D baselines.

Abstract

Explore similar work

Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation

Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas

One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Image