3D Scene Editing

Follow this topic Ask about this topic

3D scene editing turns instructions, images, or direct user input into changes to a reconstructed or generated world, supporting design, content creation, and simulation. Approaches include editing native representations such as 3D Gaussian splats, meshes, and neural fields, as well as editing 2D views with diffusion models and lifting the results into 3D; newer systems also use scene graphs, language agents, and structured action planning. Work spans local object changes, removal and insertion, layout editing, relighting, and dynamic scenes, with evaluation increasingly checking edit fidelity, locality, preservation of untouched regions, and consistency across viewpoints. The central tension is that flexible, prompt-faithful edits can damage geometry or surrounding content, while reliable preservation and multi-view consistency can limit the scale of changes or increase computation.

Latest papers

91 papers11 papers in the last four weeks, up 175% on the four weeks before. 0.1% of all new papers.

ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
Jun 3, 2025Martin JJ. Bucher, Iro Armeni3D Scene Editing 3D Scene Representation
Stanford University

Jun 3, 2025J/K move · Enter open · S save

ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing

Martin JJ. Bucher, Iro Armeni

Stanford University

3D Scene Editing 3D Scene Representation Autoregressive Generation Indoor Scene Generation Text-Guided 3D Scene Editing

Open paper AskPDF arXiv

Scene synthesis and editing has emerged as a promising direction in computer graphics. Current trained approaches for 3D indoor scene generation either oversimplify object semantics through one-hot class encodings (e.g., 'chair' or 'table'), require masked diffusion for editing, ignore room boundaries, or rely on floor plan renderings that fail to capture complex layouts. LLM-based methods enable richer semantics via natural language, but lack editing functionality, are limited to rectangular layouts, or rely on weak spatial reasoning from implicit world models. We introduce ReSpace, a generative framework for autoregressive text-driven 3D indoor scene synthesis and editing. Our approach features a compact structured scene representation with explicit room boundaries that enables asset-agnostic deployment and frames scene manipulation as a next-token prediction task, supporting object addition, removal, and swapping via natural language. We employ supervised fine-tuning with a preference alignment stage to train a specialized language model for object addition that accounts for user instructions, spatial geometry, object semantics, and scene-level composition. We further introduce a voxelization-based evaluation metric capturing fine-grained geometric violations beyond 3D bounding boxes. Experiments surpass state-of-the-art on object addition and achieve superior human-perceived quality on the application of full scene synthesis, despite not being trained on it.