cs.CVNov 19, 2026

SceneAnything: Agentic Reconstruction of Large-Scale Mesh Scenes from a Single Video

Authors: Bowen Xue, Saeed Hadadan, Fabrice Rousselle, Zahra Montazeri, Ravi Ramamoorthi, Milos Hasan

Organizations: NVIDIA · University of Manchester · University of California, San Diego

Abstract

Reconstructing an editable 3D scene from video requires matching visible structure while plausibly completing unseen surfaces. Agents can write scene-construction programs, but direct reconstruction attempts often produce errors in layout, scale, and lighting. We introduce SceneAnything, an automated system for reconstructing mesh scenes from real and generated videos, enabling text-to-scene construction when paired with video generation. A software harness coordinates agents, pretrained vision models, and graphics tools through an explicit reconstruction protocol. Shared camera, depth, and segmentation estimates ground component construction: a scene-owner agent plans the environment and briefs authors, who write parameterized Blender programs. A visibility- and fit-based level-of-detail rule selects the construction level, while unseen regions are completed and marked as inferred. Assembled scenes use fitted lighting and supported motion, and revisions undergo programmatic checks and independent visual comparisons. Accepted programs and lessons are reused across scenes. We evaluate on 100 scenes; subset comparisons show improved original-view fidelity over direct-agent reconstruction and higher novel-view content similarity than the compared video-to-3D methods. We will release the harness, agent instructions, reusable libraries, and scenes.

Figures & tables

Explore similar work

CardsList
  1. ReplicateAnyScene: Zero-Shot Video-to-3D Composition via Textual-Visual-Spatial Alignment

    Apr 12, 2026Mingyu Dong, Chong Xia, Mingyuan Jia +43D Reconstruction3D Scene Generation

  2. SceneAgent: An Agentic Pipeline from 3D and 4D Captures to Simulation Environments

    Oct 26, 2026Luke Hollis, Tianxing Fan, Heng YangRobotics SimulationPhysics-Aware 3D Scene Reconstruction

  3. CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction

    Oct 12, 2026Jialei Wang, Zehan Wang, XiaoYang Wu +63D Reconstruction BenchmarksDynamic Scene Reconstruction