cs.CVAug 29, 2026

Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding

Authors: Boyu Cai, Li Yang, Yan Xu, Wei Liu, Nian Liu, Sikui Zhang, Yan Wang, Chunfeng Yuan, 

Organizations: School of Information Science and Technology, ShanghaiTech University · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information, Institute of Automation, Chinese Academy of Sciences (CASIA) · Electronic Engineering Department, The Chinese University of Hong Kong · Deepeleph Intelligent Technology

Abstract

The integration of novel view synthesis (NVS) and open-vocabulary segmentation (OVS) has recently yielded powerful feed-forward 3D foundation models. However, their inherent reliance on static-scene assumptions leads to severe misalignment of spatial features in unconstrained dynamic environments. To bridge this critical gap, we propose SPAR, a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation. Furthermore, we introduce a dynamic-region-aware end-to-end training paradigm that structurally couples motion estimation with multi-view visual and semantic learning. This unified approach enables the network to inherently resolve motion conflicts and distill multi-view consistent, temporally stable scene representations from dynamic inputs. Extensive experiments on the challenging D-RE10K benchmark demonstrate that SPAR achieves state-of-the-art performance. Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively. Despite being trained in a self-supervised manner, our model achieves an mIoU of 88.5% for motion mask prediction. Furthermore, our analysis reveals a strong inter-task synergy between photometric scene reconstruction and semantic understanding, where semantic synthesis learning consistently enhances photometric fidelity in novel view rendering. Code will be available at https://github.com/dmucby/SPAR.

Figures & tables

Explore similar work

CardsList
  1. SemGS: Feed-Forward Semantic 3D Gaussian Splatting from Sparse Views for Generalizable Scene Understanding

    Mar 3, 2026Sheng Ye, Zhen-Hui Dong, Ruoyu Fan +23D Scene Understanding3D Gaussian Splatting

  2. FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views

    Apr 10, 2026Chaoyi Zhou, Run Wang, Feng Luo +43D ReconstructionFeed-Forward 3D Reconstruction

  3. BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model

    Feb 26, 2026Yuci Han, Charles Toth, John E. Anderson +2Novel View SynthesisMulti-View Image Synthesis