cs.CVJan 19, 2026

SplatReasoner: Enhancing Embodied Reasoning and Grounding by Novel View Synthesis

Authors: Kim Yu-Ji, Dahye Lee, Kim Jun-Seong, Nam Hyeon-Woo, GeonU Kim, Yongjin Kwon, Yu-Chiang Frank Wang, Jaesung Choe, +1 more

Organizations: POSTECH · KAIST · ETRI · NVIDIA

Abstract

Vision-Language Models (VLMs) have demonstrated strong reasoning capabilities over images and videos, yet their application to embodied scene understanding often constrained by the fixed viewpoints stored in episodic RGB-D memories. These observations may fail to capture query-relevant evidence due to occlusions, object truncation, restricted fields of view, or suboptimal view composition. We present SplatReasoner, a framework that introduces novel view synthesis into the VLM reasoning process by leveraging 3D Gaussian Splatting (3DGS). Given a user query about a 3D scene, SplatReasoner retrieves relevant observations and synthesizes query-conditioned viewpoints that reveal the visual evidence needed to answer the query and ground the referred entities in 3D. Experiments show that query-conditioned novel view synthesis improves both embodied reasoning and 3D grounding over fixed-view memory and language-embedded 3DGS baselines.

Explore similar work

CardsList
  1. OneCanvas: 3D Scene Understanding via Panoramic Reprojection

    Jun 17, 2026Bartłomiej Baranowski, Dave Zhenyu Chen, Matthias Nießner3D Scene UnderstandingSpatial Reasoning