Overview

4D scene understanding uses video and multi-view observations to recover scene geometry, track persistent objects, and interpret their motion and interactions. Common approaches include dynamic NeRFs and Gaussian splatting, point-cloud and occupancy models, scene graphs, and vision-language models that connect visual evidence with queries or actions. These methods support tasks such as novel-view rendering, occupancy forecasting, dynamic grounding, and robot planning, with evaluations on settings including nuScenes, Waymo, and HyperNeRF. A central challenge is balancing detailed, temporally consistent representations with efficient inference and reliable generalization, especially when observations are incomplete or language models must ground their answers in 3D motion.

Latest papers

CardsList

64 papers9 papers in the last four weeks, down 44% on the four weeks before. 0.1% of all new papers.

  1. π4π^4: A Foundation Model for 4D Scene Geometry and Motion

    Nov 17, 2026Yifan Wang, Weiguang Zhao, Ni Yang +44D Scene UnderstandingMulti-View 3D Reconstruction

  2. IGRecon4D: Dynamic-Object-Aware Feed-Forward Reconstruction of Driving Scenes

    Nov 16, 2026Yifei Tong, Ziyang Huang, Yizhao Han +2Dynamic Scene Reconstruction4D Scene Understanding

  3. UGG: Unified Generative Geometry Model

    Nov 10, 2026Amani Kiruga, Zhiyi Li, Ruojin Cai +44D Scene UnderstandingCamera Pose Estimation

  4. Trace4D: Grounded Spatiotemporal Reasoning via 4D State Sequences

    Nov 2, 2026Wooseok Jang, Seungryong Kim, Yan Lu +1Spatiotemporal ReasoningTemporal Reasoning in VLMs

  5. Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling

    Oct 29, 2026Haitang Feng, Zhuoman Liu, Zhennan Wang +5Embodied World ModelsMultimodal Video Generation

  6. R4DGS: Referring Segmentation in 4D Gaussian Splatting

    Oct 21, 2026Bangpu Chen, Yaxuan Li, Shirui Peng +5Multimodal Grounding4D Scene Understanding

  7. Cross-Visibility-Gap Identity Persistence for 4D Scene Graphs

    Oct 20, 2026Jiahui Hu, Li Gao, Yu Liu +14D Scene UnderstandingMulti-Object Tracking

  8. Gen4P: Video Generators are 4D Perceivers

    Oct 15, 2026Yitong Xia, Zeren Jiang, Chuanxia ZhengVideo Depth Estimation4D Scene Understanding

  9. Act4R: Action-conditioned 4D reconstruction for kinematics-free control

    Oct 12, 2026Zhaoliang Zhang, Chenhao Yu, Zhimin Shao +2Scene Flow EstimationVision-Based Robot Control

  10. InfraWorld: Motion-Aware World Modeling for Roadside 4D Occupancy Forecasting

    Oct 10, 2026Songkai Wang, Xiaokai Bai, Lei Yang +1Spatiotemporal Forecasting3D Occupancy Prediction

  11. SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning

    Oct 10, 2026Chaoyue Li, Shoudong Han, Haoyang Wu +4Spatiotemporal Reasoning4D Scene Understanding

  12. CoCam4D: Geometry-Aware Cooperative 4D Perception for Camera-Only Autonomous Driving

    Oct 8, 2026Soham Pahari, Sudip Das, Arindam Das +1Autonomous Driving PerceptionCooperative Perception

  13. DeepTopoClustering: Unsupervised Derivation of Surface Process Taxonomy from 4D Point Clouds for Topographic Monitoring

    Oct 7, 2026Jiapan Wang, Daan Hulskemper, Mathilde Letard +2Deep Clustering4D Scene Understanding

  14. LighTROcc: Lightweight 4D Occupancy Forecasting via Instance-Centric 3D Gaussians

    Oct 7, 2026Hwanhee Jung, SeungHyeon Kim, Inkyu Koo +33D Occupancy Prediction4D Scene Understanding

  15. SCOPE-4D: Endoscopic 4D Geometry Foundation Models

    Oct 1, 2026Chaoyi Zhou, Zhongpai Gao, Anwesa Choudhuri +6Endoscopic 3D ReconstructionEndoscopy

  16. Dyna3: VLM-Guided Training-Free 4D Reconstruction via Depth Foundation Models

    Oct 1, 2026Xinhao Xiang, Weiyang Li, Zhijie Zheng +2Multi-View Depth Estimation4D Reconstruction

  17. A Compact Explicit 4D Representation for Dynamic Scenes

    Oct 1, 2026Di Yang, Zhihao Li, Yanhai Xiong +1Dynamic Scene ReconstructionNovel View Synthesis

  18. ChronoGraph: Functional 4D Scene Graphs with Vision-Language Models for Interaction Understanding and Grounded Planning

    Sep 30, 2026Chenyangguang Zhang, Malgorzata Gwiazda, Guanlong Jiao +5Graph-Based Planning4D Scene Understanding

  19. S4VY: Segment Anything in Feed-Forward 4D Visual Geometry

    Sep 29, 2026Jingdong Zhang, Xin Li, Jan Kautz +2Instance SegmentationPromptable Image Segmentation

  20. MVVBench: Benchmarking 4D Reasoning in Vision-Language Models

    Sep 25, 2026Hyungjin Chung, Byeongjun Park, Joonseok Lee +3Reasoning for VLMsVideo Reasoning

  21. Kairos: Grounded Forecasting of Presence and Directional Flow in 4D Scene Graphs

    Sep 23, 2026Iacopo Catalano, Julio A. Placed, Javier Civera +1Spatiotemporal ForecastingProbabilistic Forecasting

  22. Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning

    Sep 3, 2026Yijun Yang, Shenghe Zheng, Wenbo Li +8Visual Spatial Reasoning3D Spatial Reasoning

  23. CL4D: Contrastive Language-4D Pretraining for Vision-Language Reasoning in Dynamic Scenes

    Aug 19, 2026Kumal Hewagamage, Isuranga Senavirathne, Sasika Amarasinghe +4Vision-Language ModelsVision-Language Alignment

  24. Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting

    Aug 15, 2026Sitao Chen, Zhuangwei Zhuang, Hui Luo +23D Occupancy Prediction4D Scene Understanding