4D Scene Understanding
Overview
4D scene understanding uses video and multi-view observations to recover scene geometry, track persistent objects, and interpret their motion and interactions. Common approaches include dynamic NeRFs and Gaussian splatting, point-cloud and occupancy models, scene graphs, and vision-language models that connect visual evidence with queries or actions. These methods support tasks such as novel-view rendering, occupancy forecasting, dynamic grounding, and robot planning, with evaluations on settings including nuScenes, Waymo, and HyperNeRF. A central challenge is balancing detailed, temporally consistent representations with efficient inference and reliable generalization, especially when observations are incomplete or language models must ground their answers in 3D motion.
Latest papers
57 papers18 papers in the last four weeks, up 500% on the four weeks before. 0.1% of all new papers.