Depth Anything V4: Dynamic 4D Scene Reconstruction via Riemannian Flow Matching on 4D Gaussian Splatting

Jiaming Fan; Jian Lu; Jinling Jia; Chenbin Zhang

cs.CVAug 18, 2026

Depth Anything V4: Dynamic 4D Scene Reconstruction via Riemannian Flow Matching on 4D Gaussian Splatting

Authors: Jiaming Fan, Jian Lu, Jinling Jia, Chenbin Zhang

Organizations: College of Artificial Intelligence, Nanjing University of Posts and Telecommunications · College of Automation, Nanjing University of Posts and Telecommunications · Institute of Ad-vanced Technology for Carbon Neutrality, Nanjing University of Posts and Telecommunications

Abstract

We present Depth Anything V4 (DAV4), a framework for dynamic 4D scene reconstruction from monocular video. Our key contribution is the application of Riemannian Flow Matching (RFM) to 4D Gaussian Splatting parameters, defining probability paths directly on non-Euclidean manifolds (scale, rotation, opacity), ensuring all intermediate states are valid. Through controlled experiments, we isolate RFM's contribution from test-time optimization (TTO) and pre-training. A deterministic MLP baseline with the same data, architecture, and TTO achieves F-score 0.762; RFM achieves 0.806 - the +0.044 gain is RFM's isolated contribution. We provide corrected computational cost analysis: pre-training is 360 GPU-hours, amortizing for large-scale deployment (over 10,000 scenes). Uncertainty is quantified via Negative Gaussian Log-Likelihood and Expected Calibration Error. DAV4 outperforms prior Depth Anything models and per-scene 4D-GS on dynamic reconstruction and novel-view synthesis, while using no human-annotated depth labels as training losses.

Explore similar work

Dyna3: VLM-Guided Training-Free 4D Reconstruction via Depth Foundation Models
Oct 1, 2026Xinhao Xiang, Weiyang Li, Zhijie Zheng +2Multi-View Depth Estimation 4D Reconstruction
IFM Lab, University of California, Davis, CA, USA · Genies Inc., CA, USA
UniQuery4R: Unified 4D Scene Reconstruction from a Single Query
Aug 18, 2026Tiancheng Chen, Sheng Tang, Wenhua Jin +4Dynamic Scene Reconstruction 4D Reconstruction
Kosmo Research · Automotive Engineering Department, Jilin University · School of Software, Tsinghua University
Learning Efficient 4D Gaussian Representations from Monocular Videos with Flow Splatting
Jun 29, 2026Shengjun Zhang, Jinzhao Li, Xin Fei +13D Gaussian Splatting Dynamic Scene Reconstruction
1Tsinghua University · 2National University of Singapore

Oct 1, 2026J/K move · Enter open · S save

Dyna3: VLM-Guided Training-Free 4D Reconstruction via Depth Foundation Models

Xinhao Xiang, Weiyang Li, Zhijie Zheng, Abhijeet Rastogi+1

IFM Lab, University of California, Davis, CA, USA · Genies Inc., CA, USA

Multi-View Depth Estimation 4D Reconstruction Dynamic Scene Reconstruction 4D Scene Understanding

Open paper AskPDF arXiv

Recent depth foundation models like Depth Anything 3 (DA3) achieve remarkable multi-view depth estimation but assume static 3D scenes, limiting their applicability to real-world dynamic environments. Existing training-free 4D methods like Easi3R and VGGT4D rely on correspondence-trained backbones whose attention encodes cross-frame matching, a property absent in depth-only models like DA3. We present Dyna3, a training-free framework that extends DA3 for 4D dynamic scene reconstruction without any fine-tuning. Our key insight is that DA3's cross-view features, though trained only for depth consistency, implicitly encode motion-discriminative signals when combined with best-match feature search across frames. Its static surfaces find consistent matches globally, while dynamic objects cannot. We further adopt vision-language models (VLM) to automatically generate scene-specific semantic prompts for SAM 3, enabling precise instance-level segmentation that distinguishes which objects move from what objects exist. For reconstruction, we decouple the scene into a cross-frame aligned static background and per-frame dynamic point clouds. Experiments on four datasets demonstrate that Dyna3 surpasses correspondence-trained methods with +5.5pp J-Mean over state-of-the-art VGGT4D on dynamic object segmentation, while achieving up to 13x faster pose estimation and 3x faster 4D reconstruction with 4 to 8x lower memory. Dyna3 could therefore enable much denser temporal sampling that prior methods cannot support.

Abstract

Explore similar work

Dyna3: VLM-Guided Training-Free 4D Reconstruction via Depth Foundation Models

UniQuery4R: Unified 4D Scene Reconstruction from a Single Query

Learning Efficient 4D Gaussian Representations from Monocular Videos with Flow Splatting