SplatFlux: Feed-Forward 4D Gaussian Reconstruction and Generation from Video Latents

Runmin Zhang; Binglin Ren; Zhu Yu; Fudong Wang; Jiangke Lin; Yichun Shentu; Si-Yuan Cao; Yi Yuan

cs.CVOct 20, 2026

SplatFlux: Feed-Forward 4D Gaussian Reconstruction and Generation from Video Latents

Authors: Runmin Zhang, Binglin Ren, Zhu Yu, Fudong Wang, Jiangke Lin, Yichun Shentu, Si-Yuan Cao, Yi Yuan,

Organizations: Zhejiang University · Ant Group

Abstract

This work presents SplatFlux, a feed-forward latent-to-4DGS framework for both video-based 4D reconstruction and conditional 4D generation. Our key observation is that observed videos and conditionally generated content can be represented in the same pretrained video latent space, allowing both tasks to be formulated as latent-to-4DGS decoding. To construct compact and temporally coherent 4D scenes, SplatFlux introduces a static-dynamic decoupled 4D Gaussian representation that explicitly predicts dynamic probabilities for Gaussian primitives. This representation separates static scene structures from moving regions, enabling complete 4D scene construction in a feed-forward manner through static aggregation and dynamic motion interpolation. To learn static-dynamic decomposition and dynamic Gaussian motion without costly annotations, we further develop an auxiliary distillation strategy that provides pseudo dynamic masks and optical-flow cues. Extensive experiments across 3D and 4D reconstruction and generation demonstrate that SplatFlux achieves state-of-the-art performance.

Figures & tables

Figure 2: Overview of SplatFlux. SplatFlux unifies video-based reconstruction and conditional generation through a shared latent-to-4DGS decoder. It predicts a static-dynamic decoupled 4D Gaussian representation, where static Gaussians are aggregated across frames and dynamic Gaussians are propagated by local motion for arbitrary-time and arbitrary-view rendering.

Method	NVIDIA Dynamic Scene			DyCheck			DAVIS
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	mPSNR $\uparrow$	mSSIM $\uparrow$	mLPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
4DGT* Xu et al. (2025b)	15.97	0.3426	0.5408	13.98	0.4609	0.5225	17.54	0.4767	0.4581
MoVieS* Lin et al. (2025a)	18.72	0.5059	0.2527	13.89	0.4802	0.4536	19.97	0.5927	0.2516
NeoVerse Yang et al. (2026)	18.83	0.4695	0.2881	14.57	0.4639	0.4645	18.58	0.5067	0.3198
SplatFlux (Ours)	21.56	0.5879	0.1918	14.68	0.5007	0.4201	20.84	0.6083	0.2494

Table 1: Quantitative comparison of dynamic novel view synthesis on the NVIDIA Dynamic Scene, DyCheck, and DAVIS datasets. * denotes approaches that require camera pose as input. The best and second-best results are highlighted for clarity.

Figure 3: Qualitative comparison of dynamic novel view synthesis across different methods.

Method	Imaging $\uparrow$	Aesthetic $\uparrow$	CLIP $\uparrow$	Alignment $\uparrow$	Coherence $\uparrow$	Style $\uparrow$
Prometheus ( Yang et al., 2025b )	44.73	45.85	28.57	3.20	3.36	2.98
SplatFlow ( Go et al., 2025b )	48.85	53.71	29.43	3.47	3.65	3.26
VIST3A ( Go et al., 2025a )	64.87	56.96	30.18	3.67	3.86	3.40
SplatFlux (Ours)	69.18	58.08	31.35	3.72	3.97	3.48

Table 2: Quantitative comparison of text-to-3D generation on SceneBench ( Yang et al., 2025b ) . The best and second-best results are highlighted for clarity.

Method	Sub. consistency $\uparrow$	B.g. consistency $\uparrow$	Dynamic degree $\uparrow$	Aesthetic $\uparrow$	Imaging $\uparrow$
Mochi ( Team, 2024 )	95.07	94.37	54.08	52.37	56.09
HunyuanVideo ( Kong et al., 2024 )	95.94	94.78	37.76	53.51	64.69
SteerX ${}_{\textrm{Mochi}}$ ( Park et al., 2025 )	94.07	93.88	57.14	49.69	57.13
SteerX ${}_{\textrm{HunyuanVideo}}$ ( Park et al., 2025 )	96.11	95.23	37.76	53.32	61.23
SplatFlux (Ours)	96.11	95.90	74.49	64.07	70.92

Table 3: Quantitative comparison of text-to-4D generation on PenguinVideoBenchmark ( Kong et al., 2024 ) . SteerX

{}_{\textrm{Mochi/HunyuanVideo}}

denotes the variant using Mochi ( Team, 2024 ) or HunyuanVideo ( Kong et al., 2024 ) as the base model. The best and second-best results are highlighted for clarity.

Figure 4: Qualitative comparison of conditional 3D and 4D generation across different methods.

Method	Sub. consistency $\uparrow$	B.g. consistency $\uparrow$	Dynamic degree $\uparrow$	Aesthetic $\uparrow$	Imaging $\uparrow$
CogVideoX ( Yang et al., 2024 )	94.86	94.83	40.00	63.70	69.56
SteerX ${}_{\textrm{CogVideoX}}$ ( Park et al., 2025 )	95.91	95.43	31.55	64.16	71.55
Aether ( Zhu et al., 2025 )	97.97	97.03	19.72	63.89	69.85
SplatFlux (Ours)	95.95	96.01	49.86	64.84	72.07

Table 4: Quantitative comparison of image-to-4D generation on VBench-I2V ( Huang et al., 2024 ) . SteerX

{}_{\textrm{CogVideoX}}

denotes the variant using CogVideoX ( Yang et al., 2024 ) as the base model. The best and second-best results are highlighted for clarity.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domain	Real?	Dynamic?	#Scenes
ARKitScenes ( Baruch et al., 2021 )	Indoor	✓	✗	675
BEDLAM ( Black et al., 2023 )	Mixed	✗	✓	7309
CO3Dv2 ( Reizenstein et al., 2021 )	Object	✓	✗	2500
DL3DV ( Ling et al., 2024 )	Mixed	✓	✗	6370
Dynamic Replica ( Karaev et al., 2023 )	Indoor	✗	✓	483
Hypersim ( Roberts et al., 2021 )	Indoor	✗	✗	741

Appendix

Table 6: Training Datasets of SplatFlux.

Method	Sintel		Bonn		KITTI
Method	Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Rel $\downarrow$	$\delta_{1}$ $\uparrow$	Rel $\downarrow$	$\delta_{1}$ $\uparrow$
VDA ( Chen et al., 2025a )	0.855	0.340	0.301	0.465	0.272	0.535
DepthCrafter ( Hu et al., 2025a )	0.469	0.455	0.074	0.968	0.282	0.435
GeometryCrafter ( Xu et al., 2025a )	0.305	0.675	0.057	0.969	0.057	0.981
VGGT ( Wang et al., 2025c )	0.373	0.634	0.057	0.966	0.062	0.969
Pi3 ( Wang et al., 2026c )	0.281	0.663	0.049	0.975	0.038	0.985

Appendix

Table 7: Quantitative comparison of scale-invariant video depth estimation on the Sintel ( Butler et al., 2012 ) , Bonn ( Palazzolo et al., 2019 ) , and KITTI ( Geiger et al., 2013 ) datasets. The best and second-best results are highlighted for clarity.

Method	6 Views			12 Views
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
WorldMirror ( Liu et al., 2025 )	21.69	0.7050	0.2418	20.61	0.6558	0.2933
DA3 ( Lin et al., 2026 )	20.86	0.6472	0.2395	20.54	0.6241	0.2592
NeoVerse ( Yang et al., 2026 )	20.02	0.6170	0.3267	18.99	0.5550	0.3915
SplatFlux (Ours)	22.17	0.7309	0.2287	21.01	0.6731	0.2778

Appendix

Table 8: Quantitative comparison of static novel view synthesis on the DL3DV-140 benchmark ( Ling et al., 2024 ) with 6 and 12 unposed input views. The best and second-best results are highlighted.

Method	DAVIS-16				DAVIS-17				DAVIS-all
Method	JM $\uparrow$	JR $\uparrow$	FM $\uparrow$	FR $\uparrow$	JM $\uparrow$	JR $\uparrow$	FM $\uparrow$	FR $\uparrow$	JM $\uparrow$	JR $\uparrow$	FM $\uparrow$	FR $\uparrow$
Easi3R ${}_{\textrm{DUSt3R}}$ (w/ SAM2) ( Chen et al., 2025b )	71.7	80.5	71.9	78.1	64.3	71.8	64.3	69.4	56.4	63.2	55.6	58.5
Easi3R ${}_{\textrm{MonST3R}}$ (w/ SAM2) ( Chen et al., 2025b )	70.7	81.8	68.4	79.4	66.9	76.2	65.2	75.7	63.2	71.1	61.0	66.9
VGGT4D ( Hu et al., 2025b )	62.1	76.8	56.0	67.5	56.5	65.6	51.1	56.9	50.8	55.6	47.0	46.4
SplatFlux (Ours)	76.8	89.3	81.2	92.5	76.7	88.1	80.9	91.2	73.6	82.5	78.6	87.3

Appendix

Table 9: Quantitative comparison of dynamic object segmentation on the DAVIS ( Perazzi et al., 2016 ) dataset. Easi3R

{}_{\textrm{DUSt3R/MonST3R}}

denotes the variant using DUSt3R ( Wang et al., 2024 ) or MonST3R ( Zhang et al., 2025a ) as backbone, and w/ SAM2 denotes using SAM2 ( Ravi et al., 2024 ) for mask refinement. The best and second-best results are highlighted for clarity.

Figure 6: Qualitative comparison of dynamic object segmentation across different methods.

Figure 7: Qualitative comparison between the pseudo dynamic mask and the dynamic mask predicted by SplatFlux.

Figure 8: Visualization of 4DGS scenes reconstructed by SplatFlux. Rows show the input video, fixed-camera rendering, frozen-time rendering, and joint camera–time variation. Columns follow the progression of each visualization sequence.

Figure 9: Visualization of 4DGS scenes generated by SplatFlux. The corresponding text or image condition is shown on the left. For each scene, rows show fixed-camera rendering, frozen-time rendering, and joint camera-time variation. Columns follow the progression of each visualization sequence.

4D Reconstruction

4D reconstruction recovers changing 3D scenes, shapes, and motion over time from images, videos, or other sparse measurements.

See all 137 papers

Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction
May 29, 2026Mungyeom Kim, Minkyeong Jeon, Honggyu An +10Point Tracking 3D Gaussian Splatting
KAIST AI · ETH Zürich, ETH AI Center · Sony AI +1
Beyond Pixels: From Video Priors to 4D Worlds
Aug 11, 2026Zihao Liu, Xiaolong Shen, Zhenglin Zhou +2Video Diffusion Models 4D Scene Generation
ReLER, CCAI, Zhejiang University
DynStream: Online Streaming 4D Gaussian Reconstruction of Dynamic Worlds from Unposed Video
Oct 7, 2026Dingwei Xian, Xiaoyu Zhou, Yajiao Xiong +24D Reconstruction Dynamic Scene Reconstruction
Wangxuan Institute of Computer Technology, Peking University · VGI Labs Co., Ltd. · University of California, Merced

May 29, 2026J/K move · Enter open · S save

Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction

Mungyeom Kim, Minkyeong Jeon, Honggyu An, Jaewoo Jung+9

KAIST AI · ETH Zürich, ETH AI Center · Sony AI+1

Point Tracking 3D Gaussian Splatting 4D Reconstruction Dynamic Scene Reconstruction Feed-Forward 3D Reconstruction

Open paper AskPDF arXiv

Dynamic scene reconstruction from monocular video remains a fundamental challenge in computer vision. Existing feed-forward methods predict 3D Gaussians pixel-wise for each frame, suffering from duplicated Gaussians and view-dependent biases that hinder effective learning of scene motion. We present C4G, a feed-forward 4D reconstruction framework built upon a compact set of timestamp-conditioned learnable Gaussian query tokens. Each token aggregates corresponding features across the full temporal context and decodes a 3D Gaussian whose position is modulated by the target timestamp, enabling globally coherent motion modeling without per-scene optimization. To capture fine-grained details, we further introduce a video diffusion model-based rendering enhancement module. Since our framework effectively aggregates features into Gaussians, we extend this capability to feature lifting, producing a 4D feature field that supports point tracking and dynamic scene understanding. C4G achieves strong novel-view synthesis performance using significantly fewer Gaussians and without requiring camera poses, while exhibiting stronger motion modeling and robustness to large temporal gaps.

Abstract

4D Reconstruction

Explore similar work

Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction

Beyond Pixels: From Video Priors to 4D Worlds

DynStream: Online Streaming 4D Gaussian Reconstruction of Dynamic Worlds from Unposed Video