FL4N: Feed-forward Latent Model for 4D Novel View Synthesis

Eunbeen Hong; Hyeonseo Yu; Honggyu An; Jooho Son; Hyunah Ko; Soowon Son; Chaehyun Kim; Jahyeok Koo

cs.CVOct 26, 2026

FL4N: Feed-forward Latent Model for 4D Novel View Synthesis

Authors: Eunbeen Hong, Hyeonseo Yu, Honggyu An, Jooho Son, Hyunah Ko, Soowon Son, Chaehyun Kim, Jahyeok Koo,

Organizations: 1KAIST AI · 2Google DeepMind

Abstract

4D novel view synthesis (NVS) aims to render dynamic scenes across viewpoints and timestamps, which requires jointly inferring 3D structure and temporal dynamics. Existing monocular feed-forward methods often rely on explicit 4D Gaussian representations, which are prone to temporal inconsistency and severe rendering artifacts. We present FL4N, the first feed-forward latent 4D NVS model. Instead of relying on explicit 4D primitives, FL4N encodes a 4D scene prior into a unified latent representation across space and time. Our core insight is that learning temporal dynamics as residuals on top of a pretrained static latent prior enables effective 4D modeling with limited dynamic training data. To this end, we introduce a lightweight dynamic encoder that leverages static scene features to disentangle object motion from camera-induced parallax. Trained solely with an image synthesis objective, FL4N achieves state-of-the-art 4D NVS quality while rendering in real time. Its learned latent representation generalizes to real-world videos and disentangles viewpoint from time.

Figures & tables

Figure 1: Teaser. FL4N is the first feed-forward latent 4D novel view synthesis model. It models dynamics in a latent space to render temporally coherent frames, whereas Gaussian-based methods such as 4DGT ( Xu et al., 2025 ) , MoVieS ( Lin et al., 2026a ) , NeoVerse ( Yang et al., 2026 ) , and C4G ( Kim et al., 2026c ) suffer from temporal ghosting, artifacts, holes, or blur.

Figure 2: FL4N Overview. (Left) During training, context and target frames with varying views and timestamps are sampled from multi-view videos. (Right) A static encoder encodes static 3D structures (static latent) while a lightweight dynamic encoder extracts temporal residuals (residual latent) within the whole context frames. For 4D NVS, the residual latent is first adapted at the target time and added to the static latent resulting in a dynamic latent, which is then rendered into the novel view with target-view conditioned.

Figure 3: Rendered frames from the static latent and our residual time correction. In dynamic scenes, the static latent suppresses dynamic content (left) or copies it from the nearest context frame (right), whereas our residual latent places the moving content at the target time while preserving the static background.

Figure 4: PCA visualization of static and residual latent. The static latent captures camera-ray-like structure, whereas the residual latent primarily attends to moving objects and exhibits temporal variation.

Figure 5: Visualization of residual gate activation. The residual gate magnitude is higher on dynamic objects while remaining suppressed in the background.

Figure 6: Three evaluation settings given synchronized multi-view video datasets. Red outlines indicate input context frames, while blue outlines indicate target frames for evaluation.

Method	Pose	NVIDIA			N3DV			Kubric4D			CamxTime			Charge
Method	Pose	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Feed-forward Gaussian Representation
4DGT ( Xu et al., 2025 )	✓	16.127	0.3884	0.5972	17.813	0.6609	0.5131	14.148	0.5027	0.6096	17.125	0.6557	0.4893	18.017	0.7786	0.5653
MoVieS ( Lin et al., 2026a )	✓	11.818	0.2546	0.6760	20.136	0.7045	0.4605	12.411	0.3550	0.6070	17.069	0.6223	0.4488	18.709	0.7473	0.5298
NeoVerse † ( Yang et al., 2026 )	✗	15.050	0.4135	0.4626	12.953	0.4806	0.5697	8.954	0.2573	0.6721	10.432	0.3110	0.5905	15.224	0.5211	0.5919
C4G † ( Kim et al., 2026c )	✗	16.572	0.4182	0.6203	18.444	0.7050	0.5246	14.931	0.5656	0.6442	20.527	0.7056	0.4369	18.535	0.7797	0.5279

Table 1: Quantitative results on Novel View Synthesis: FL4N outperforms all Gaussian-based methods on all benchmark datasets, showing the strength of our latent representation bypassing explicit primitives. Context pose input further improves the metrics, achieving the state of the art. Best , second best , and third best results are highlighted.

Method	Pose	NVIDIA			N3DV			Kubric4D			CamxTime			Charge
Method	Pose	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Feed-forward Gaussian Representation
4DGT ( Xu et al., 2025 )	✓	15.937	0.3867	0.6052	16.097	0.6380	0.5483	15.680	0.5424	0.5786	17.024	0.6573	0.4939	17.939	0.7756	0.5617
MoVieS ( Lin et al., 2026a )	✓	15.785	0.4614	0.5328	19.710	0.7400	0.4414	16.988	0.6368	0.4174	17.340	0.6470	0.4224	20.113	0.7771	0.5084
NeoVerse † ( Yang et al., 2026 )	✗	14.449	0.3819	0.4890	12.447	0.4729	0.5681	8.885	0.2256	0.6793	10.276	0.3091	0.5911	15.220	0.5380	0.5940
C4G † ( Kim et al., 2026c )	✗	16.403	0.4096	0.6304	17.039	0.6829	0.5374	15.270	0.5572	0.6348	20.175	0.7009	0.4453	18.410	0.7640	0.5324

Table 2: On the challenging Cross-Time View Synthesis task, FL4N substantially outperforms all Gaussian-based methods on all benchmark datasets, even without test-time pose alignment ( † ), showing the model successfully learns 4D priors and decouples view and time in the latent space. Best , second best , and third best results are highlighted.

Method	Pose	DyCheck			ADT			TUM
Method	Pose	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Feed-forward Gaussian Representation
4DGT ( Xu et al., 2025 )	✓	13.743	0.4079	0.6233	16.710	0.4999	0.5651	14.380	0.4468	0.5612
MoVieS ( Lin et al., 2026a )	✓	11.755	0.3420	0.6428	15.432	0.4698	0.5828	15.683	0.5079	0.4426
NeoVerse † ( Yang et al., 2026 )	✗	16.756	0.4663	0.4566	16.179	0.4384	0.5296	16.120	0.5266	0.3981
C4G † ( Kim et al., 2026c )	✗	15.755	0.4735	0.6111	18.900	0.5813	0.5833	17.508	0.5518	0.4951

Table 3: On Novel Time Synthesis , FL4N achieves the state-of-the-art accuracy by a large margin over all methods, showing the strong generalization of our latent-based approach at novel interpolated timestamps. Best , second best , and third best results are highlighted.

Variant	Components				Full frame			Dynamic region
Variant	Residual branch	Static anchoring	Mask loss	Frozen renderer	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
(i)	✗	–	✗	✗	$21.902$	$0.7272$	$0.2872$	$13.161$	$0.4851$	$0.4375$
(ii)	✓	✓	✓	✗	$22.147$	$0.6807$	$0.3095$	$13.828$	$0.4855$	$0.4647$
(iii)	✓	✓	✗	✓	$22.929$	$0.7405$	$0.2623$	$14.112$	$0.5002$	$0.4288$
(iv)	✓	✗	✓	✓	$23.608$	$0.7692$	$0.2461$	$14.253$	$0.5169$	$0.4363$
FL4N	✓	✓	✓	✓	23.666	0.7734	0.2369	14.264	0.5192	0.4189

Table 4: Ablation studies on Kubric4D, on full frames and dynamic region. Each component contributes to our final formulation.

Method	sec / view $\downarrow$	FPS $\uparrow$	PSNR $\uparrow$
4DGT	0.040	25.00	12.15
MoVieS	0.037	27.02	14.10
NeoVerse	0.027	37.03	10.74
NeoVerse †	14.127	0.07	13.86
C4G	0.036	27.77	14.25
C4G †	23.885	0.04	16.76

Table 5: Runtime. † denotes 2,000 step test-time pose alignment.

Figure 7: Qualitative comparison on 4D Novel View Synthesis. Gaussian-based methods often exhibit holes, object inconsistencies, and blurred details. Our method produces more coherent and visually consistent results, demonstrating the superiority of our latent-based representation over explicit geometry primitives.

Region	Feat.	$\delta_{\text{avg}}$	$\delta$ 1	$\delta$ 4	$\delta$ 16
All	$\mathbf{z}_{\text{st}}$	27.33	5.65	25.37	53.40
All	$\mathbf{z}_{\text{dyn}}$	29.04	5.76	26.50	57.64
		( $+$ 1.71)	( $+$ 0.11)	( $+$ 1.13)	( $+$ 4.24)
Dyn	$\mathbf{z}_{\text{st}}$	23.45	3.10	21.14	49.03
Dyn	$\mathbf{z}_{\text{dyn}}$	25.29	3.12	22.37	53.68
		( $+$ 1.84)	( $+$ 0.02)	( $+$ 1.23)	( $+$ 4.65)

Table 6: Zero-shot matching on DAVIS. Dynamic latent outperform static latent by encoding temporal information.

Region	Feat.	$\Delta$ 4		$\Delta$ 8
Region	Feat.	AbsRel $\downarrow$	$\delta<1.25\uparrow$	AbsRel $\downarrow$	$\delta<1.25\uparrow$
All	$\mathbf{z}_{\text{st}}$	0.0944	0.8908	0.0868	0.9081
	$\mathbf{z}_{\text{dyn}}$	0.0863	0.9035	0.0751	0.9326
		( $-$ 0.0081)	( $+$ 0.0127)	( $-$ 0.0117)	( $+$ 0.0244)
Dyn	$\mathbf{z}_{\text{st}}$	0.1178	0.8699	0.1149	0.8790
	$\mathbf{z}_{\text{dyn}}$	0.1050	0.8941	0.0996	0.9099

Table 7: Probing video depth on Kubric. Dynamic latent yields consistent gains, with larger gains at greater temporal strides.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A: Visualization of static latent. The static latent encodes camera pose, with similar colors for cameras pointing in the same direction. Red boxes highlight overlapping regions across context frames, which exhibit similar color gradients.

Figure B: Visualization of residual latent. (Left) Ground-truth RGB frame overlaid with the dynamic mask. (Right) Corresponding residual latent grid. Dynamic objects exhibit more pronounced residual latents as the target pose and time diverge from the input, indicating a larger discrepancy from the context frames.

Method	Pose	NVS						CTVS
		Kubric4D			Charge			Kubric4D			Charge
		PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Feed-forward 4D Gaussian Representation
4DGT ( Xu et al., 2025 )	✓	12.089	0.2406	0.0305	17.703	0.6516	0.1127	12.405	0.2557	0.0303	17.708	0.6588	0.1157
MoVieS ( Lin et al., 2026a )	✓	11.401	0.1741	0.0318	17.969	0.6079	0.1130	11.797	0.2034	0.0307	18.068	0.6119	0.1147
NeoVerse † ( Yang et al., 2026 )	✗	10.847	0.1527	0.0381	16.292	0.5161	0.1242	10.254	0.1372	0.0403	16.152	0.4966	0.1307

Appendix

Table A: 4D NVS result for Dynamic region. † denotes 2,000 step test-time pose alignment, the best , second best , and third best results in each column are highlighted. Metrics are restricted to the moving pixels and are computed over the same target subset, and their full-frame counterparts for the same cells are reported in Tables 1 and 2 .

Figure C: Further qualitative comparison. Gaussian-based methods exhibit holes, object inconsistencies, and blurred details. Our method produces more coherent and visually consistent results, demonstrating the superiority of our latent-based representation over explicit geometry primitives.

Method	DAVIS-2016				DAVIS-2017				DAVIS-all
Method	JM $\uparrow$	JR $\uparrow$	FM $\uparrow$	FR $\uparrow$	JM $\uparrow$	JR $\uparrow$	FM $\uparrow$	FR $\uparrow$	JM $\uparrow$	JR $\uparrow$	FM $\uparrow$	FR $\uparrow$
Easi3R ${}_{\text{dust3r}}$ ( Chen et al., 2025 )	50.10	55.77	43.40	37.25	46.86	50.54	39.06	30.05	44.10	50.85	35.16	27.24
Easi3R ${}_{\text{monst3r}}$ ( Chen et al., 2025 )	54.93	68.00	45.29	47.30	54.75	66.16	44.09	42.36	51.64	63.06	40.98	38.49
MonST3R ( Zhang et al., 2025 )	40.42	40.39	49.54	52.12	38.07	36.05	48.24	49.01	36.98	34.52	47.03	46.72
DAS3R ( Xu et al., 2024a )	41.13	38.67	44.50	36.94	44.51	43.95	46.71	44.96	43.33	38.93	45.24	38.78
VGGT4D ( Hu et al., 2026 )	62.12	76.80	56.04	67.49	56.45	65.62	51.09	56.85	50.75	55.59	47.04	46.43

Appendix

Table B: Quantitative results on dynamic regions. On the dynamic object segmentation task, Our method substantially outperforms other methods for 3D or 4D geometry estimation. Best and second best results are highlighted.

Figure D: Qualitative comparison of dynamic masks. Our method show more accurate dynamic object masks with clearer object boundary than VGGT4D ( Hu et al., 2026 ) .

Figure E: Generalization to the number of context frames , reporting PSNR on CamxTime as the number of conditioning frames varies at inference. The vertical line marks the $8$ frames the model is trained with. The rendering quality remains on a flat plateau as the number of context frames increases to $32+$ . The dynamic encoder generalizes to substantially longer temporal contexts than trained and effectively exploits temporal context far beyond the number of frames seen during training.

Variant	$\Delta$ 1			$\Delta$ 2			$\Delta$ 4			$\Delta$ 8
Variant	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Baselines
Static NVS model	21.90	0.727	0.287	20.87	0.716	0.308	20.49	0.705	0.325	20.35	0.700	0.332
Static NVS model, fine-tuned	21.83	0.668	0.320	21.30	0.664	0.334	21.32	0.666	0.338	21.33	0.665	0.339
Architecture
Unfrozen renderer	22.15	0.681	0.309	21.57	0.676	0.325	21.55	0.676	0.332	21.55	0.676	0.334

Appendix

Table C: Extensive ablation on Kubric4D across different temporal strides. Best and second best results are highlighted.

Region	Feat.	$\delta_{\text{avg}}$	$\delta$ 1	$\delta$ 2	$\delta$ 4	$\delta$ 8	$\delta$ 16
All	$\mathbf{z}_{\text{st}}$	27.33	5.65	12.43	25.37	39.78	53.40
All	$\mathbf{z}_{\text{dyn}}$	29.04	5.76	12.50	26.50	42.81	57.64
		( $+$ 1.71)	( $+$ 0.11)	( $+$ 0.07)	( $+$ 1.13)	( $+$ 3.03)	( $+$ 4.24)
Dyn	$\mathbf{z}_{\text{st}}$	23.45	3.10	9.12	21.14	34.84	49.03
Dyn	$\mathbf{z}_{\text{dyn}}$	25.29	3.12	9.20	22.37	38.05	53.68
		( $+$ 1.84)	( $+$ 0.02)	( $+$ 0.08)	( $+$ 1.23)	( $+$ 3.21)	( $+$ 4.65)

Appendix

Table D: Zero-shot matching on DAVIS. Performance on all threshold are reported. Dynamic latent demonstrates consistently better zero-shot matching accuracy than static latent on all thresholds.

Region	Feat.	$\Delta$ 1		$\Delta$ 2		$\Delta$ 4		$\Delta$ 8
Region	Feat.	AbsRel $\downarrow$	$\delta<1.25\uparrow$	AbsRel $\downarrow$	$\delta<1.25\uparrow$	AbsRel $\downarrow$	$\delta<1.25\uparrow$	AbsRel $\downarrow$	$\delta<1.25\uparrow$
All	$\mathbf{z}_{\text{st}}$	0.1129	0.8550	0.1023	0.8729	0.0944	0.8908	0.0868	0.9081
	$\mathbf{z}_{\text{dyn}}$	0.1037	0.8722	0.0932	0.8937	0.0863	0.9035	0.0751	0.9326
		( $-$ 0.0092)	( $+$ 0.0171)	( $-$ 0.0092)	( $+$ 0.0208)	( $-$ 0.0081)	( $+$ 0.0127)	( $-$ 0.0117)	( $+$ 0.0244)
Dyn	$\mathbf{z}_{\text{st}}$	0.1180	0.8597	0.1194	0.8614	0.1178	0.8699	0.1149	0.8790
	$\mathbf{z}_{\text{dyn}}$	0.1127	0.8747	0.1078	0.8824	0.1050	0.8941	0.0996	0.9099

Appendix

Table E: Video depth probed on Kubric. Performance on all strides are reported. The gain is larger at larger temporal gap.

Region	Feat.	$\Delta$ 1			$\Delta$ 2			$\Delta$ 4			$\Delta$ 8
Region	Feat.	AJ $\uparrow$	$<\delta_{\mathrm{avg}}^{x}\uparrow$	OA $\uparrow$	AJ $\uparrow$	$<\delta_{\mathrm{avg}}^{x}\uparrow$	OA $\uparrow$	AJ $\uparrow$	$<\delta_{\mathrm{avg}}^{x}\uparrow$	OA $\uparrow$	AJ $\uparrow$	$<\delta_{\mathrm{avg}}^{x}\uparrow$	OA $\uparrow$
All	$\mathbf{z}_{\text{st}}$	37.63	52.35	70.40	36.15	51.65	69.20	33.10	49.00	66.42	28.92	45.82	63.42
	$\mathbf{z}_{\text{dyn}}$	40.45	54.84	73.25	39.80	54.76	72.61	36.59	51.73	69.79	32.29	47.60	67.06
		( $+$ 2.82)	( $+$ 2.49)	( $+$ 2.85)	( $+$ 3.64)	( $+$ 3.11)	( $+$ 3.41)	( $+$ 3.49)	( $+$ 2.73)	( $+$ 3.37)	( $+$ 3.38)	( $+$ 1.78)	( $+$ 3.64)
Dyn	$\mathbf{z}_{\text{st}}$	32.26	47.85	66.12	30.83	47.39	64.80	28.57	45.69	62.65	24.84	42.88	59.72
	$\mathbf{z}_{\text{dyn}}$	34.82	49.86	69.23	34.09	49.53	68.65	32.01	47.70	66.64	28.75	44.81	64.38

Appendix

Table F: Point tracking probed on DAVIS. Tracking performance across different temporal strides are reported.

Figure F: Qualitative comparison of point tracking from static and dynamic features. Compared to static features, dynamic features produce more temporally coherent point tracks by better following objects and distinguishing them from the background.

Figure G: Qualitative comparison of depth map prediction from static and dynamic features. Compared to static features, dynamic features produce clearer depth maps, with more coherent depth predictions across objects and the background.

Method	Alignment	ADT			TUM
Method	Alignment	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NeoVerse ( Yang et al., 2026 )	✗	12.217	0.2982	0.6391	14.398	0.4159	0.4587
	✓	16.179	0.4384	0.5296	16.120	0.5266	0.3981
		( $+$ 3.962)	( $+$ 0.1402)	( $-$ 0.1095)	( $+$ 1.721)	( $+$ 0.1107)	( $-$ 0.0606)
C4G ( Kim et al., 2026c )	✗	12.963	0.4487	0.6870	12.378	0.3935	0.5984
	✓	18.900	0.5813	0.5833	17.508	0.5518	0.4951

Appendix

Table G: Test-time pose-alignment results for pose-free methods. Alignment optimizes each target pose against the ground-truth target image for 2,000 steps, and the green row reports its effect. Best , second best , and third best results are highlighted.

Figure H: More qualitative comparisons on NVIDIA across all scenes. Our method shows much better rendering quality than all other methods based on 3D Gaussian.

Figure I: More qualitative comparisons on N3DV across all scenes. Our method shows much better rendering quality than all other methods based on 3D Gaussian.

Figure J: More qualitative comparisons on TUM across all scenes.

Figure K: More qualitative comparisons on DyCheck across all scenes.

Figure L: More qualitative comparisons on ADT across all scenes. Our method shows much better rendering quality than all other methods based on 3D Gaussian. Gaussian-based approaches show severe artifacts such as holes caused by misaligned/mispositioned Gaussians.

Figure M: More qualitative comparisons on Charge across all scenes. Our method shows much better rendering quality than all other methods based on 3D Gaussian. Gaussian-based approaches show severe artifacts such as holes caused by misaligned/mispositioned Gaussians.

Figure N: Qualitative comparisons across view-time grid on NVIDIA. Red outlines indicate context frames. Compared with other methods, our methods show substantially better rendering quality with clear view-time disentanglement.

Figure O: Qualitative comparisons across view-time grid on N3DV. Red outlines indicate context frames. Compared with other methods, our methods show substantially better rendering quality with clear view-time disentanglement.

Figure P: Qualitative comparisons across view-time grid on Charge. Red outlines indicate context frames. Compared with other methods, our methods show substantially better rendering quality with clear view-time disentanglement.

Figure Q: Qualitative results across view-time grid on NVIDIA. Red outlines indicate context frames. Our method successfully synthesizes images at novel view and time, using target conditioning on predicted latent, without relying on explicit geometric primitives.

Figure R: Qualitative results across view-time grid on CamxTime. Red outlines indicate context frames. Our method successfully synthesizes images at novel view and time, using target conditioning on predicted latent, without relying on explicit geometric primitives.

Figure S: Qualitative results across view-time grid on Kubric4D. Red outlines indicate context frames. Our method successfully synthesizes images at novel view and time, using target conditioning on predicted latent, without relying on explicit geometric primitives.

Dynamic Novel View Synthesis

Dynamic novel view synthesis creates time-consistent images or videos from unseen camera positions in scenes that change over time.

See all 41 papers

Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction
May 29, 2026Mungyeom Kim, Minkyeong Jeon, Honggyu An +10Point Tracking 3D Gaussian Splatting
KAIST AI · ETH Zürich, ETH AI Center · Sony AI +1
LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesis
Mar 20, 2026Stanislaw Szymanowicz, Minghao Chen, Jianyuan Wang +2Novel View Synthesis 3D Representation Learning
Visual Geometry Group, University of Oxford · Meta AI
SplatFlux: Feed-Forward 4D Gaussian Reconstruction and Generation from Video Latents
Oct 20, 2026Runmin Zhang, Binglin Ren, Zhu Yu +64D Reconstruction Dynamic Scene Reconstruction
Zhejiang University · Ant Group

May 29, 2026J/K move · Enter open · S save

Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction

Mungyeom Kim, Minkyeong Jeon, Honggyu An, Jaewoo Jung+9

KAIST AI · ETH Zürich, ETH AI Center · Sony AI+1

Point Tracking 3D Gaussian Splatting 4D Reconstruction Dynamic Scene Reconstruction Feed-Forward 3D Reconstruction

Open paper AskPDF arXiv

Dynamic scene reconstruction from monocular video remains a fundamental challenge in computer vision. Existing feed-forward methods predict 3D Gaussians pixel-wise for each frame, suffering from duplicated Gaussians and view-dependent biases that hinder effective learning of scene motion. We present C4G, a feed-forward 4D reconstruction framework built upon a compact set of timestamp-conditioned learnable Gaussian query tokens. Each token aggregates corresponding features across the full temporal context and decodes a 3D Gaussian whose position is modulated by the target timestamp, enabling globally coherent motion modeling without per-scene optimization. To capture fine-grained details, we further introduce a video diffusion model-based rendering enhancement module. Since our framework effectively aggregates features into Gaussians, we extend this capability to feature lifting, producing a 4D feature field that supports point tracking and dynamic scene understanding. C4G achieves strong novel-view synthesis performance using significantly fewer Gaussians and without requiring camera poses, while exhibiting stronger motion modeling and robustness to large temporal gaps.

Abstract

Dynamic Novel View Synthesis

Explore similar work

Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction

LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesis

SplatFlux: Feed-Forward 4D Gaussian Reconstruction and Generation from Video Latents