Wind on Trees: Testing Physical Grounding in Dynamic 4D Gaussian Splatting

Weiying Chen; Edmond Lou

cs.CVSep 15, 2026

Wind on Trees: Testing Physical Grounding in Dynamic 4D Gaussian Splatting

Authors: Weiying Chen, Edmond Lou

Organizations: University of Alberta

Abstract

Monocular reconstruction of wind-driven vegetation is severely underconstrained: motion along the viewing direction is largely unobservable, a moving canopy offers few reliable correspondences, and nearly the entire scene is dynamic, providing little static reference. Directly-learned deformation fields in 4D Gaussian Splatting therefore optimize photometric consistency rather than recover the motion that produced it. We replace that field with a physically parameterized deformation prior: one damped harmonic oscillator per rigid part, driven by the observed wind and integrated by differentiable RK4, supervised photometrically alone. To test whether such a prior is physically grounded rather than merely well fit, we build a controlled synthetic testbed of three procedurally generated trees spanning an order of magnitude in skeleton complexity, whose per-part natural frequency follows from its own geometry and whose damping ratio is a fixed constant, both held out of training. On it, we measure held-out views, temporal extrapolation, zero-shot transfer to unseen wind speeds, and recovery of the physical parameters themselves. The prior costs appearance fidelity on in-distribution views and extrapolates markedly better outside the training window and the training wind, while parameter recovery is far weaker than it first appears: frequency recovery survives an untrained null control on only the sparsest of the three trees, and damping is not recovered at all.

Figures & tables

Figure 1: Does a reconstruction recover physics, or replay an animation? Both models are trained on the top row’s wind only. Columns 1–2: branch-tip trajectories tracked through the ground-truth frames, over the 300-frame training window (grey) and continuing into the 50-frame temporal-extrapolation tail (blue). Columns 3–5: a single instant from inside the training window, with the ground-truth canopy outline overlaid on both predictions, so only the wind differs between rows. On the training wind (top), both models sit on the outline: 5.4 px and 2.7 px of canopy-centroid error, and no photometric comparison separates them. Extrapolate the wind to $0.71v_{0}$ (bottom) and the Deformation MLP drifts off the outline (32.6 px) while ours stays on it (0.1 px). Deformation MLP + wind is a Fourier-encoded MLP deformation field over canonical position and time [ 26 ] , additionally conditioned on the preceding 6 s of observed wind subsampled to 48 values, and trained under the same photometric loss as ours.

Figure 2: Overview of Our Method. Our method takes two inputs: a monocular video from a fixed camera, and the wind speed $v(t)$ recorded alongside it. Canonical Gaussians are hard-assigned to $K$ rigid parts by $k$ -means on position (colours are parts), and each part learns $\omega$ , $\zeta$ , a force gain, a swing axis, and a pivot. The wind drives one damped oscillator per part, which differentiable RK4 integrates into a swing angle $\theta(t)$ . Each part is then rigidly rotated by $\theta(t)$ around its pivot, with the rotation applied to Gaussian position and orientation, and rasterized. The wind is a measurement, not supervision: the only training signal is photometric, and nothing supervises $\omega$ or $\zeta$ . Trails are swing paths of tracked canopy points, grey for the observed window and blue for prediction at unseen times, views, and wind speeds. Ground-truth $(\omega,\zeta)$ is held out: it only scores the recovered values, and never enters training.

Figure 3: Qualitative Results across Three Trees and Three Axes of Generalization. Every model is trained on one wind condition, one camera and the first 300 frames. Unseen times are frames beyond that window, unseen views are held-out azimuths, and unseen wind is a condition never trained on ( $0.71v_{0}$ ), evaluated beyond the training window, since at training timestamps a time-indexed field need only replay. Numbers are per-panel PSNR (dB), green where ours is best and orange where a baseline beats it; each panel shows the frame whose ours-minus-baseline gap is closest to that split’s mean.

Figure 4: The collapse is located exactly at the boundary. Per-frame PSNR at the training camera and the training wind, so the only thing that changes at frame 301 is that the timestamps leave the fitted window. On the medium and dense trees, the Deformation MLP tracks ours inside the window, wind-conditioned or not, and then falls away at once past it, below even the undeformed canonical pose, while ours continues. The sparse tree separates nothing here either.

	Sparse (22)		Medium (224)		Dense (562)
	view	time	view	time	view	time
Static pose	24.99	25.16	25.35	24.70	25.08	24.09
Deformation MLP	25.45	25.02	27.88	23.07	26.59	23.19
+ wind	26.39	24.75	27.93	23.53	26.44	23.67
Deformable-3DGS	26.01	24.25	27.66	23.07	26.58	22.65
+ wind	25.49	24.53	27.75	23.56	26.49	23.31

Table 1: Two axes of generalization, all three trees. PSNR (dB). view : held-out azimuths at the training wind and inside the training window, so only the camera is new. time : frames beyond the training window, at the training wind and camera. “Static pose” is the undeformed canonical pose, the reference a deformation model must clear to be doing anything at all. Best per column in bold.

Tree	Model	Median rel. err.	log-log $R^{2}$
Sparse (22 bones)	Untrained null	4.56%	0.981
Sparse (22 bones)	Trained ( $K{=}32$ )	1.80%	0.810
Medium (224 bones)	Untrained null	1.77%	0.996
Medium (224 bones)	Trained ( $K{=}32$ )	1.60%	0.944
Dense (562 bones)	Untrained null	1.56%	0.998
Dense (562 bones)	Trained ( $K{=}32$ )	2.40%	0.843

Table 2: Frequency recovery against an untrained null. The null draws each

\omega_{p}

log-uniformly over the ground-truth range and pushes it through the same solver and the same trajectory matcher as the trained model; entries are the median of 8 draws. Median relative error is taken over matched parts (lower is better);

R^{2}

is that of

\log\hat{\omega}_{p}

against

\log\omega_{p}

. Bold marks the better of each pair on median relative error.

Split	Tree / unseen wind	Static	no wind	+ wind	no wind	+ wind	Ours
			Deformation MLP		Deformable-3DGS
Training timestamps	Sparse (22 bones), $0.84v_{0}$	25.31	25.90	27.04	27.10	27.07	25.84
	Sparse (22 bones), $0.71v_{0}$	25.48	25.82	26.49	26.06	26.13	25.87
	Medium (224 bones), $0.84v_{0}$	27.48	28.95	29.65	29.48	29.70	30.12
	Medium (224 bones), $0.71v_{0}$	28.29	28.07	28.76	28.34	28.69	30.43
	Dense (562 bones), $0.84v_{0}$	26.53	28.27	28.32	27.90	27.79	28.22

Table 3: Zero-shot transfer to unseen wind. PSNR (dB); every model is fit to

v_{0}

only and evaluated on two conditions it never saw. The two blocks differ solely in which frames are scored: training timestamps (top) and frames beyond the training window (bottom). “Static” is the undeformed canonical pose, the reference a deformation model must clear to be doing anything at all. Deformable-3DGS columns are the full-capacity field of [ 26 ] . Best per row in bold.

Ours $-$ (Deformation MLP $+$ wind)	$v_{0}$	$0.84v_{0}$	$0.71v_{0}$
Sparse (22 bones)	$-1.42$	$-1.21$	$-0.61$
Medium (224 bones)	$-0.60$	$+0.46$	$+1.67$
Dense (562 bones)	$-0.03$	$-0.11$	$+0.75$

Table 4: Margin against the wind-conditioned baseline. Each entry is ours minus Deformation MLP

+

wind in PSNR (dB) at training timestamps, on the training condition

v_{0}

and on the two conditions never seen. Positive means ours is ahead.

Explore similar work

Oct 3, 2026cs.CV

Sparse-View 4D Gaussian Splatting via Spatiotemporal Priors and Generative Assistance

We present a 4D Gaussian Splatting framework for the Sparse-View Track of the SIGGRAPH Asia 2026 Volumetric Video Challenge, which requires dynamic scene reconstruction from only six cameras with wide baselines. To achieve robust dynamic reconstruction under such sparse views, our framework integrates three components. (1) Region-adaptive spatial priors: We use foreground masks to guide Gaussian initialization and mask voting to control densification separately for the dynamic foreground and static background. Background geometry is regularized using monocular depth aligned to metric scale. (2) Motion-consistent temporal priors: We provide supervision at intermediate times through frame interpolation and constrain projected Gaussian motion with estimated optical flow. (3) Generative assistance: We place virtual cameras in the widest angular gaps and restore their rendered images using a diffusion-based model conditioned on camera poses. The restored images are iteratively incorporated into training as pseudo-supervision. On the validation set, our framework improves full-frame PSNR from 25.60 dB for the baseline to 29.75 dB. On the official test benchmark, it achieves 30.04 dB full-frame PSNR and 27.88 dB foreground PSNR, ranking first overall in the Sparse-View Track.

Shengqi Wang, Zhengxian Yang, Kaiwen Tian +6

Department of Automation, Tsinghua University, Beijing, China · JD.com, Beijing, China · Beijing MEET YUAN Co., Ltd, Beijing, China

Dynamic Scene Reconstruction 3D Gaussian Splatting

Ask PDF

May 25, 2026cs.CV

Accelerated Spatio-Temporal Gaussian Rendering via Kinematic and Semantic Priors

Reconstructing dynamic 3D scenes from multi-view videos is a foundational task for robotics, AR/VR, and digital twins. While 3D Gaussian Splatting (3DGS) provides state-of-the art scene representation, its dynamic adaptations (4DGS) face significant challenges. Early MLP-based deformation models excel at interpolating within training frames but fail at temporal extrapolation --- important feature for forecasting in predictive systems. To address this, physically grounded approaches have been proposed to enable robust future forecasting, while, concurrently, semantic 4DGS methods have been developed for scene analysis and language-driven prompting. However, there is currently no framework that unites both physical consistency and semantic understanding. In this paper, we propose the first, to the best of our knowledge, unified 4DGS approach that is both physically grounded and semantically aware. Furthermore, to optimize computational efficiency, we introduce a novel rigid-body constraint mechanism. Instead of predicting the physical dynamics for every Gaussian particle, our model predicts the motion of a single anchor Gaussian per semantic object, updating the remaining Gaussians via relative transformations. This architectural design significantly accelerates novel view synthesis compared to an unconstrained baseline, while enabling object-accurate extrapolation and intuitive semantic interaction. Our project page: https://be2rlab.github.io/r5dgs/.

Denis Gridusov, Maxim Popov, Sergey Kolyubin

ITMO University, St. Petersburg, Russia

3D Gaussian Splatting 4D Reconstruction Dynamic Scene Reconstruction Novel View Synthesis Spatiotemporal Forecasting

Ask PDF

Jun 22, 2026cs.CV

Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild

Reconstructing dynamic non-rigid objects from monocular video requires integrating visual cues from direct observations with data-driven priors over geometry and appearance. Prior approaches either learn to directly predict 4D representations from visual input or initialize a 3D representation that is subsequently deformed and refined based on video evidence. However, the former are constrained by the scarcity of 4D training data, while the latter leverage priors only for the initial reconstruction and rely solely on video supervision thereafter; neither handles complex in-the-wild scenarios with large deformations and occlusions well. We present Lift4D, a test-time optimization framework that addresses both limitations. First, we adapt an existing single-view 3D reconstruction model to yield temporally consistent per-frame predictions via causal latent conditioning, providing a coherent initialization for a deformable 3D Gaussian Splatting representation. We then ``sculpt'' this representation to match the input video through an occlusion-aware optimization that faithfully recovers visible surface details while completing unobserved regions using a view-conditioned diffusion prior. We demonstrate that Lift4D clearly improves over prior 4D reconstruction methods, particularly on challenging in-the-wild sequences with severe occlusions and non-rigid motion.

Yehonathan Litman, Xiaoxuan Ma, Manan Shah +4

Carnegie Mellon University

4D Reconstruction Dynamic Scene Reconstruction Deformable Shape Reconstruction

Ask PDF