cs.CVApr 2, 2026
SaveResonance4D: Frequency-Domain Motion Supervision for Preset-Free Physical Parameter Learning in 4D Dynamic Physical Scene Simulation
Organizations: Xidian University, China · Jimei University, China · Sun Yat-sen University, China
Abstract
Physics-driven 4D dynamic simulation from static 3D scenes remains constrained by an overlooked contradiction: reliable motion supervision often relies on online video diffusion or optical-flow pipelines whose computational cost exceeds that of the simulator itself. Existing methods further simplify inverse physical modeling by optimizing only partial material parameters, limiting realism in scenes with complex materials and dynamics. We present Resonance4D, a physics-driven 4D dynamic simulation framework that couples 3D Gaussian Splatting with the Material Point Method through lightweight yet physically expressive supervision. Our key insight is that dynamic consistency can be enforced without dense temporal generation by jointly constraining motion in complementary domains. To this end, we introduce Dual-domain Motion Supervision (DMS), which combines spatial structural consistency for local deformation with frequency-domain spectral consistency for oscillatory and global dynamic patterns, substantially reducing training cost and memory overhead while preserving physically meaningful motion cues. To enable stable full-parameter physical recovery, we further combine zero-shot text-prompted segmentation with simulation-guided initialization to automatically decompose Gaussians into object-part-level regions and support joint optimization of full material parameters. Experiments on both synthetic and real scenes show that Resonance4D achieves strong physical fidelity and motion consistency while reducing peak GPU memory from over 35,GB to around 20,GB, enabling high-fidelity physics-driven 4D simulation on a single consumer-grade GPU.
Figures & tables
Figure 1. Motivation for dual-domain motion analysis and part-level physical modeling. (a) By extracting horizontal scanlines at a fixed height and stacking them over time, an input video is converted into a spatiotemporal slice that explicitly unfolds motion trajectories into a 2D structural pattern. (b) In this representation, oscillation exhibits a clearer periodic trajectory and a more pronounced dominant frequency peak than deformation. (c) Different density settings lead to markedly different motion behaviors: compared with uniform density, a more plausible part-level density distribution produces more natural swaying and restoring motion. The figure contains three panels. Panel (a) shows how a - spatiotemporal slice is formed by extracting a horizontal scanline at a fixed height from each video frame and stacking these scanlines over time, with width on the horizontal axis and time on the vertical axis. Panel (b) shows two examples of - maps and their corresponding frequency spectra: a deformation example with smoother, less regular trajectories and a lower dominant frequency peak, and an oscillation example with a clearer periodic trajectory marked by a red curve and a higher dominant peak. Panel (c) shows two motion sequences generated with different density settings. The top sequence uses uniform density. The bottom sequence uses different densities for the leaf and stem, producing a larger swaying amplitude and a clearer restoring trend over time.
| Method | Constitutive Parameters | Physical Property | ||||
|---|---|---|---|---|---|---|
| Young’s | Poisson’s | Yield | Plastic | Friction | Density | |
| Mod. | Ratio | Stress | Visc. | Angle | ||
| PhysDreamer ( Zhang et al., 2024 ) | ||||||
| DreamPhysics ( Huang et al., 2025 ) | ||||||
| Physics3D ( Liu et al., 2024a ) | ||||||
| PhysFlow ( Liu et al., 2025 ) | ||||||
Table 1. Comparison of learnable physical parameters across different methods. A checkmark indicates that the corresponding parameter is explicitly optimized, while a blank entry indicates that it is not updated.
Figure 2. Resonance4D is a unified framework for physics-driven dynamic simulation from static reconstruction. Its key insight is that effective motion supervision does not require heavy online video priors; lightweight dual-domain spatial-spectral constraints are sufficient to guide physically meaningful dynamics. Guided by this insight, the framework first constructs a reference motion video, then identifies movable regions through automatic part-level assignment, next obtains a reliable starting point via simulation-driven initialization over the feasible physical parameter space, and finally performs part-level physical parameter optimization under Dual-domain Motion Supervision. The figure presents a block diagram of the Resonance4D pipeline. A static reconstructed scene is shown on the left as input. A legend at the top indicates icons for rendering, simulation, and comparison. The middle-left area contains two modules: Reference Video Generation, which converts a motion text prompt into a reference video through an offline video diffusion model, and Part-level Assignment, which uses text prompts, a vision-language model, lifting, and filtering to obtain moving parts. The upper-right module, Simulation-driven Initialization, evaluates multiple candidate physical parameter sets by simulation, rendering, and comparison with the reference video. The lower-right module, Training, runs an MPM simulator on the object, renders the simulated frames, and applies dual-domain motion supervision based on structural consistency and oscillation consistency. Arrows connect the modules to indicate the order of processing and the optimization loop.
| Method | Proc. & Year | Alocasia | Carnations | Hat | Telephone | All | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MS-SSIM | PSNR | LPIPS | Memory | MS-SSIM | PSNR | LPIPS | Memory | MS-SSIM | PSNR | LPIPS | Memory | MS-SSIM | PSNR | LPIPS | Memory | MS-SSIM | PSNR | LPIPS | Memory | ||
| DreamPhysics | AAAI2025 | 0.8143 | 18.52 | 0.3904 | 32.21 | 0.7458 | 17.55 | 0.2199 | 34.12 | 0.5068 | 14.69 | 0.2118 | 33.06 | 0.7886 | 21.18 | 0.5117 | 34.27 | 0.7139 | 17.98 | 0.3334 | 33.42 |
| Physics3D | - | 0.6807 | 16.89 | 0.4236 | 31.53 | 0.6715 | 17.05 | 0.1964 | 33.91 | 0.5057 | 14.75 | 0.1970 | 32.85 | 0.7929 | 20.94 | 0.4011 | 33.94 | 0.6627 | 17.41 | 0.3046 | 33.06 |
| PhysFlow | CVPR2025 | 0.5785 | 15.60 | 0.4874 | 38.38 | 0.6448 | 16.03 | 0.2460 | 38.53 | 0.5091 | 14.76 | 0.2004 | 36.68 | 0.7996 | 21.07 | 0.3943 | 40.48 | 0.6330 | 16.87 | 0.3320 | 38.52 |
| Ours | - | 0.7958 | 18.40 | 0.3958 | 22.32 | 0.7900 | 19.21 | 0.1257 | 22.14 | 0.5109 | 14.80 | 0.1966 | 21.65 | 0.8325 | 21.82 | 0.4246 | 20.97 | 0.7323 | 18.56 | 0.2857 | 21.77 |
Table 2. Quantitative comparison on four scenes from the PhysDreamer dataset. We report video fidelity metrics, including MS-SSIM, PSNR, and LPIPS, together with peak GPU memory usage. The column “all” denotes the average results over all four scenes. Best results are highlighted in bold, respectively.
Figure 3. Visual comparison on four scenes from the PhysDreamer dataset. For each method, we show representative video frames, the motion-region ww-tt visualization, and the corresponding FFT spectrum. The - maps reveal temporal motion patterns along a fixed spatial coordinate, while the FFT plots further characterize their frequency responses. In the carnations scene, our method better preserves the oscillatory motion property, producing clearer periodic patterns and frequency peaks that are closer to the real-world reference. In the telephone scene, our method achieves the closest alignment with the real-world motion in both the spatiotemporal trajectory and the spectral distribution. The figure is organized into an upper group and a lower group, each corresponding to one scene. In each group, five rows are shown for DreamPhysics, Physics3D, PhysFlow, Ours, and RealWorld. For every row, the left section contains a sequence of video frames sampled at multiple time steps. The middle section shows a corresponding - spatiotemporal slice extracted along a fixed spatial coordinate, with horizontal position on the horizontal axis and time on the vertical axis. The right section shows the corresponding FFT spectrum, with frequency on the horizontal axis and amplitude on the vertical axis. The upper group depicts a flower scene, and the lower group depicts a hanging telephone scene.
Figure 4. Qualitative comparison on the bird and toothpaste scenes at consecutive time steps. The bird scene corresponds to relatively stiff material behavior, while toothpaste exhibits softer dynamics. Our method better matches the material-dependent motion in both cases. PAC-NeRF tends to produce overly soft deformation in the stiff bird scene, whereas PhysFlow is overly rigid. In the toothpaste scene, PhysFlow shows an unnatural rebound in shape evolution, which deviates from the ground truth. The figure presents a qualitative comparison on two scenes across consecutive time steps. Four columns are shown for GT, Ours, PhysFlow, and PAC-NeRF. In each column, the upper row group shows a sequence from the bird scene, and the lower row group shows a sequence from the toothpaste scene. Each sequence contains multiple shapes sampled at successive times, arranged from top to bottom. The bird shapes are shown in red, and the toothpaste shapes are shown in green to blue.
| Method | Proc. & Year | CD | HD95 | F-score |
|---|---|---|---|---|
| PAC-NeRF | ICLR 2023 | 0.0074 | 0.1013 | 0.4329 |
| PhysFlow | CVPR 2025 | 0.0085 | 0.1113 | 0.3647 |
| Ours | - | 0.0063 | 0.0914 | 0.4256 |
Table 3. Quantitative comparison on the PAC-NeRF dataset. We report CD, HD95, and F-score. Best results are highlighted in bold.
| Spatial Domain | Frequency Domain | Metrics | |||
|---|---|---|---|---|---|
| L1 | SSIM | FFT | MS-SSIM | PSNR | LPIPS |
| 0.7268 | 18.4173 | 0.2913 | |||
| 0.7284 | 18.4463 | 0.2899 | |||
| 0.7304 | 18.5486 | 0.2885 | |||
| 0.7323 | 18.5570 | 0.2857 | |||
Table 4. Ablation study on dual-domain motion supervision. We compare different combinations of spatial-domain and frequency-domain supervision.
| Method | MS-SSIM | PSNR | LPIPS |
|---|---|---|---|
| GPT Init. | 0.6273 | 16.8943 | 0.3866 |
| GPT Init. + Training | 0.6272 | 16.8653 | 0.3852 |
| Sampling Init. | 0.7296 | 18.4854 | 0.2886 |
| Sampling Init. + Training | 0.7323 | 18.5570 | 0.2857 |
| Avg. sampling init. time : 14.0 min | |||
| Four scenes, 1080P, 32 videos, 16 frames. | |||
Table 5. Ablation study on initialization strategies. We compare GPT-based initialization and simulation-driven initialization, both before and after optimization, and also report the average time cost of simulation-driven initialization.
| MS-SSIM | PSNR | LPIPS | |
|---|---|---|---|
| Particle Update | 0.7241 | 18.2771 | 0.2935 |
| Object Update | 0.7269 | 18.4318 | 0.2909 |
| Part Update | 0.7323 | 18.5570 | 0.2857 |
Table 6. Ablation study on parameter update granularity. We compare particle-level, object-level, and part-level updates.
| MS-SSIM | PSNR | LPIPS | |
|---|---|---|---|
| W/O density Update | 0.7099 | 17.9778 | 0.3020 |
| With density Update | 0.7323 | 18.5570 | 0.2857 |
Table 7. Effect of density optimization in the proposed framework.
More in this topic
Physics-Based Simulation
Physics-based simulation models physical systems with equations and numerical solvers, increasingly augmented by learned models for speed and control.
See all 228 papers