cs.LGMar 6, 2026

Frequency-Separable Hamiltonian Neural Network for Multi-Timescale Dynamics

Authors: Yaojun Li, Yulong Yang, Christine Allen-Blanchette

Organizations: Princeton University, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

While Hamiltonian mechanics provides a powerful inductive bias for neural networks modeling dynamical systems, Hamiltonian Neural Networks and their variants often fail to capture complex temporal dynamics spanning multiple timescales. This limitation is commonly linked to the spectral bias of deep neural networks, which favors learning low-frequency, slow-varying dynamics. Prior approaches have sought to address this issue through symplectic integration schemes that enforce energy conservation or by incorporating geometric constraints to impose structure on the configuration-space. However, such methods either remain limited in their ability to fully capture multiscale dynamics or require substantial domain specific assumptions. In this work, we exploit the observation that Hamiltonian functions admit decompositions into explicit fast and slow modes and can be reconstructed from these components. We introduce the Frequency-Separable Hamiltonian Neural Network (FS-HNN), which parameterizes the system Hamiltonian using multiple networks, each governed by Hamiltonian dynamics and trained on data sampled at distinct timescales. We further extend this framework to partial differential equations by learning a state- and boundary-conditioned symplectic operators. Empirically, we show that FS-HNN improves long-horizon extrapolation performance on challenging dynamical systems and generalizes across a broad range of ODE and PDE problems.

Figures & tables

Figure 1: Frequency-Separable Hamiltonian Neural Network. Hamiltonian systems that exhibit dynamics with multiple timescales and be decoupled into subsystems each representing one timescale. An observation utrainu_{\mathrm{train}} is subsampled using interval I1I_{1} , I2I_{2} , and I3I_{3} to form subsampled dataset u(k)u^{(k)} . The single scale HNN Mk\mathcal{M}_{k} is trained on subsampled data individually. The multiscale model Mmulti\mathcal{M}_{\mathrm{multi}} combines pretrained single scale HNN trajectories to predict rollout on the original resolution.
Figure 2: Phase space comparison for ODE systems. We show the MSE error and phase space portraits of the predicted rollout of FS-HNN, MLP, HNN ( Greydanus et al., 2019 ) , and SympNet ( Jin et al., 2020 ) for the (Top) ideal pendulum, (Middle) double pendulum, and (Bottom) Fermi-Pasta-Uam-Tsingou systems. FS-HNN achieves significantly lower prediction error compared to baseline networks. Some baselines are omitted from the MSE plot due to excessively large predictive error. Exact trajetocy prediction results can be checked in Figure 6 in Appendix C .
ModelRes.PendulumDoubleFermi-Pasta-
PendulumUlam-Tsingou
MLPLow5.24×10−25.24\times 10^{-2}6.47×1006.47\times 10^{0}1.99×10−11.99\times 10^{-1}
Med1.27×10−31.27\times 10^{-3}1.28×1001.28\times 10^{0}2.39×10−22.39\times 10^{-2}
High1.40×10−41.40\times 10^{-4}4.99×10−14.99\times 10^{-1}1.98×10−21.98\times 10^{-2}
HNNLow2.10×1002.10\times 10^{0}1.60×1001.60\times 10^{0}2.63×10−22.63\times 10^{-2}
Med1.05×10−11.05\times 10^{-1}8.67×10−18.67\times 10^{-1}1.28×10−21.28\times 10^{-2}
Table 1: Rollout accuracy for ODE systems. We report the rollout MSE error of FS-HNN, MLP, HNN ( Greydanus et al., 2019 ) , and SympNet ( Jin et al., 2020 ) on the pendulum, double pendulum, and Fermi-Pasta-Ulam systems. We evaluate single resolution FS-HNN (trained on only low, medium, or high resolution data) as well as the multiscale FS-HNN (trained on all frequencies). Both single resolution and multiscale FS-HNN achieves significantly lower prediction error compared to baseline networks.
Figure 3: Flow field comparison for PDE systems. We report the rollout MSE error of FS-HNN, PHNN ( Eidnes and Lye, 2024 ) , and FNO ( Li et al., 2020 ) on the (Top) SWE with Gaussian pulse initialization, (Middle) Taylor-Green vortex systems, and (Bottom) SWE with random initialization. FS-HNN learns qualitatively similar flow fields and achieves significantly lower prediction error compared to baseline networks. Results are plotted for the potential height(SWE)/pressure distribution(Taylor-Green) channel, results for the remaining states are given in Figure 7 in Appendix D .
ModelRes.SWE PulseSWE RandomTaylor-Green
PHNNHigh8.57×1028.57\times 10^{2}1.10×1001.10\times 10^{0}3.43×10−43.43\times 10^{-4}
FNOHigh5.31×10−35.31\times 10^{-3}2.75×10−22.75\times 10^{-2}3.43×10−43.43\times 10^{-4}
FS-HNNLow1.31×10−21.31\times 10^{-2}2.73×10−22.73\times 10^{-2}4.95×10−74.95\times 10^{-7}
Med3.75×10−33.75\times 10^{-3}8.31×10−48.31\times 10^{-4}1.91×10−71.91\times 10^{-7}
High5.71×10−45.71\times 10^{-4}1.36×10−41.36\times 10^{-4}8.33×10−88.33\times 10^{-8}
Com.3.67×10−4\mathbf{3.67\times 10^{-4}}1.40×10−5\mathbf{1.40\times 10^{-5}}5.01×10−8\mathbf{5.01\times 10^{-8}}
Table 2: Rollout accuracy for PDE systems. We report the rollout MSE error of FS-HNN, PHNN ( Eidnes and Lye, 2024 ) , and FNO ( Li et al., 2020 ) on the SWE under pulse initial condition, SWE under random initial condition, and Taylor-Green vortex. We evaluate single resolution FS-HNN (trained on only low, medium, or high resolution data) as well as the multiscale FS-HNN (trained on all frequencies). Both single resolution and multiscale FS-HNN achieves significantly lower prediction error compared to baseline networks.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Relative energy deviation in ODE settings. The energy is not strictly conserved; instead, it shows a small periodic perturbation. Even so, FS-HNN matches the ground truth most closely.
Figure 5: Relative energy deviation in PDE settings. Due to the dataset setup, energy is not strictly conserved; however, the variation remains small. We therefore report energy changes in relative terms. Overall, FS-HNN matches the ground truth most closely
Figure 6: Exact trajectory comparisons of FS-HNN against benchmark methods. While the fit is not perfect, FS-HNN achieves satisfactory performance relative to existing models.
IntervalPendulumDouble pendulumFermi-PastaPulse_sweRandom_sweTaylor-Green
1dt4.92×10−34.92\times 10^{-3}3.44×10−13.44\times 10^{-1}9.64×10−49.64\times 10^{-4}5.71×10−45.71\times 10^{-4}1.36×10−41.36\times 10^{-4}8.33×10−88.33\times 10^{-8}
2dt2.91×10−12.91\times 10^{-1}4.59×10−14.59\times 10^{-1}3.49×10−33.49\times 10^{-3}3.75×10−33.75\times 10^{-3}8.31×10−48.31\times 10^{-4}1.91×10−71.91\times 10^{-7}
3dt2.06×1002.06\times 10^{0}5.82×1005.82\times 10^{0}2.08×10−32.08\times 10^{-3}1.31×10−21.31\times 10^{-2}2.73×10−22.73\times 10^{-2}4.95×10−74.95\times 10^{-7}
4dt1.91×1001.91\times 10^{0}4.13×1004.13\times 10^{0}2.89×10−22.89\times 10^{-2}4.12×10−34.12\times 10^{-3}2.06×10−22.06\times 10^{-2}2.06×10−52.06\times 10^{-5}
5dt2.10×1002.10\times 10^{0}1.22×1001.22\times 10^{0}3.84×10−23.84\times 10^{-2}1.32×10−21.32\times 10^{-2}3.84×10−23.84\times 10^{-2}4.20×10−54.20\times 10^{-5}
6dt1.99×1001.99\times 10^{0}1.06×1001.06\times 10^{0}6.98×10−26.98\times 10^{-2}2.72×10−22.72\times 10^{-2}3.41×10−23.41\times 10^{-2}3.40×10−53.40\times 10^{-5}
Appendix
Table 3: Rollout errors across different intervals by single models of FS-HNN. Models trained at higher resolution usually achieve better fits, but this advantage diminishes as the interval becomes larger. Therefore, the choice of resolution–interval combinations does not need to be fixed.
 FS-HNN(ODE)FS-HNN(PDE)HNN(Combined)MLPSympNetPHNNFNO
Size3.21×1063.21\times 10^{6}2.04×1062.04\times 10^{6}2.82×1062.82\times 10^{6}1.32×1061.32\times 10^{6}0.76×1060.76\times 10^{6}4.80×1064.80\times 10^{6}4.216×1064.216\times 10^{6}
Appendix
Table 4: Model parameter count, which shows that all models are compared in a relatively fair condition. Some models are not scaled further because, as the networks become deeper and wider, their fitting performance can actually degrade. For these models, we therefore stop increasing the number of parameters
Figure 7: Supplementary experiment results for PDE cases. From top to bottom is ground truth, FS-HNN, PHNN and FNO.
More in this topic

Hamiltonian Neural Networks

Hamiltonian Neural Networks learn energy functions and physical dynamics with Hamilton’s equations for structure-aware simulation and prediction.

See all 30 papers

Explore similar work

CardsList

  1. Symplectic Neural Networks for learning Generalized Hamiltonians

    Jun 25, 2026Harsh Choudhary, Vyacheslav Kungurtsev, Chandan Gupta +2Dynamical SystemsSymplectic RNNs

    Computer Science Department, Indraprasth Institute of Information Technology, New Delhi, India 110020 · Athena Research and Innovation Center, Marousi, Greece 15125 · HSBC Holdings, Singapore

  2. Extrapolating the emergence of Hamiltonian chaos with random-feature Hamiltonian neural networks

    Jul 31, 2026Jaesung ChoiNeural Surrogate ModelingDynamical Systems

    Center for Artificial Intelligence and Natural Sciences, Korea Institute for Advanced Study, Seoul 02455, Korea

  3. Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs

    May 29, 2026Qihong Yang, Qiaolin HeNeural PDE SolversPDE Solving

    School of Mathematics, Sichuan University, city=Chengdu, postcode=610065, country=China