Improving Hamiltonian Learning through Local Energy Decomposition

Yaojun Li; Yulong Yang; Christine Allen-Blanchette

cs.LGNov 2, 2026

Improving Hamiltonian Learning through Local Energy Decomposition

Authors: Yaojun Li, Yulong Yang, Christine Allen-Blanchette

Organizations: Princeton University

Abstract

Hamiltonian neural networks provide a principled framework for learning conservative dynamics. A scalar Hamiltonian is parameterized by a neural network and the vector field is derived using Hamilton's equations. However, preserving Hamiltonian structure does not guarantee that the energy function itself is represented accurately. When modeled by a conventional multilayer perceptron, the Hamiltonian inherits the spectral bias of coordinate-based neural representations, favoring smooth over fine-scale structure. These fine-scale errors can be amplified by differentiation when computing the dynamics. To address this limitation, we introduce Local Multiscale Hamiltonian Neural Network (LM-HNN), which restructures the Hamiltonian according to its physical interaction structure. LM-HNN decomposes the energy into kinetic and potential components and represents them as sums of shared local functions over multiple interaction neighborhoods. This replaces a single high-dimensional approximation problem with lower-dimensional, reusable interaction laws, making fine-scale energy structure easier to resolve while preserving Hamiltonian dynamics. Across low-dimensional, interacting, and fast--slow systems, LM-HNN consistently achieves more accurate Hamiltonian derivatives and long-horizon rollouts than competing structure-preserving models, while substantially extending the effective spectral range of the learned energy. The same structured representation also generalizes more reliably to energy levels not observed during training. These results show that accuracy depends on the Hamiltonian representation as well as its structure, and that incorporating locality, decomposition, and shared interaction laws provides a simple and effective route to more accurate and generalizable Hamiltonian learning.

Figures & tables

Figure 1: Spectral bias in the learned Hamiltonian on the FPUT chain ( $\mathbf{N=8}$ ). (a, b) Residual Hamiltonian spectral power during training on the unperturbed FPUT chain, normalized by the target power at each Fourier wavenumber $k$ . (c, d) Recovery of the added force component during training on the perturbed chain, where $k$ now denotes the prescribed perturbation wavenumber, varied while keeping the force amplitude fixed. At matched parameter counts, LM-HNN reduces higher-wavenumber residuals more rapidly and recovers finer imposed interactions.

Model	Deriv. MSE		Rollout MSE		Deriv. MSE		Rollout MSE
	FPUT				Pendulum
HNN	$(3.6{\scriptstyle\pm 0.3})$	${\scriptstyle\times 10^{-2}}$	$(2.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$(1.9{\scriptstyle\pm 0.7})$	${\scriptstyle\times 10^{-6}}$	$(2.7{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-5}}$
Fourier ( $\sigma$ =0.5)	$(1.8{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-1}}$	$(4.1{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$(1.2{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-5}}$	$(3.0{\scriptstyle\pm 3.2})$	${\scriptstyle\times 10^{-2}}$
Fourier ( $\sigma$ =2)	$(1.8{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$(3.7{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{0}}$	$(3.7{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-4}}$	$(9.3{\scriptstyle\pm 3.7})$	${\scriptstyle\times 10^{-2}}$
SIREN ( $\omega_{0}$ =10)	$(8.1{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$(3.3{\scriptstyle\pm 0.3})$	${\scriptstyle\times 10^{0}}$	$(1.9{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-7}}$	$(3.3{\scriptstyle\pm 2.8})$	${\scriptstyle\times 10^{-4}}$
SIREN ( $\omega_{0}$ =30)	$(3.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$(5.4{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{0}}$	$(1.6{\scriptstyle\pm 1.1})$	${\scriptstyle\times 10^{-5}}$	$(1.4{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{0}}$

Table 1: Baselines for mitigating spectral bias. Derivative and rollout MSE for FPUT and pendulum for baseline mitigation techniques applied to HNN at matched parameter count. The structural approach of LM-HNN outperforms both Fourier features and sinusoidal activations.

System	HNN		SympNet		FS-HNN		LM-HNN
Pendulum	$(1.3{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{0}}$	$(1.0{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{0}}$	$(6.1{\scriptstyle\pm 1.2})$	${\scriptstyle\times 10^{-1}}$	$\bm{(2.7{\scriptstyle\pm 2.4})}$	$\bm{{\scriptstyle\times 10^{-8}}}$
Double Pendulum	$(1.7{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{0}}$	$(3.5{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{0}}$	$(2.2{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{0}}$	$\bm{(8.1{\scriptstyle\pm 5.3})}$	$\bm{{\scriptstyle\times 10^{-6}}}$
FPUT	$(1.8{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$(1.8{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$(2.2{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$\bm{(6.5{\scriptstyle\pm 1.1})}$	$\bm{{\scriptstyle\times 10^{-2}}}$
FPUT-Toda	$(1.7{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{0}}$	$(1.9{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{0}}$	$(2.3{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{0}}$	$\bm{(1.3{\scriptstyle\pm 0.4})}$	$\bm{{\scriptstyle\times 10^{-2}}}$
FPUT-Morse	$(1.2{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$(1.5{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{0}}$	$(1.9{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$\bm{(4.8{\scriptstyle\pm 1.5})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
Fast-Slow	$(4.8{\scriptstyle\pm 0.8})$	${\scriptstyle\times 10^{-1}}$	$(1.9{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{0}}$	$(2.7{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{0}}$	$\bm{(1.4{\scriptstyle\pm 0.4})}$	$\bm{{\scriptstyle\times 10^{-1}}}$

Table 2: Normalized rollout MSE at unseen energy levels. The FPUT chains are trained on

E/N\in[0.05,2.0]

and tested at

2.5

and

3.0

per site; the pendulum is trained on

E\in[0.05,4]

and tested at

6

and

8

; and the double pendulum is trained on

E\in[0.5,5]

and tested at

7

and

9

. The fast–slow row is a scaled-initial-condition test: the test states are multiplied by

1.5

, which places 9 of the 30 above the largest training energy of

2.48

(Appendix A.2 ). Baseline methods exhibit increased rollout error at unseen energy levels whilst LM-HNN maintains low error.

Figure 2: Vector-field accuracy beyond the training-energy range. Vector-field error $\lVert J\nabla H_{\theta}(\bm{z})-\dot{\bm{z}}\rVert_{2}$ , shown as a local root-mean-square for (a, b) the pendulum phase plane and (c, d) the FPUT chain ( $N=8$ ) on the slice $\bm{q}=a\mathbf{u}$ , $\bm{p}=b\mathbf{v}$ along fixed random unit directions. Blue curves mark the maximum training energy, and each pair shares a logarithmic color scale. LM-HNN maintains lower errors beyond this boundary, supporting generalization to unseen energy levels.

System	HNN		SympNet		FS-HNN		LM-HNN (ours)
Rollout MSE
Pendulum	$(2.7{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-5}}$	$(2.9{\scriptstyle\pm 1.2})$	${\scriptstyle\times 10^{-2}}$	$(6.2{\scriptstyle\pm 2.8})$	${\scriptstyle\times 10^{-6}}$	$\bm{(4.1{\scriptstyle\pm 2.5})}$	$\bm{{\scriptstyle\times 10^{-8}}}$
Double Pendulum	$(4.3{\scriptstyle\pm 5.0})$	${\scriptstyle\times 10^{-2}}$	$(1.7{\scriptstyle\pm 0.3})$	${\scriptstyle\times 10^{0}}$	$(1.9{\scriptstyle\pm 0.9})$	${\scriptstyle\times 10^{-2}}$	$\bm{(3.2{\scriptstyle\pm 3.2})}$	$\bm{{\scriptstyle\times 10^{-6}}}$
FPUT	$(2.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$(6.2{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-2}}$	$(1.2{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-2}}$	$\bm{(4.9{\scriptstyle\pm 0.6})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
FPUT-Toda	$(7.0{\scriptstyle\pm 0.9})$	${\scriptstyle\times 10^{-1}}$	$(1.7{\scriptstyle\pm 0.7})$	${\scriptstyle\times 10^{-1}}$	$(2.6{\scriptstyle\pm 0.8})$	${\scriptstyle\times 10^{-1}}$	$\bm{(6.7{\scriptstyle\pm 4.7})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
FPUT-Morse	$(5.1{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-1}}$	$(1.3{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$(2.1{\scriptstyle\pm 0.3})$	${\scriptstyle\times 10^{-1}}$	$\bm{(1.5{\scriptstyle\pm 0.4})}$	$\bm{{\scriptstyle\times 10^{-3}}}$

Table 3: Rollout and derivative MSE across six conservative systems. SympNet learns a discrete symplectic map and therefore does not provide a continuous vector field for the derivative comparison. LM-HNN achieves the lowest rollout and derivative error on every system, showing consistent improvement across low-dimensional, lattice, and fast–slow dynamics.

Ablation Variant	Param.	FPUT		FPUT-Toda
LM-HNN	26,892	$(4.9{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-3}}$	$\mathbf{(6.7{\scriptstyle\pm 4.7})}$	$\mathbf{{\scriptstyle\times 10^{-3}}}$
w/o kinetic–potential split	26,736	$(3.8{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-2}}$	$(7.1{\scriptstyle\pm 1.2})$	${\scriptstyle\times 10^{-2}}$
w/o locality	26,932	$(5.5{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-2}}$	$(3.8{\scriptstyle\pm 1.0})$	${\scriptstyle\times 10^{-1}}$
w/o weight sharing	228,537	$\mathbf{(3.5{\scriptstyle\pm 0.8})}$	$\mathbf{{\scriptstyle\times 10^{-3}}}$	$(1.9{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-2}}$

Table 4: Structural ablations of LM-HNN. Normalized rollout MSE on the FPUT and Toda chains (

N=8

). Locality and kinetic–potential decomposition improve accuracy, while weight sharing reduces model size without a consistent accuracy penalty. Full metrics are reported in Appendix B .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

FPUT
Variant	Params	Deriv. MSE		Rollout MSE		Peak err.
LM-HNN, $\mathcal{R}=\{0,1,2\}$ (full)	26,892	$(5.6{\scriptstyle\pm 0.9})$	${\scriptstyle\times 10^{-4}}$	$(4.9{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-3}}$	$0.83$	${\scriptstyle\pm 0.06\%}$
single radius $\mathcal{R}=\{0\}$	8,708	$\bm{(2.6{\scriptstyle\pm 0.4})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(2.5{\scriptstyle\pm 0.8})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{0.17}$	$\bm{{\scriptstyle\pm 0.04\%}}$
single radius $\mathcal{R}=\{2\}$	9,220	$(1.5{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-3}}$	$(2.0{\scriptstyle\pm 0.8})$	${\scriptstyle\times 10^{-2}}$	$1.65$	${\scriptstyle\pm 0.30\%}$
no kinetic–potential split	26,736	$(2.1{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-3}}$	$(3.8{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-2}}$	$1.58$	${\scriptstyle\pm 0.14\%}$
no locality (global $T$ , $V$ networks)	26,932	$(4.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-3}}$	$(5.5{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-2}}$	$3.00$	${\scriptstyle\pm 0.14\%}$

Appendix

Table 5: Ablation of the LM-HNN structure. We perform ablation on the FPUT and FPUT-Toda chains (

N=8

). Removing locality causes the largest degradation, followed by removing the kinetic–potential decomposition. Removing weight sharing increases the parameter count substantially without providing a consistent accuracy gain. Single-radius variants are included here for completeness and studied in more detail in Table 6 .

System	Radii $\mathcal{R}$	Params	Deriv. MSE		Rollout MSE		Peak err.
Nearest-neighbor	$\{0\}$	8,708	$\bm{(2.6{\scriptstyle\pm 0.4})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(2.5{\scriptstyle\pm 0.7})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{0.17}$	$\bm{{\scriptstyle\pm 0.03\%}}$
chain	$\{1\}$	8,964	$(1.0{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-3}}$	$(4.1{\scriptstyle\pm 1.8})$	${\scriptstyle\times 10^{-3}}$	$0.72$	${\scriptstyle\pm 0.20\%}$
	$\{2\}$	9,220	$(1.5{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-3}}$	$(2.0{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-2}}$	$1.65$	${\scriptstyle\pm 0.24\%}$
	$\{0,1,2\}$	26,892	$(5.6{\scriptstyle\pm 0.7})$	${\scriptstyle\times 10^{-4}}$	$(4.9{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-3}}$	$0.83$	${\scriptstyle\pm 0.05\%}$
	HNN	26,971	$(6.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-3}}$	$(2.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$4.86$	${\scriptstyle\pm 0.11\%}$
Two-scale	$\{0\}$	8,708	$(5.0{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-2}}$	$(8.7{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-1}}$	$9.56$	${\scriptstyle\pm 0.01\%}$

Appendix

Table 6: Ablation of the receptive-field. We perform ablation on two chains with different interaction ranges (

N=8

\alpha=0.25

\beta=0.10

): the nearest-neighbor FPUT chain, and the two-scale chain of Eq. ( 30 ). The best single radius changes between the two systems, and the radius that wins on the nearest-neighbor chain,

\mathcal{R}=\{0\}

, is the one that cannot represent the second-neighbor interaction at all. The multiscale set

\mathcal{R}=\{0,1,2\}

is close to the best choice on both chains and attains the lowest derivative error of any model on the two-scale chain.

Kinetic coupling	Kinetic head	Deriv. MSE		Rollout MSE		Peak err.
Standard FPUT	quadratic, diagonal	$\bm{(5.6{\scriptstyle\pm 0.9})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(4.9{\scriptstyle\pm 0.6})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$\bm{0.83}$	$\bm{{\scriptstyle\pm 0.06\%}}$
( $A=I$ )	local sum, $\mathcal{R}=\{0\}$	$(6.7{\scriptstyle\pm 0.8})$	${\scriptstyle\times 10^{-4}}$	$(1.0{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-2}}$	$1.04$	${\scriptstyle\pm 0.12\%}$
	local sum, $\mathcal{R}=\{0,1,2\}$	$(6.5{\scriptstyle\pm 0.9})$	${\scriptstyle\times 10^{-4}}$	$(1.1{\scriptstyle\pm 0.3})$	${\scriptstyle\times 10^{-2}}$	$0.95$	${\scriptstyle\pm 0.10\%}$
Tridiagonal	quadratic, diagonal	$(9.0{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-2}}$	$(1.6{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{0}}$	$16.11$	${\scriptstyle\pm 0.01\%}$
coupling	local sum, $\mathcal{R}=\{0\}$	$(9.0{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-2}}$	$(1.6{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{0}}$	$16.23$	${\scriptstyle\pm 0.04\%}$
	local sum, $\mathcal{R}=\{0,1,2\}$	$\bm{(6.6{\scriptstyle\pm 0.8})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(1.5{\scriptstyle\pm 0.5})}$	$\bm{{\scriptstyle\times 10^{-2}}}$	$\bm{1.42}$	$\bm{{\scriptstyle\pm 0.26\%}}$

Appendix

Table 7: Kinetic coupling and kinetic-head design for

N=8

. Systems include

A=I

for the standard FPUT chain; tridiagonal coupling

A=I+0.3\,\mathrm{tridiag}(1)

, which introduces

p_{i}p_{i+1}

terms; and global coupling

A=I+\tfrac{1}{N}\mathbf{1}\mathbf{1}^{\top}

. We compare the diagonal quadratic form of Eq. ( 10 ), the local-summation form of Eq. ( 12 ), and a dense Cholesky form

\tfrac{1}{2}\bm{p}^{\top}L_{\theta}(\bm{q})L_{\theta}(\bm{q})^{\top}\bm{p}

. The results show that the kinetic representation should follow the coupling structure: diagonal for uncoupled momenta, local for finite-range coupling, and dense for global coupling.

$N$	HNN	LM-HNN	HNN		LM-HNN		HNN		LM-HNN
	Params		Derivative MSE				Rollout MSE
4	27,031	26,892	$(9.7{\scriptstyle\pm 0.8})$	${\scriptstyle\times 10^{-4}}$	$\bm{(1.5{\scriptstyle\pm 0.2})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(3.2{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-2}}$	$\bm{(9.4{\scriptstyle\pm 0.6})}$	$\bm{{\scriptstyle\times 10^{-4}}}$
6	27,005	26,892	$(3.3{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-3}}$	$\bm{(5.0{\scriptstyle\pm 0.4})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(9.6{\scriptstyle\pm 2.6})$	${\scriptstyle\times 10^{-2}}$	$\bm{(2.6{\scriptstyle\pm 0.4})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
8	26,971	26,892	$(6.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-3}}$	$\bm{(5.6{\scriptstyle\pm 0.9})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(2.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$\bm{(4.9{\scriptstyle\pm 0.6})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
10	26,929	26,892	$(8.8{\scriptstyle\pm 1.1})$	${\scriptstyle\times 10^{-3}}$	$\bm{(4.3{\scriptstyle\pm 0.8})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(2.3{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-1}}$	$\bm{(5.1{\scriptstyle\pm 1.6})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
12	26,879	26,892	$(1.1{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-2}}$	$\bm{(7.0{\scriptstyle\pm 1.7})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(5.2{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$\bm{(4.6{\scriptstyle\pm 1.1})}$	$\bm{{\scriptstyle\times 10^{-3}}}$

Appendix

Table 8: Scaling with chain length

N

at matched parameter count. We scale the chain length on the FPUT system with

\alpha=0.25

and

\beta=0.10

. From

N=4

N=16

, the rollout error of LM-HNN remains between approximately

9\times 10^{-4}

and

6\times 10^{-3}

, while that of the conventional HNN grows from

3\times 10^{-2}

5\times 10^{-1}

	Deriv. MSE				Rollout MSE				Peak err. (%)
$(\alpha,\beta)$	HNN		LM-HNN		HNN		LM-HNN		HNN		LM-HNN
(0.25, 0.10)	$(6.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-3}}$	$\bm{(5.6{\scriptstyle\pm 0.9})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(2.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$\bm{(4.9{\scriptstyle\pm 0.6})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$4.86$	${\scriptstyle\pm 0.13}$	$\bm{0.83}$	$\bm{{\scriptstyle\pm 0.06}}$
(0.50, 0.10)	$(6.6{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-3}}$	$\bm{(8.7{\scriptstyle\pm 2.5})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(3.7{\scriptstyle\pm 0.7})$	${\scriptstyle\times 10^{-1}}$	$\bm{(1.8{\scriptstyle\pm 1.0})}$	$\bm{{\scriptstyle\times 10^{-2}}}$	$6.03$	${\scriptstyle\pm 0.54}$	$\bm{0.96}$	$\bm{{\scriptstyle\pm 0.10}}$
(1.00, 0.30)	$(1.8{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-2}}$	$\bm{(1.2{\scriptstyle\pm 0.2})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$(8.2{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-1}}$	$\bm{(2.2{\scriptstyle\pm 0.8})}$	$\bm{{\scriptstyle\times 10^{-2}}}$	$9.68$	${\scriptstyle\pm 0.88}$	$\bm{1.24}$	$\bm{{\scriptstyle\pm 0.12}}$
(0.25, 0.30)	$(2.3{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-2}}$	$\bm{(9.0{\scriptstyle\pm 1.6})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(2.9{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-1}}$	$\bm{(6.1{\scriptstyle\pm 1.2})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$5.88$	${\scriptstyle\pm 0.35}$	$\bm{0.93}$	$\bm{{\scriptstyle\pm 0.06}}$
(0.00, 0.50)	$(4.6{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-2}}$	$\bm{(7.5{\scriptstyle\pm 0.8})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$(4.6{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$\bm{(3.9{\scriptstyle\pm 1.2})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$8.29$	${\scriptstyle\pm 0.38}$	$\bm{0.76}$	$\bm{{\scriptstyle\pm 0.11}}$

Appendix

Table 9: Parameter matched comparison of LM-HNN and HNN. FPUT chain (

N=8

) across elastic parameters

(\alpha,\beta)

, comparing the parameter-matched HNN and LM-HNN. As the interaction becomes more nonlinear, the conventional HNN degrades substantially, while LM-HNN remains comparatively stable; its peak error stays between approximately

0.8\%

and

1.2\%

across the sweep.

System	Model	$\sigma_{H}$		$k_{\mathrm{eff}}$	$\\|\nabla\delta H\\|_{\mathrm{rms}}$
Pendulum	HNN	$(4.5{\scriptstyle\pm 0.8})$	${\scriptstyle\times 10^{-4}}$	$4.1$	$(1.9{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-3}}$
	FS-HNN	$(1.3{\scriptstyle\pm 0.9})$	${\scriptstyle\times 10^{-3}}$	$2.9$	$(3.0{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-3}}$
	LM-HNN	$\bm{(2.7{\scriptstyle\pm 0.9})}$	$\bm{{\scriptstyle\times 10^{-5}}}$	$\bm{4.2}$	$\bm{(1.1{\scriptstyle\pm 0.5})}$	$\bm{{\scriptstyle\times 10^{-4}}}$
Double Pendulum	HNN	$(1.5{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-2}}$	$4.8$	$(7.1{\scriptstyle\pm 0.7})$	${\scriptstyle\times 10^{-2}}$
	FS-HNN	$(8.4{\scriptstyle\pm 1.5})$	${\scriptstyle\times 10^{-2}}$	$3.6$	$(2.9{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$
	LM-HNN	$\bm{(3.8{\scriptstyle\pm 1.5})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{9.1}$	$\bm{(3.4{\scriptstyle\pm 1.2})}$	$\bm{{\scriptstyle\times 10^{-3}}}$

Appendix

Table 10: Spectral diagnostics on validation states. RMS Hamiltonian residual

\sigma_{H}

, effective residual wavenumber

k_{\mathrm{eff}}

, and RMS vector-field error

\|\nabla\delta H\|_{\mathrm{rms}}

for HNN, FS-HNN, and LM-HNN. LM-HNN has the smallest Hamiltonian residual and vector-field error on every system. Its remaining residual can have a larger effective wavenumber because the lower-frequency error has already been substantially reduced. The diagnostics follow the Fourier-domain analysis of spectral bias in Rahaman et al. (2019) and the frequency principle of Xu et al. (2020) , which examine the residual of a coordinate network by wavenumber over its input domain and across training;

k_{\mathrm{eff}}

is defined in Equation 38 from Parseval’s identity, and the link between the wavenumber content of the residual and the vector-field error follows the kernel analysis of Wang et al. (2022) .

Figure 3: Residual spectra along probe lines. Left: the Hamiltonian residual

\delta H

along a line obtained by displacing one site, for HNN and LM-HNN on the standard (a) FPUT chain and the (b) perturbed FPUT chain with

k=4

. Middle: power spectra of the target Hamiltonian and the residual of each model. Right: power spectrum of the corresponding vector-field error (solid) together with

k^{2}

times the Hamiltonian-residual spectrum (dotted). Differentiation amplifies higher-wavenumber residuals, while LM-HNN reduces the residual over the spectral range occupied by the target.

Perturbation wavenumber $k$	0		1		2
HNN derivative MSE	$(6.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-3}}$	$(3.3{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-3}}$	$(1.1{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-2}}$
LM-HNN derivative MSE	$\bm{(5.6{\scriptstyle\pm 0.9})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(4.9{\scriptstyle\pm 1.2})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(2.1{\scriptstyle\pm 0.5})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
HNN rollout MSE	$(2.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$(1.2{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$(2.0{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-1}}$
LM-HNN rollout MSE	$\bm{(4.9{\scriptstyle\pm 0.6})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$\bm{(5.0{\scriptstyle\pm 1.1})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$\bm{(1.2{\scriptstyle\pm 0.3})}$	$\bm{{\scriptstyle\times 10^{-2}}}$
Perturbation ignored (deriv. MSE)	–		$1.1$	${\scriptstyle\times 10^{-1}}$	$1.2$	${\scriptstyle\times 10^{-1}}$
HNN force recovered	–		$97\%$		$91\%$

Appendix

Table 11: Spectral-bias sweep on the perturbed FPUT chain (

N=8

). We report the derivative and rollout errors of the HNN and LM-HNN against the perturbation wavenumber

k

, and the fraction of the perturbation force recovered,

1-\mathrm{MSE}/\mathrm{MSE}_{\mathrm{ignored}}

. The HNN loses the perturbation between

k=2

and

k=4

; LM-HNN recovers it up to

k=6

$k$	HNN		SympNet		FS-HNN		LM-HNN (ours)
0	$(2.0{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$(6.2{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-2}}$	$(1.2{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-2}}$	$\bm{(4.9{\scriptstyle\pm 0.6})}$	$\bm{{\scriptstyle\times 10^{-3}}}$
$2$	$(2.0{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-1}}$	$(6.3{\scriptstyle\pm 1.0})$	${\scriptstyle\times 10^{-2}}$	$(1.7{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$\bm{(1.2{\scriptstyle\pm 0.3})}$	$\bm{{\scriptstyle\times 10^{-2}}}$
$4$	$(1.0{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{0}}$	$(1.2{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$(6.7{\scriptstyle\pm 2.5})$	${\scriptstyle\times 10^{-1}}$	$\bm{(1.4{\scriptstyle\pm 0.7})}$	$\bm{{\scriptstyle\times 10^{-2}}}$
$8$	$(8.5{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$(8.3{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-1}}$	$(9.8{\scriptstyle\pm 0.3})$	${\scriptstyle\times 10^{-1}}$	$\bm{(5.0{\scriptstyle\pm 0.1})}$	$\bm{{\scriptstyle\times 10^{-1}}}$
$16$	$(4.2{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-1}}$	$(3.4{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-1}}$	$(4.9{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-1}}$	$\bm{(1.8{\scriptstyle\pm 0.0})}$	$\bm{{\scriptstyle\times 10^{-1}}}$

Appendix

Table 12: Rollout MSE on the perturbed FPUT (

N=8

) with increasing

k

. LM-HNN achieves the lowest error at every tested wavenumber. At

k=4

, its rollout error remains below that of every competing baseline on the plain FPUT chain (

k=0

), demonstrating a substantially extended effective spectral range.

$k$	HNN		SympNet		FS-HNN		LM-HNN
0	$4.86$	${\scriptstyle\pm 0.13\%}$	$2.12$	${\scriptstyle\pm 0.11\%}$	$2.99$	${\scriptstyle\pm 0.85\%}$	$\bm{0.83}$	$\bm{{\scriptstyle\pm 0.06\%}}$
$2$	$5.16$	${\scriptstyle\pm 0.56\%}$	$1.55$	${\scriptstyle\pm 0.10\%}$	$4.08$	${\scriptstyle\pm 1.18\%}$	$\bm{1.16}$	$\bm{{\scriptstyle\pm 0.12\%}}$
$4$	$11.37$	${\scriptstyle\pm 0.27\%}$	$3.16$	${\scriptstyle\pm 0.87\%}$	$9.22$	${\scriptstyle\pm 1.19\%}$	$\bm{0.69}$	$\bm{{\scriptstyle\pm 0.20\%}}$
$8$	$9.99$	${\scriptstyle\pm 0.53\%}$	$9.91$	${\scriptstyle\pm 0.83\%}$	$10.82$	${\scriptstyle\pm 0.37\%}$	$\bm{7.16}$	$\bm{{\scriptstyle\pm 0.13\%}}$
$16$	$7.58$	${\scriptstyle\pm 0.48\%}$	$6.85$	${\scriptstyle\pm 0.59\%}$	$8.06$	${\scriptstyle\pm 0.61\%}$	$\bm{4.77}$	$\bm{{\scriptstyle\pm 0.05\%}}$

Appendix

Table 13: Peak error on the perturbed FPUT (

N=8

) with variable perturbation wavenumber

k

. LM-HNN achieves the lowest peak error at every tested wavenumber and remains below

1.2\%

for

k\leq 4

Figure 4: Rollouts from unseen initial conditions. (a) FPUT chain (

N=8

), one position coordinate, one momentum coordinate, and the normalized full-state error over 240 steps. (b) fast–slow oscillator (

\varepsilon=0.01

), the slow coordinate

q_{2}

over the full horizon, the fast coordinate

q_{1}

over the last two fast periods, and the normalized full-state error over 1000 steps. LM-HNN stays on the reference trajectory of the chain while the baselines accumulate phase and amplitude errors; on the fast–slow oscillator it has the lowest error over the whole horizon, following the slow drift most closely and keeping the fast phase, whereas the HNN drifts below the slow orbit in the middle of the horizon and SympNet and FS-HNN lose the fast phase.

System	HNN	SympNet	FS-HNN	LM-HNN (ours)
Pendulum	8,828	8,832	8,072	8,834
Double pendulum	17,803	17,920	17,468	17,798
FPUT	26,971	26,880	26,468	26,892
FPUT-Toda	26,971	26,880	26,468	26,892
FPUT-Morse	26,971	26,880	26,468	26,892
Fast–slow	22,041	22,016	21,488	21,957

Appendix

Table 14: Parameter counts of LM-HNN and baselines. The models are chosen to use comparable parameter budgets so that the accuracy comparison primarily reflects differences in representation rather than raw model capacity.

System	Model	$\sigma_{H}$		$\\|\nabla\delta H\\|_{\mathrm{rms}}$		$k_{\mathrm{eff}}$
Pendulum	HNN	$(4.5{\scriptstyle\pm 0.8})$	${\scriptstyle\times 10^{-4}}$	$(1.9{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-3}}$	$4.1$	${\scriptstyle\pm 0.1}$
	LM-HNN	$\bm{(2.7{\scriptstyle\pm 0.9})}$	$\bm{{\scriptstyle\times 10^{-5}}}$	$\bm{(1.1{\scriptstyle\pm 0.5})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$4.2$	${\scriptstyle\pm 0.3}$
Double pendulum	HNN	$(1.5{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-2}}$	$(7.1{\scriptstyle\pm 0.7})$	${\scriptstyle\times 10^{-2}}$	$4.8$	${\scriptstyle\pm 0.4}$
	LM-HNN	$\bm{(3.8{\scriptstyle\pm 1.5})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(3.4{\scriptstyle\pm 1.2})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$9.1$	${\scriptstyle\pm 0.4}$
FPUT ( $N{=}8$ )	HNN	$(1.7{\scriptstyle\pm 0.1})$	${\scriptstyle\times 10^{-1}}$	$(3.1{\scriptstyle\pm 0.0})$	${\scriptstyle\times 10^{-1}}$	$1.8$	${\scriptstyle\pm 0.1}$
	LM-HNN	$\bm{(4.4{\scriptstyle\pm 0.0})}$	$\bm{{\scriptstyle\times 10^{-2}}}$	$\bm{(9.4{\scriptstyle\pm 0.7})}$	$\bm{{\scriptstyle\times 10^{-2}}}$	$2.1$	${\scriptstyle\pm 0.2}$

Appendix

Table 15: Hamiltonian and spectral diagnostics and rollout errors from unseen initial conditions inside and above the training energy range. LM-HNN has smaller Hamiltonian residual, vector-field error, and rollout error across the tested systems. On several systems, much of its remaining residual power lies beyond the

k_{99}

bandwidth containing

99\%

of the target Hamiltonian power.

Model	Params	200 steps		1000 steps		200 steps		1000 steps
		full state				slow pair
HNN	22,041	$(2.2{\scriptstyle\pm 2.6})$	${\scriptstyle\times 10^{-3}}$	$(8.7{\scriptstyle\pm 3.7})$	${\scriptstyle\times 10^{-3}}$	$(1.4{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-1}}$	$(1.6{\scriptstyle\pm 0.2})$	${\scriptstyle\times 10^{-1}}$
SympNet	22,016	$(1.2{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{-1}}$	$(1.3{\scriptstyle\pm 0.6})$	${\scriptstyle\times 10^{0}}$	$(5.8{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{0}}$	$(7.4{\scriptstyle\pm 0.7})$	${\scriptstyle\times 10^{0}}$
FS-HNN	21,488	$(1.7{\scriptstyle\pm 0.4})$	${\scriptstyle\times 10^{-3}}$	$(2.1{\scriptstyle\pm 0.9})$	${\scriptstyle\times 10^{-3}}$	$(1.9{\scriptstyle\pm 0.5})$	${\scriptstyle\times 10^{-1}}$	$(3.0{\scriptstyle\pm 1.4})$	${\scriptstyle\times 10^{-1}}$
LM-HNN	21,957	$\bm{(2.3{\scriptstyle\pm 1.6})}$	$\bm{{\scriptstyle\times 10^{-4}}}$	$\bm{(1.8{\scriptstyle\pm 0.5})}$	$\bm{{\scriptstyle\times 10^{-3}}}$	$\bm{(2.8{\scriptstyle\pm 0.5})}$	$\bm{{\scriptstyle\times 10^{-2}}}$	$\bm{(4.1{\scriptstyle\pm 1.1})}$	$\bm{{\scriptstyle\times 10^{-2}}}$

Appendix

Table 16: Fast–slow oscillator (

\varepsilon=0.01

) rollout error. We report the rollout MSE over the 30 unseen test initial conditions after 200 and 1000 steps with

\Delta t=0.1

, reported for the full state and for the slow pair

(q_{2},p_{2})

. LM-HNN achieves the lowest error in every column, improving both the complete dynamics and the weak slow component.

Hamiltonian Learning

See all 8 papers

Mar 6, 2026

Frequency-Separable Hamiltonian Neural Network for Multi-Timescale Dynamics

While Hamiltonian mechanics provides a powerful inductive bias for neural networks modeling dynamical systems, Hamiltonian Neural Networks and their variants often fail to capture complex temporal dynamics spanning multiple timescales. This limitation is commonly linked to the spectral bias of deep neural networks, which favors learning low-frequency, slow-varying dynamics. Prior approaches have sought to address this issue through symplectic integration schemes that enforce energy conservation or by incorporating geometric constraints to impose structure on the configuration-space. However, such methods either remain limited in their ability to fully capture multiscale dynamics or require substantial domain specific assumptions. In this work, we exploit the observation that Hamiltonian functions admit decompositions into explicit fast and slow modes and can be reconstructed from these components. We introduce the Frequency-Separable Hamiltonian Neural Network (FS-HNN), which parameterizes the system Hamiltonian using multiple networks, each governed by Hamiltonian dynamics and trained on data sampled at distinct timescales. We further extend this framework to partial differential equations by learning a state- and boundary-conditioned symplectic operators. Empirically, we show that FS-HNN improves long-horizon extrapolation performance on challenging dynamical systems and generalizes across a broad range of ODE and PDE problems.

Yaojun Li, Yulong Yang, Christine Allen-Blanchette

Princeton University, USA

Hamiltonian Neural Networks Dynamical Systems Neural PDE Solvers

Ask PDF

Oct 31, 2026

Learning momenta for Hamiltonian-based models

There has been increasing interest in methodologies that incorporate physics priors into neural network architectures to enhance their modeling capabilities. Among these methodologies, Hamiltonian neural networks (HNNs) and their variants have gained considerable attention. These architectures explicitly encode Hamiltonian mechanics both in their structure and loss function, resulting in desirable properties and additional modeled information. A crucial limitation of HNNs is their reliance on data on the target system's generalized momenta. This severely hampers their applicability when the analytical form of the generalized momenta is unknown, which frequently is the case in real-world applications. The goal of the present work is to develop a general extension of Hamiltonian-based architectures that addresses this issue by simultaneously learning the system's generalized momenta, Hamiltonian structure, and dynamics from data. The proposed architecture learns the generalized momenta when their analytical form is unknown, and only (potentially non-sequential) data on the state and its derivatives is available. A dissipative mass-spring system, a double pendulum, and a rolling disk under nonholonomic constraints are considered as canonical examples to assess the performance of the proposed architecture.

Ignacio Puiggros T., A. Srikantha Phani

Department of Mechanical Engineering, The University of British Columbia, 2054-6250 Applied Science Lane, Vancouver, V6T 1Z4, British Columbia, Canada

Hamiltonian Neural Networks Physics-Informed ML System Identification

Ask PDF

Jun 25, 2026

Symplectic Neural Networks for learning Generalized Hamiltonians

Hamiltonian Neural Networks (HNNs) integrate physical priors into neural models by learning a system's Hamiltonian, improving generalization and sample efficiency. Identifying the system Hamiltonian from noisy observations of state variables is a challenging task. For simulations to faithfully reflect the long-term behavior of Hamiltonian systems, especially energy conservation, it is essential to use symplectic integrators, which preserve the system's geometric structure. This fidelity comes at a cost: implicit symplectic integrators are more computationally intensive and make backpropagation through the ODE solver non-trivial. However, by leveraging the fact that symplectic discretizations of the adjoint system yield the same sensitivities associated by backpropagation, we obtain an efficient method of training the Neural Network parameters. In our work, we explore this alternate method of HNN training under noisy observation of trajectories with our HNN model based on an implicit symplectic integrator. Computationally, a predictor-corrector based ODE solver and fixed point iteration help to mitigate the computational cost of the implicit timestepping, resulting in more efficient generation of gradient updates. We showcase the numerical advantage, in experiments, in system identification and energy preservation on a range of non-separable, chaotic systems and the efficient computation and memory complexity of our method. We also observe that the post-processing of the learned Hamiltonian using backward error analysis yields a modified Hamiltonian that is a more accurate approximation of the true Hamiltonian without the need to use more accurate discretizations of the flow map.

Harsh Choudhary, Vyacheslav Kungurtsev, Chandan Gupta +2

Computer Science Department, Indraprasth Institute of Information Technology, New Delhi, India 110020 · Athena Research and Innovation Center, Marousi, Greece 15125 · HSBC Holdings, Singapore

Dynamical Systems Symplectic RNNs Hamiltonian Neural Networks System Identification Neural ODEs Hamiltonian Dynamics

Ask PDF

Abstract

Hamiltonian Learning

Explore similar work

Frequency-Separable Hamiltonian Neural Network for Multi-Timescale Dynamics

Learning momenta for Hamiltonian-based models

Symplectic Neural Networks for learning Generalized Hamiltonians