math.NAOct 18, 2026

Integrable building blocks for symplectic neural networks: learned-type layers, fiber-preserving layers, and what a single layer can do

Authors: Andrew Qing He

Organizations: Duke University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Exactly symplectic neural networks are built almost exclusively from shear maps and gain expressivity only through depth. We ask what else a single explicit symplectic layer can be. Observing that every layer in use is the exact flow of a Hamiltonian that is a function of commuting integrals, we obtain three families of layers: a learned-type layer, in which every conjugate pair is rotated, squeezed or sheared by a state-dependent amount and the type is found by training; moving-plane shears, residual symplectic maps outside the known ridge-function class; and fiber-preserving layers, which combine an unconstrained point transformation with a fiber translation at the cost of one linear solve. We characterize residual symplectic layers x↦x+J∇H(x)x \mapsto x + J\nabla H(x): in one degree of freedom they are exactly the shears along a line, while from two degrees of freedom on a strictly larger class exists. Conjugating a learned-type layer by a symplectic network yields an exact one-parameter group with an exactly conserved learned Hamiltonian. In experiments on sixteen systems with one to forty degrees of freedom, against SympNets, HénonNets and learned-plane shear networks, layers built on nonlinear integrals are far more accurate when the Hamiltonian contains functions of those integrals. On nonlinear Schrödinger lattices with six to forty sites, twist layers alternating with linear symplectic maps are over a hundred times more accurate than every shear-based network, and with an on-site potential of learned form three to four orders of magnitude more accurate. On chains of Tao oscillators with the step size as an input, stretch layers are 45 to 70 times more accurate than learned-plane shears and recover the Hamiltonian to 1%. On generic non-separable systems, fiber layers improve on learned-plane shears by at most a factor of two, and widening the networks does not help any architecture.

Figures & tables

systemnnHamiltonianremarks
pendulum1p2/2−cos⁡qp^{2}/2-\cos qseparable
saddle1p2/2−log⁡cosh⁡qp^{2}/2-\log\cosh qseparable, hyperbolic
double well1p2/2−q2/2+q4/4p^{2}/2-q^{2}/2+q^{4}/4separable, saddle and two centers
Tao1(q2+1)(p2+1)/2(q^{2}+1)(p^{2}+1)/2non-separable, in class
Hénon–Heiles2standardseparable
coupled Tao2∑i(qi2+1)(pi2+1)/2+q1p1q2p2/2\sum_{i}(q_{i}^{2}+1)(p_{i}^{2}+1)/2+q_{1}p_{1}q_{2}p_{2}/2non-separable, in class
Table 1: Test systems. “In class” marks Hamiltonians of the form V(q)+K(p)+W(q⊙p)V(q)+K(p)+W(q\odot p) .
Figure 1: Training loss per epoch (lines) and test error every 250 epochs (circles) for one representative run per model and system. Adam for 5000 epochs, then L-BFGS (dotted line).
Figure 2: Width study at fixed depth, two degrees of freedom. Lines join the same architecture at hidden sizes (20,10)(20,10) , (40,20)(40,20) , (80,40)(80,40) ; isolated points are deeper networks.
Figure 3: Depth study: median test error against the number of layers, by layer family.
systemQPQPQPG-SympNetlearned-planewith Swith W
pendulum2.8⋅10−72.8\cdot 10^{-7}5.6⋅10−75.6\cdot 10^{-7}1.0⋅10−71.0\cdot 10^{-7}1.2⋅10−71.2\cdot 10^{-7}6.0⋅10−86.0\cdot 10^{-8}
saddle5.1⋅10−85.1\cdot 10^{-8}3.2⋅10−73.2\cdot 10^{-7}2.6⋅10−72.6\cdot 10^{-7}1.2⋅10−71.2\cdot 10^{-7}5.3⋅10−85.3\cdot 10^{-8}
Hénon–Heiles3.1⋅10−73.1\cdot 10^{-7}9.9⋅10−79.9\cdot 10^{-7}9.2⋅10−89.2\cdot 10^{-8}3.3⋅10−73.3\cdot 10^{-7}2.2⋅10−72.2\cdot 10^{-7}
Tao8.0⋅10−38.0\cdot 10^{-3}2.3⋅10−32.3\cdot 10^{-3}6.7⋅10−56.7\cdot 10^{-5}1.6⋅10−51.6\cdot 10^{-5}8.2⋅10−58.2\cdot 10^{-5}
coupled Tao7.0⋅10−47.0\cdot 10^{-4}7.9⋅10−47.9\cdot 10^{-4}2.1⋅10−52.1\cdot 10^{-5}1.3⋅10−61.3\cdot 10^{-6}1.1⋅10−51.1\cdot 10^{-5}
Tao chain, 6 sites4.9⋅10−44.9\cdot 10^{-4}–4.2⋅10−54.2\cdot 10^{-5}6.0⋅10−76.0\cdot 10^{-7}1.1⋅10−61.1\cdot 10^{-6}
Table 2: Step size as input: one-step test MSE, median of three seeds (five for the Tao-type rows); hmax⁡=0.5h_{\max}=0.5 , and 0.30.3 for the last four rows. The learned-plane column reports the better of two configurations of the layer of [ 27 ] : ours (two hidden layers, as for the other models) and the one of the author’s public code (one hidden layer, initialization at scale 0.010.01 , 12–24 layers); on the Tao-type rows the latter is better. Six layers for n≤2n\leq 2 , eight for the chains (periodic chains of Tao oscillators coupled through qipiqi+1pi+1/2q_{i}p_{i}q_{i+1}p_{i+1}/2 ).
Figure 4: Tao’s system, step size as input. Level sets of the true Hamiltonian (gray) and of the sum of the layer generators (dashed), at the same level values.
Figure 5: Tao’s system, step size as input: true orbits (gray) and the learned maps iterated 80 times with step 0.250.25 from six test initial conditions; right: q(t)q(t) on the outermost orbit.
Figure 6: Chain of ten Tao oscillators, step size as input: true solution (gray) and the learned maps iterated 30 times with step 0.30.3 from one test initial condition (first two sites shown); bottom right: error against time.
Figure 7: Tao’s system: true solution (gray) and learned solutions, trained on t≤2t\leq 2 .
Figure 8: Pendulum: conjugated learned-type flow against step-size-aware networks, trained on t≤2t\leq 2 . Left: prediction error against time to t=60t=60 . Middle: predictions at t∈[40,60]t\in[40,60] (dots) on the true orbits. Right: level sets of the learned conserved quantity over the true energy levels.
Figure 9: Saddle–center system: true solution (gray) and learned solutions.
Figure 10: Periodic Toda lattice with three particles, zero total momentum: true positions (gray) and learned solutions, trained on t≤2t\leq 2 (shaded).
Figure 11: Double well: true orbits (gray) and 60 iterates of the learned time-1 map.
Figure 12: Polar map on the annular domain: image of a coordinate grid under the true map (gray) and under three learned maps (dashed).
Figure 13: Pendulum: true orbits (gray) and 60 iterates of three learned time-1 maps.
Figure 14: Charged particle in a non-uniform magnetic field ( n=2n=2 ): true solution (gray) and 30 iterates of the learned time- 0.50.5 maps.
systemFGFGqq / pp shears ×8\times 8learned-plane ×8\times 8 [ 27 ]
charged particle2.0⋅10−52.0\cdot 10^{-5} (0.9(0.9 – 17)17)2.5⋅10−42.5\cdot 10^{-4}4.0⋅10−54.0\cdot 10^{-5} (2.8(2.8 – 6.9)6.9)
double pendulum1.7⋅10−51.7\cdot 10^{-5} (1.6(1.6 – 1.9)1.9)3.5⋅10−53.5\cdot 10^{-5}3.0⋅10−53.0\cdot 10^{-5} (2.3(2.3 – 12)12)
coupled Tao1.9⋅10−51.9\cdot 10^{-5}3.6⋅10−53.6\cdot 10^{-5}1.6⋅10−51.6\cdot 10^{-5}
Hénon–Heiles5.6⋅10−75.6\cdot 10^{-7}4.7⋅10−74.7\cdot 10^{-7}8.6⋅10−78.6\cdot 10^{-7}
Table 3: Two degrees of freedom, fixed-time maps: one-step test MSE, median (range) over five seeds, about 2300 parameters per model.
 cubicsaturable
model61020401020
ULULULUL (site-wise twist)8.6⋅10−8\mathbf{8.6\cdot 10^{-8}}1.3⋅10−7\mathbf{1.3\cdot 10^{-7}}1.4⋅10−7\mathbf{1.4\cdot 10^{-7}}2.4⋅10−7\mathbf{2.4\cdot 10^{-7}}1.3⋅10−7\mathbf{1.3\cdot 10^{-7}}2.0⋅10−7\mathbf{2.0\cdot 10^{-7}}
KLKLKLKL (fixed cubic form)4.3⋅10−74.3\cdot 10^{-7}5.0⋅10−75.0\cdot 10^{-7}7.2⋅10−77.2\cdot 10^{-7}1.2⋅10−61.2\cdot 10^{-6}1.3⋅10−41.3\cdot 10^{-4}1.4⋅10−41.4\cdot 10^{-4}
TLTLTLTL (general twist)1.1⋅10−61.1\cdot 10^{-6}2.9⋅10−62.9\cdot 10^{-6}7.0⋅10−67.0\cdot 10^{-6}3.5⋅10−53.5\cdot 10^{-5}6.6⋅10−66.6\cdot 10^{-6}2.1⋅10−52.1\cdot 10^{-5}
TPTQTPTQ (twist + shears)5.6⋅10−65.6\cdot 10^{-6}1.3⋅10−51.3\cdot 10^{-5}7.6⋅10−57.6\cdot 10^{-5}2.6⋅10−42.6\cdot 10^{-4}––
WLWLWLWL (learned-type)2.2⋅10−52.2\cdot 10^{-5}2.5⋅10−52.5\cdot 10^{-5}2.4⋅10−52.4\cdot 10^{-5}3.2⋅10−53.2\cdot 10^{-5}1.8⋅10−51.8\cdot 10^{-5}2.1⋅10−52.1\cdot 10^{-5}
Table 4: Lattices: one-step test MSE of the time- 0.30.3 map, median over five seeds (three for ULULULUL at 6 and 40 sites and on the saturable lattice, and for the last three rows). “Learned-plane” is the better of our configuration and the author’s. Bottom: Strang splitting with the true Hamiltonian, for orientation.
Figure 15: Cubic lattice: test error against the number of sites (median and range over seeds). Dotted: Strang splitting with the true Hamiltonian.
Figure 16: Cubic ten-site lattice: test error against the number of training points (median and range over three to five seeds).
Figure 17: Cubic lattice with 20 sites: true solution (gray) and thirty iterates of each learned time- 0.30.3 map from one test initial condition (first two sites shown).
Figure 18: Saturable lattice with 20 sites: true solution (gray) and thirty iterates of each learned time- 0.30.3 map from one test initial condition.
Figure 19: Cubic lattice with 40 sites: true solution (gray) and thirty iterates of each learned time- 0.30.3 map from one test initial condition.
Figure 20: Cubic lattice with 20 sites: learned time- 0.30.3 maps iterated 300 times; medians over 50 orbits and five seeds of the error against the true solution and of the relative errors in energy and mass.
system ( nn )FGFGqq / pp shears ×8\times 8learned-plane ×8\times 8WPWQWPWQ
three-link pendulum (3)9.6⋅10−5\mathbf{9.6\cdot 10^{-5}}7.2⋅10−47.2\cdot 10^{-4}1.4⋅10−41.4\cdot 10^{-4}8.4⋅10−48.4\cdot 10^{-4}
two charged particles (4)1.0⋅10−4\mathbf{1.0\cdot 10^{-4}}2.9⋅10−42.9\cdot 10^{-4}1.4⋅10−41.4\cdot 10^{-4}1.6⋅10−41.6\cdot 10^{-4}
FPUT chain (8)6.1⋅10−56.1\cdot 10^{-5}3.5⋅10−5\mathbf{3.5\cdot 10^{-5}}9.8⋅10−59.8\cdot 10^{-5}–
Table 5: Generic systems in higher dimension: one-step test MSE, median over five seeds (three for the FPUT chain).
Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 21: Lattices (Table 4 ): every seed of every model.
Figure 22: Tao and coupled Tao, step size as input (Table 2 ): every seed.
Figure 23: Chains of Tao oscillators, step size as input (Table 2 ): every seed. Rows are models; TPTQTPTQ is the twist network, which carries the wrong integral for this system.
Figure 24: Learned-plane shears in the configuration of the author’s code ( tz:layers:width ), on the lattices and the Tao-type systems, with the standard and the four-fold training budget.
Figure 25: Two degrees of freedom, fixed-time maps (Table 3 ) and step size as input: every seed.
systemfixed corelearned, standardlearned, near-paraboliclearned, ψ\psi near identity
pendulum10467
Tao10585
saddle8959
Appendix
Table 6: Reliability of conjugated flows over ten seeds: number of runs with test MSE below 10−310^{-3} . “Fixed” uses the correct type (twist for pendulum and Tao, stretch for the saddle); “learned” uses the learned-type core from the standard or the near-parabolic initialization of MjM_{j} ; “ ψ\psi near identity” starts the change of variables close to the identity.
assignmenttest MSE per seedlowest training loss
E, H (correct)5.5⋅10−75.5\cdot 10^{-7} , 8.2⋅10−78.2\cdot 10^{-7} , 1.5⋅10−61.5\cdot 10^{-6} , 2.0⋅10−62.0\cdot 10^{-6} , 6.2⋅10−66.2\cdot 10^{-6}1.6⋅10−71.6\cdot 10^{-7}
E, E3.8⋅10−33.8\cdot 10^{-3} – 2.5⋅10−22.5\cdot 10^{-2}1.7⋅10−31.7\cdot 10^{-3}
H, E9.9⋅10−39.9\cdot 10^{-3} – 3.1⋅10−23.1\cdot 10^{-2}4.0⋅10−34.0\cdot 10^{-3}
H, H5.3⋅10−35.3\cdot 10^{-3} – 1.3⋅10−21.3\cdot 10^{-2}1.3⋅10−31.3\cdot 10^{-3}
Appendix
Table 7: Saddle–center system: conjugated flows with the core’s types fixed in advance, five seeds each. The correct assignment (elliptic, hyperbolic) is separated from the others by four orders of magnitude in training loss.
modelfitst=10t=10t=40t=40
conjugated, types fixed to E, E, P56.9⋅10−36.9\cdot 10^{-3}2.8⋅10−22.8\cdot 10^{-2}
conjugated, types fixed to E, E, E37.2⋅10−37.2\cdot 10^{-3}2.8⋅10−22.8\cdot 10^{-2}
conjugated, twist core37.1⋅10−37.1\cdot 10^{-3}2.6⋅10−22.6\cdot 10^{-2}
conjugated, learned-type core16.8⋅10−36.8\cdot 10^{-3}2.6⋅10−22.6\cdot 10^{-2}
stepped qq / pp shears ×8\times 8 , iterated51.8⋅10−21.8\cdot 10^{-2}5.8⋅10−25.8\cdot 10^{-2}
stepped learned-plane shears ×8\times 8 , iterated53.1⋅10−23.1\cdot 10^{-2}2.1⋅10−12.1\cdot 10^{-1}
Appendix
Table 8: Periodic Toda lattice with three particles, trained on t≤2t\leq 2 : runs that fit (of five) and RMSE at t=10t=10 and t=40t=40 for zero-momentum initial states, medians over the runs that fit.
Figure 26: Reliability sweep: test error of every seed of the conjugated learned-type model on the pendulum, Tao’s system and the saddle, for two initializations of MjM_{j} .
Figure 27: Saddle–center system: every seed of the conjugated models with fixed types.
Figure 28: Long-time prediction at twenty times the training horizon: conjugated flows (one evaluation) against stepped networks (twenty steps), for orbits that stay in the training box.
systemfamily23468 layers
pendulumqq / pp shears3.0⋅10−23.0\cdot 10^{-2}4.6⋅10−44.6\cdot 10^{-4}4.2⋅10−74.2\cdot 10^{-7}2.3⋅10−72.3\cdot 10^{-7}3.5⋅10−73.5\cdot 10^{-7}
 learned-plane (ours)1.6⋅10−41.6\cdot 10^{-4}2.3⋅10−62.3\cdot 10^{-6}1.8⋅10−61.8\cdot 10^{-6}7.1⋅10−77.1\cdot 10^{-7}3.8⋅10−73.8\cdot 10^{-7}
 learned-type1.5⋅10−51.5\cdot 10^{-5}1.1⋅10−61.1\cdot 10^{-6}4.2⋅10−74.2\cdot 10^{-7}2.2⋅10−72.2\cdot 10^{-7}2.5⋅10−72.5\cdot 10^{-7}
Tao, T=0.5T=0.5qq / pp shears6.6⋅10−26.6\cdot 10^{-2}3.8⋅10−23.8\cdot 10^{-2}3.7⋅10−33.7\cdot 10^{-3}4.7⋅10−64.7\cdot 10^{-6}7.0⋅10−67.0\cdot 10^{-6}
 learned-plane (ours)2.8⋅10−22.8\cdot 10^{-2}7.9⋅10−37.9\cdot 10^{-3}1.4⋅10−41.4\cdot 10^{-4}5.7⋅10−55.7\cdot 10^{-5}1.1⋅10−51.1\cdot 10^{-5}
 learned-type8.0⋅10−38.0\cdot 10^{-3}5.3⋅10−55.3\cdot 10^{-5}1.9⋅10−51.9\cdot 10^{-5}1.5⋅10−61.5\cdot 10^{-6}1.2⋅10−61.2\cdot 10^{-6}
Appendix
Table 9: Depth study (Figure 3 ): median test MSE over two seeds.
modelunit square[12,32]×[0,3π2][\tfrac{1}{2},\tfrac{3}{2}]\times[0,\tfrac{3\pi}{2}]
WPWQW7.2⋅10−67.2\cdot 10^{-6}1.9⋅10−21.9\cdot 10^{-2}
PQWPQ8.4⋅10−68.4\cdot 10^{-6}2.3⋅10−32.3\cdot 10^{-3}
CPQDP (cotangent lifts)1.7⋅10−51.7\cdot 10^{-5}2.6⋅10−42.6\cdot 10^{-4}
learned-plane (ours)2.2⋅10−52.2\cdot 10^{-5}5.3⋅10−45.3\cdot 10^{-4}
PQPQP2.6⋅10−52.6\cdot 10^{-5}2.5⋅10−32.5\cdot 10^{-3}
PQSPQ (network of [ 15 ] )2.7⋅10−52.7\cdot 10^{-5}2.7⋅10−32.7\cdot 10^{-3}
Appendix
Table 10: Polar map of [ 15 ] : test MSE, median of three seeds, five-layer models.
More in this topic

Structure-Preserving Neural Networks

See all 14 papers

Explore similar work

CardsList

  1. Symplectic Neural Networks for learning Generalized Hamiltonians

    Jun 25, 2026Harsh Choudhary, Vyacheslav Kungurtsev, Chandan Gupta +2Dynamical SystemsSymplectic RNNs

    Computer Science Department, Indraprasth Institute of Information Technology, New Delhi, India 110020 · Athena Research and Innovation Center, Marousi, Greece 15125 · HSBC Holdings, Singapore

  2. Symplectic Neural Operators for Learning Infinite Dimensional Hamiltonian Systems

    May 15, 2026Yeang Makara, Yusuke Tanaka, Takashi Matsubara +1PDE Operator LearningNeural Operators

    Graduate School of Science Kobe University 1-1 Rokkodai-cho, Nada-ku, Kobe, Japan · NTT Communication Science Laboratories 2-4, Hikaridai, Seika-cho, Soraku-gun, Kyoto, Japan · Faculty of Information Science and Technology Hokkaido University Kita 14, Nishi 9, Kita-ku, Sapporo, Hokkaido, Japan +1

  3. CoSynFlow: Conformal Symplectic Neural Flows for Cross-System Prediction of Dissipative Hamiltonian Dynamics

    Aug 1, 2026Baige Xu, Takaharu YaguchiNeural Operators

    RIKEN AIP · Motooka Nishi-ku, Fukuoka, Japan · IMI, Kyushu University