cs.AISep 26, 2026

Why Directly Learning Periodic Trajectories Can Fail

Authors: Kaixin Zheng, Anita Layton

Organizations: University of Waterloo Waterloo, ON, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Operator learning of periodic solutions requires deciding how simulation data should be recorded and represented. A natural choice is to integrate long enough for transients to decay and record a window wide enough to contain at least one full period of all trajectories. We find that these conservative choices can make the resulting trajectories difficult to learn, even when the underlying periodic orbits vary regularly with system parameters. Unaligned trajectories generalize poorly even within the training distribution. Phase alignment substantially improves in-distribution generalization, but models trained on a fixed physical-time window still have large errors on trajectories with periods outside the training range. We explain both failures through a common mechanism: frequency differences accumulate over time, so the target phase varies rapidly with the parameters. Predictors that cannot track this variation incur a population MSE floor in both settings; for fixed window prediction, we also derive a per-sample lower bound. We then study one of the simplest representations that escape these floors: learning an aligned, normalized waveform and its period separately. We establish regularity of the decoupled targets under ODE assumptions and show experimentally that this approach avoids both failures in ODE systems and a PDE case study.

Figures & tables

Goodwin
Norm.Align.Train mean (SD)Val mean (SD)Val minVal max
IID split
✓\checkmark✓\checkmark3.36 ⁣× ⁣10−4  (9.98 ⁣× ⁣10−7)3.36\!\times\!10^{-4}\;(9.98\!\times\!10^{-7})3.79 ⁣× ⁣10−4  (3.54 ⁣× ⁣10−6)3.79\!\times\!10^{-4}\;(3.54\!\times\!10^{-6})3.74 ⁣× ⁣10−43.74\!\times\!10^{-4}3.83 ⁣× ⁣10−43.83\!\times\!10^{-4}
×\times✓\checkmark3.03 ⁣× ⁣10−5  (1.55 ⁣× ⁣10−6)3.03\!\times\!10^{-5}\;(1.55\!\times\!10^{-6})5.48 ⁣× ⁣10−5  (4.59 ⁣× ⁣10−6)5.48\!\times\!10^{-5}\;(4.59\!\times\!10^{-6})4.94 ⁣× ⁣10−54.94\!\times\!10^{-5}6.17 ⁣× ⁣10−56.17\!\times\!10^{-5}
×\times×\times7.74 ⁣× ⁣10−4  (8.74 ⁣× ⁣10−5)7.74\!\times\!10^{-4}\;(8.74\!\times\!10^{-5})9.97 ⁣× ⁣10−1  (1.96 ⁣× ⁣10−2)9.97\!\times\!10^{-1}\;(1.96\!\times\!10^{-2})9.76 ⁣× ⁣10−19.76\!\times\!10^{-1}1.031.03
✓\checkmark×\times1.03 ⁣× ⁣10−3  (4.40 ⁣× ⁣10−5)1.03\!\times\!10^{-3}\;(4.40\!\times\!10^{-5})9.70 ⁣× ⁣10−1  (1.56 ⁣× ⁣10−2)9.70\!\times\!10^{-1}\;(1.56\!\times\!10^{-2})9.53 ⁣× ⁣10−19.53\!\times\!10^{-1}9.91 ⁣× ⁣10−19.91\!\times\!10^{-1}
Table 1: ODE waveform ablations over five training seeds. Checkmarks indicate period normalization and phase alignment. Experiments under 5 training seeds are included. SD: standard deviation.
Figure 1: FM1 shift sweep. Validation MSE divided by σphase2\sigma^{2}_{\mathrm{phase}} against the burn-in shift cc . Markers show the mean over five seeds. The range across seeds is below 8% of the mean at every point except Gold95 at c=100c=100 h (14%), which lies in the transition. Dotted line: σphase2\sigma^{2}_{\mathrm{phase}} .
Figure 2: Population validation MSE normalized by σphase2\sigma^{2}_{\mathrm{phase}} across the WW sweep, for Goodwin (left) and GOLD95 (right). Error bars are one standard deviation over 5 seeds.
Figure 3: Per-sample OOD error for Goodwin (left) and gold95 (right). Blue: bin means; red: bin minima; shading: one standard deviation across seeds; dotted line: min⁡{z2,1}\min\{z^{2},1\} .
SystemStatisticWaveform MSEPeriod error (%)
GoodwinMean(1.25±0.20)×10−4(1.25\pm 0.20){\times}10^{-4}0.099±0.0020.099\pm 0.002
 Median(7.33±1.26)×10−5(7.33\pm 1.26){\times}10^{-5}0.071±0.0030.071\pm 0.003
 95th(4.02±0.73)×10−4(4.02\pm 0.73){\times}10^{-4}0.255±0.0150.255\pm 0.015
Gold95Mean(7.61±0.37)×10−4(7.61\pm 0.37){\times}10^{-4}0.426±0.0170.426\pm 0.017
 Median(2.04±0.12)×10−4(2.04\pm 0.12){\times}10^{-4}0.313±0.0160.313\pm 0.016
 95th(4.17±0.35)×10−3(4.17\pm 0.35){\times}10^{-3}1.193±0.0691.193\pm 0.069
Table 2: Snapshot-based validation. Entries are mean ±\pm standard deviation over five training seeds. Waveform MSE is measured in the encoded state space, where each state component has unit variance over the training set.
 Norm.AlignSplitTrain MSEVal MSE
Main✓\checkmark✓\checkmarkIID9.65 ⁣× ⁣10−79.65\!\times\!10^{-7}9.25 ⁣× ⁣10−79.25\!\times\!10^{-7}
1×\times✓\checkmarkIID7.74 ⁣× ⁣10−67.74\!\times\!10^{-6}7.61 ⁣× ⁣10−67.61\!\times\!10^{-6}
2✓\checkmark×\timesIID9.80 ⁣× ⁣10−19.80\!\times\!10^{-1}1.001.00
3×\times×\timesIID9.85 ⁣× ⁣10−19.85\!\times\!10^{-1}1.001.00
4✓\checkmark✓\checkmarkOOD3.07 ⁣× ⁣10−63.07\!\times\!10^{-6}1.18 ⁣× ⁣10−21.18\!\times\!10^{-2}
5×\times✓\checkmarkOOD5.63 ⁣× ⁣10−65.63\!\times\!10^{-6}2.66 ⁣× ⁣10−12.66\!\times\!10^{-1}
Table 3: FitzHugh–Nagumo PDE representation ablations.
Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Goodwingold95
ParamBaselineLowerUpperParamBaselineLowerUpper
a1a_{1}8.375.8610.88vsv_{s}0.760.530.99
KK1.370.961.78vmv_{m}0.650.460.85
b1b_{1}1.000.701.30KmK_{m}0.500.350.65
α1\alpha_{1}1.000.701.30ksk_{s}0.380.270.49
β1\beta_{1}0.600.420.78vdv_{d}0.950.671.24
Appendix
Table 4: Sobol sampling box. Every interval is ±30%\pm 30\% of the baseline value, except the Goodwin Hill exponent, which is sampled on nH∈[10,14]n_{H}\in[10,14] .
ModelArchitectureGridEpochsLRMin. LR
Period MLPdepth 2, width 128–1005 ⁣× ⁣10−45\!\times\!10^{-4}10−610^{-6}
Waveform FNO16 modes, 128 channels12810010−310^{-3}10−510^{-5}
Appendix
Table 5: Default configuration for the period and waveform models.
RowNorm.Align.SplitWaveform target
Main✓\checkmark✓\checkmarkIIDAligned cycle
1×\times✓\checkmarkIIDAligned fixed window
2✓\checkmark×\timesIIDUnaligned cycle
3×\times×\timesIIDUnaligned fixed window
4✓\checkmark✓\checkmarkOODAligned cycle
5×\times✓\checkmarkOODAligned fixed window
Appendix
Table 6: Constructions used in the ODE ablations.
Systemcc (h)σphase2\sigma^{2}_{\mathrm{phase}}Best val MSE /σphase2/\sigma^{2}_{\mathrm{phase}}Best epochFinal val MSE /σphase2/\sigma^{2}_{\mathrm{phase}}
Goodwin000.57050.57056.870×10−4±7.9×10−66.870{\times}10^{-4}\pm 7.9{\times}10^{-6}9494 – 99996.921×10−4±1.30×10−56.921{\times}10^{-4}\pm 1.30{\times}10^{-5}
 110.57300.57307.034×10−4±8.7×10−67.034{\times}10^{-4}\pm 8.7{\times}10^{-6}9696 – 99997.099×10−4±1.09×10−57.099{\times}10^{-4}\pm 1.09{\times}10^{-5}
 10100.57180.57188.539×10−4±2.03×10−58.539{\times}10^{-4}\pm 2.03{\times}10^{-5}9696 – 99998.618×10−4±2.57×10−58.618{\times}10^{-4}\pm 2.57{\times}10^{-5}
 1001000.57210.57211.0055±0.00181.0055\pm 0.001800 – 111.8241±0.02931.8241\pm 0.0293 †\dagger
 10310^{3}0.57210.57211.0013±0.00071.0013\pm 0.000711 – 331.8134±0.01991.8134\pm 0.0199 †\dagger
 10410^{4}0.57200.57201.0034±0.00051.0034\pm 0.000500 – 111.7855±0.03381.7855\pm 0.0338 †\dagger
Appendix
Table 7: FM1 shift sweep, mean ±\pm std over five seeds. Val MSE is divided by σphase2\sigma^{2}_{\mathrm{phase}} at the same cc ; the σphase2\sigma^{2}_{\mathrm{phase}} column gives its unnormalized value. Best epoch is the range over seeds, zero based.
Hidden channels128
OptimizerAdam
Learning rate10−310^{-3}
Weight decay00
LR schedulecosine annealing, ηmin⁡=10−5\eta_{\min}=10^{-5}
Epochs100
Effective batch size32 (no gradient accumulation needed)
Appendix
Table 8: FM2 window sweep: training hyperparameters (identical for both systems).
WW (h)Final MSE/ σphase2\sigma^{2}_{\mathrm{phase}}Best MSE/ σphase2\sigma^{2}_{\mathrm{phase}}
4.720.073 ±\pm 0.0090.064 ±\pm 0.005
9.440.305 ±\pm 0.0070.289 ±\pm 0.014
18.880.805 ±\pm 0.0270.774 ±\pm 0.043
37.761.171 ±\pm 0.0581.144 ±\pm 0.052
75.521.321 ±\pm 0.1431.251 ±\pm 0.090
302.081.633 ±\pm 0.1411.051 ±\pm 0.012
Appendix
Table 9: Goodwin: WW sweep, five seeds per WW .
WW (h)Final MSE/ σphase2\sigma^{2}_{\mathrm{phase}}Best MSE/ σphase2\sigma^{2}_{\mathrm{phase}}
24.430.135 ±\pm 0.0150.109 ±\pm 0.009
48.860.514 ±\pm 0.0500.492 ±\pm 0.048
97.711.076 ±\pm 0.0420.990 ±\pm 0.050
195.431.219 ±\pm 0.0431.164 ±\pm 0.042
390.851.333 ±\pm 0.0691.228 ±\pm 0.072
1563.402.244 ±\pm 0.1211.114 ±\pm 0.042
Appendix
Table 10: GOLD95: WW sweep, five seeds per WW .
   RSD ofRSD of
SystemWW (h)Bins belowbin mean (%)bin min (%)
Goodwin4.726 / 138.816.8
 9.445 / 138.812.9
 18.886 / 136.312.5
 37.765 / 1311.511.3
 75.522 / 138.714.7
Appendix
Table 11: Per-sample results over the WW sweep.
SystemVariedRangePeriod (h)
Goodwinδ1\delta_{1}[0.56,1.04][0.56,1.04][4.36,5.53][4.36,5.53]
gold95vdv_{d}[0.66,1.26][0.66,1.26][21.58,26.03][21.58,26.03]
Appendix
Table 12: Single parameter period sweeps used to check Assumption 2.
SystemSeedwaveform MSEPeriod error (%)
Goodwin01.43×10−41.43{\times}10^{-4} / 8.29×10−58.29{\times}10^{-5} / 5.03×10−45.03{\times}10^{-4}0.1000.100 / 0.0720.072 / 0.2560.256
 19.88×10−59.88{\times}10^{-5} / 5.71×10−55.71{\times}10^{-5} / 3.37×10−43.37{\times}10^{-4}0.0980.098 / 0.0670.067 / 0.2580.258
 21.47×10−41.47{\times}10^{-4} / 8.38×10−58.38{\times}10^{-5} / 4.56×10−44.56{\times}10^{-4}0.0980.098 / 0.0700.070 / 0.2460.246
 31.14×10−41.14{\times}10^{-4} / 6.23×10−56.23{\times}10^{-5} / 3.53×10−43.53{\times}10^{-4}0.0970.097 / 0.0720.072 / 0.2380.238
 41.21×10−41.21{\times}10^{-4} / 8.05×10−58.05{\times}10^{-5} / 3.64×10−43.64{\times}10^{-4}0.1030.103 / 0.0760.076 / 0.2780.278
Gold9507.32×10−47.32{\times}10^{-4} / 1.88×10−41.88{\times}10^{-4} / 3.95×10−33.95{\times}10^{-3}0.4330.433 / 0.3200.320 / 1.1901.190
Appendix
Table 13: Per-seed snapshot-based validation. Entries are mean / median / 95th percentile over validation samples.
QuantitySymbolValue
Activator diffusionDuD_{u}11
Inhibitor diffusionDvD_{v}11
Domain lengthℓ\ell5050
Fixed reaction parameteraa0.10.1
Sampled parameterbb[0.35,0.65][0.35,0.65]
Sampled parameterε\varepsilon[0.035,0.065][0.035,0.065]
Appendix
Table 14: FitzHugh–Nagumo PDE and data-generation settings.
SetSamplesPeriod rangeCycles in WOODW_{\mathrm{OOD}}
Training2000200037.0537.05 – 42.9442.941.001.00 – 1.161.16
Validation50050052.4352.43 – 55.8555.850.770.77 – 0.820.82
Appendix
Table 15: Period-based OOD split for the FitzHugh–Nagumo experiments.
SettingWaveform FNOPeriod MLP
Architecture1616 modes/dim., 6464 channelsdepth 22 , width 128128
Activation–GELU
Batch size16163232
Epochs5050200200
OptimizerAdamAdam
Initial learning rate10−310^{-3}10−310^{-3}
Appendix
Table 16: FitzHugh–Nagumo model and training configuration.
More in this topic

Operator Learning

Operator learning studies models that map functions or fields to functions, enabling fast prediction across scientific and dynamical systems.

See all 99 papers

Explore similar work

CardsList

  1. Physics is the Best Teacher: Consistency Learning for Time-Invariant Operators of Chaotic Dynamics

    Oct 2, 2026Lufang Chiang, Jiachen Yao, Thomas Y. L. Lin +1Chaotic Time-Series PredictionChaotic Dynamical Systems

    National Taiwan University · California Institute of Technology · University of Washington

  2. Symbolic Neural ODEs: Learning interpretable models from time-series data

    Aug 22, 2026Nibodh Boddupalli, Jeff MoehlisNonlinear System IdentificationSystem Identification

    Dept. of Mechanical Engineering, University of California, Santa Barbara, CA 93106

  3. Neural operator discovery from heterogeneous trajectories

    Jul 25, 2026Zituo Chen, Qiaofeng Li, Jiaxin Hu +1Latent Dynamics ModelingNonlinear System Identification

    Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, 02139, MA, USA. · State Key Laboratory of Fluid Power & Mechatronic Systems, Zhejiang University, 866 Yuhangtang Road, Hangzhou, 310058, Zhejiang Province, China.