When Does Jacobian Fine-Tuning Stabilize Neural PDE Emulators?

Shreeyam Bangera; Manya Gupta

cs.LGOct 30, 2026

When Does Jacobian Fine-Tuning Stabilize Neural PDE Emulators?

Authors: Shreeyam Bangera, Manya Gupta

Organizations: IIT Roorkee

Abstract

Neural emulators of chaotic PDEs are trained on one-step prediction but deployed autoregressively, where their errors compound. Jacobian-based regularisers aim to curb this growth, but in a chaotic system part of the growth is physical and cannot be trained away. We separate the two sources of this error with a first-order decomposition. We compare the emulator's error with a counterfactual that propagates the same one-step mistakes through the solver's Jacobian, giving an amplification ratio: the factor by which the model's own dynamics amplify its errors. We study 21 configurations across four PDEs, five architectures and two model sizes (8-10k and 33k parameters), with errors measured at one Lyapunov time. Jacobian fine-tuning reduced the error by more than 10% for 1 of 34 seeds whose amplification ratio was at most 1.25, and for all 20 seeds whose ratio was at least four; for the latter, the error fell by a factor of 8 (small models) and 55 (33k models) in geometric mean over seeds. The gain approximately equals the one-step accuracy cost of fine-tuning divided by the amplification ratio. Matching the solver's Jacobian along the model's own error direction (EDJM) outperformed random directions at the same cost on 28 of 35 seeds; input-noise injection performed comparably to EDJM.

Figures & tables

	systems	architectures	parameters	seeds
Main study	KS, gKS, CGLE	CNN, ResNet, U-Net, FNO, Transformer	8–10k	3
Scaling check	KS, gKS, CGLE, Nik.	ResNet, FNO	33k	5 †

Table 1: Study design. The main study has 14 configurations and 42 seeds (KS CNN excluded); the scaling check has 7 configurations and 32 seeds. † Two seeds for gKS-ResNet, which showed no effect on both and was not extended.

Figure 1: Example on one system: actual error

\|e_{n}\|

(solid) and counterfactual error

\|e^{P}_{n}\|

(dashed) for two small emulators of gKS, one with

\rho\approx 1

and the configuration with the highest amplification ratio on this system. Thin lines: the three seeds; thick lines: their geometric mean over 30 test trajectories. Dotted: one Lyapunov time. Shaded: error in excess of the counterfactual. The

\rho

in the panel titles is computed from the thick (seed-averaged) curves; Table 4 reports the median over seeds (7.1 for the CNN). All configurations are shown in Figures 5 and 6 .

Figure 2: Error ratio

r

( Equation 14 ) after EDJM fine-tuning, final checkpoint, against the amplification ratio of the starting model. Each point is one seed; below the dotted line is an improvement of more than 10%. Grey:

\rho\leq 1.25

; blue:

\rho\geq 4

Figure 3: (a) Amplification ratio before and after EDJM fine-tuning (dashed: unchanged). (b) Error ratio against

c/\rho

, where

c

is the one-step cost ( Equation 15 ); dashed:

r=c/\rho

. Colour: system; marker shape: architecture; filled: 8–10k parameters; hollow: 33k. Final checkpoints, 74 seeds.

Figure 4: Per-seed error ratio of EDJM against (a) RJM and (b) Noise, on all seeds with

\rho>1.25

, validation-selected checkpoints. Points in the shaded region favour EDJM; points on the diagonal at

r=1

are seeds where both runs kept the starting model. Colour: system; marker shape: architecture; filled: 8–10k; hollow: 33k. Panel (b) shows the small models only; the 33k comparison is in Table 6 .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

system	parameters	$L$	$\Delta t$ (substeps)	channels	$\lambda$ per step	1 LT [steps]
KS	–	60	0.25 (1)	1	0.0175	57
gKS	$\delta=0.2$ , $\nu=1$	200	0.25 (1)	1	0.0081	123
CGLE	$b_{1}=3.5$ , $b_{3}=1.4$	250	0.2 (4)	2	0.0085	118
Nikolaevskiy	$r_{\mathrm{N}}=0.02$	200	10.0 (5)	1	0.0918	11

Appendix

Table 2: Systems.

\Delta t

is the time advanced by one emulator step, integrated with the given number of solver substeps.

r_{\mathrm{N}}

is the supercriticality parameter

r

of Cox and Matthews [7] , renamed to avoid a clash with the error ratio.

architecture	configuration	parameters
8–10k parameters (main study)
CNN	5 hidden convolutions, width 23, kernel 3, no internal skips	8.2k
ResNet	8 residual blocks, width 13, kernel 3, ReLU	8.4k
U-Net	3 levels, average-pool down, repeat up, skip concatenation	10.3k
FNO	4 blocks, width 9, 12 modes, GELU (gKS: width 6, 32 modes)	8.9k (9.9k)
Transformer	patches of 4 points, 2 pre-norm blocks, width 20, 2 heads	8.1k

Appendix

Table 3: Architectures. Parameter counts are for one channel; the two-channel CGLE models are slightly larger. On gKS the small FNO uses 32 modes so that it covers the unstable band of the larger domain.

8–10k parameters (14 configurations, 42 seeds)
configuration	seeds	$\rho$	error ratio $r$	improved
CGLE FNO	3	0.88	1	0/3
KS FNO	3	0.93	2.2	0/3
KS U-Net	3	1	2.3	0/3
gKS FNO	3	1	1	1/3
gKS ResNet	3	1	1.4	0/3

Appendix

Table 4: Per configuration, ordered by

\rho

within each size: median amplification ratio of the starting models, geometric-mean error ratio after EDJM (final checkpoint), and seeds with

r<0.9

8–10k parameters (17 / 14 / 11 seeds)
	$\rho\leq 1.25$	$1.25<\rho<4$	$\rho\geq 4$
Cont. training	0/17 (0.97)	0/14 (1.08)	0/11 (1.50)
Noise	1/17 (0.95)	3/14 (0.87)	9/11 (0.25)
RJM	0/17 (1.71)	5/14 (0.87)	10/11 (0.18)
EDJM	0/17 (1.73)	9/14 (0.82)	11/11 (0.13)
33k parameters (17 / 6 / 9 seeds)

Appendix

Table 5: Seeds improved by more than 10% at one Lyapunov time, and in parentheses the geometric-mean error ratio over seeds, grouped by the amplification ratio of the starting model; final checkpoints.

	comparison	EDJM better	sign test $p$	geo. mean ratio
8–10k ( $\rho>1.25$ , 25 seeds)	vs RJM	16/20	0.012	0.86
	vs Cont. training	19/25	0.015	0.36
	vs Noise	16/25	0.23	0.72
33k ( $\rho>1.25$ , 15 seeds)	vs RJM	12/15	0.035	0.69
	vs Cont. training	14/15	0.001	0.07
	vs Noise (KS ResNet)	0/5	0.063	1.12

Appendix

Table 6: Paired comparisons of the error at one Lyapunov time, validation-selected checkpoints. Ties (both runs kept the starting model) are dropped from the counts and sign tests; the geometric-mean ratio of EDJM error to the other method’s is over all seeds, ties included. Input noise at 33k uses the best of three noise levels.

	8–10k parameters		33k parameters
evaluated at	$\rho\leq 1.25$	$\rho\geq 4$	$\rho\leq 1.25$	$\rho\geq 4$
0.5 Lyapunov times	1/18	7/7	0/17	7/7
1 Lyapunov time	0/17	11/11	1/17	9/9
1.5 Lyapunov times	2/17	10/11	2/18	9/9

Appendix

Table 7: Seeds whose error improved by more than 10% after EDJM fine-tuning (final checkpoint), with

\rho

and the error ratio both read at the given time. The middle row is the analysis of the main text.

Figure 5: Actual error (solid) and counterfactual error (dashed) for every small-model configuration; thin lines are the three seeds, thick lines their geometric mean over 30 test trajectories, and

\rho

is the median over seeds at one Lyapunov time (dotted). The KS CNN, excluded from all analyses, is shown in grey.

Figure 6: As Figure 5 , for the 33k-parameter models (five seeds; two for gKS ResNet).

PDE Surrogate Modeling

PDE surrogate modeling builds fast approximations of partial differential equation solutions for simulation, prediction, and design.

See all 347 papers

Eigenanalysis framework for autoregressive neural emulators of multi-scale chaotic dynamics
Aug 17, 2026Conrad Ainslie, Pedram Hassanzadeh, Michael W. Mahoney +1Neural Network Stability Neural Surrogate Modeling
Department of Applied Mathematics, University of California, Santa Cruz, Santa Cruz, 95064, CA · Department of Geophysical Sciences, University of Chicago, Chicago, IL 60637 · Committee on Computational and Applied Mathematics, University of Chicago, Chicago, IL 60637 +3
From Numerical Simulators of PDEs to Neural Emulators and Back
Aug 25, 2026Felix KoehlerPDE Surrogate Modeling PDE Solving
Technische Universität München TUM School of Computation, Information and Technology
Error-Conditioned Neural Solvers
Jun 25, 2026Haina Jiang, Liam Wang, Peng-Chen Chen +4Neural PDE Solvers PDE Surrogate Modeling
University of Michigan · KAIST AI · Los Alamos National Laboratory

Aug 17, 2026J/K move · Enter open · S save

Eigenanalysis framework for autoregressive neural emulators of multi-scale chaotic dynamics

Conrad Ainslie, Pedram Hassanzadeh, Michael W. Mahoney, Ashesh Chattopadhyay

Department of Applied Mathematics, University of California, Santa Cruz, Santa Cruz, 95064, CA · Department of Geophysical Sciences, University of Chicago, Chicago, IL 60637 · Committee on Computational and Applied Mathematics, University of Chicago, Chicago, IL 60637+3

Neural Network Stability Neural Surrogate Modeling Chaotic Time-Series Prediction Chaotic Dynamical Systems Jacobian Regularization Multiscale Time-Series Modeling

Open paper AskPDF arXiv

Neural autoregressive models have rapidly emerged as powerful emulators of high-dimensional chaotic systems, yet their long-term instability and error growth remain poorly understood, leading to ad-hoc solutions. Here, we develop an eigenanalysis framework that reveals the dynamical origin of this error growth. By analyzing the Jacobian of the learned one-step update map with respect to the state, we show how inference-time error growth, and thus model stability, is governed by its spectral radius. Direct-step architectures (models that predict the next state from the previous one) generically admit unstable eigenvalues with magnitudes exceeding one, explaining the rapid divergence of these widely used models. In contrast, integration-constrained models (where the time derivative is estimated and integrated with a higher-order integrator) collapse their eigenspectrum onto the unit circle, yielding neutral stability and a universal linear error-scaling law. The largest eigenvalue of this Jacobian provides an architecture-agnostic, a priori diagnostic of short-term skill, long-term stability, and spectral bias, without requiring an expensive rollout. Leveraging this theory, we introduce a stability-promoting loss that explicitly regularizes Jacobian-driven error amplification, improving both forecast accuracy and dynamical robustness. Demonstrated across

29

models spanning two architectures, several explicit and implicit integrators, and multiple loss functions on the Kuramoto-Sivashinsky system, our results establish a theoretical foundation for the design and evaluation of neural emulators of chaotic multi-scale dynamics. More broadly, our framework is a step toward the kind of a priori stability analysis that numerical analysis provides for discretizations of differential equations and that scientific machine learning currently lacks.

Abstract

PDE Surrogate Modeling

Explore similar work

Eigenanalysis framework for autoregressive neural emulators of multi-scale chaotic dynamics

From Numerical Simulators of PDEs to Neural Emulators and Back

Error-Conditioned Neural Solvers