Learning the Entropic Relaxation: Stable Coarse-Grid Emulation with Neural Discrete Equilibrium

Jose Antonio Lara Benitez; Kareem Hegazy; Anastasis Kratsios; Ivan Dokmanić; Maarten V. de Hoop; Michael W. Mahoney

cs.LGNov 2, 2026

Learning the Entropic Relaxation: Stable Coarse-Grid Emulation with Neural Discrete Equilibrium

Authors: Jose Antonio Lara Benitez, Kareem Hegazy, Anastasis Kratsios, Ivan Dokmanić, Maarten V. de Hoop, Michael W. Mahoney

Organizations: Rice University and LBNL · ICSI and University of California at Berkeley · McMaster University and the Vector Institute · University of Basel ·

Abstract

The ability to emulate complicated and computationally-expensive physical systems in an efficient and stable manner is a popular motivation for scientific machine learning (SciML). Stable numerical solvers tend to be computationally expensive, as they attempt to resolve small (spatial and temporal) scales that receive and dissipate spectral energy. Learned coarse-grid forecasters remove the dissipative channel, allowing unresolved frequencies to re-emerge as grid-scale noise (aliasing). In this paper, we use the Neural Discrete Equilibrium (NeurDE) method to explore stable emulation at extremely coarse grids, demonstrating physically-principled control over entropy dynamics that reinforces the physical system's dissipative mechanism. Entropically-stabilized NeurDE produces stable long-time rollouts for thousands of forecasting steps on large Reynolds number turbulent flows ( $\mathrm{Re}=100{,}000$ ) with very coarse grids (down to $\mathbb{R}^{16\times16}$ ). At these very coarse grid scales, entropically-stabilized NeurDE can be more stable than even our stabilized data-generation numerical method.

Figures & tables

Panel of Figure 1 — Figure 1: Coarse discretization interrupts scale-to-scale energy transfer. Solid curves show the resolved spectral power at successive stages of a rollout; dashed lines indicate an idealized inertial-range decay. As turbulence develops, a coarse representation loses the high-wavenumber dissipative tail and can accumulate grid-scale content, unless the unresolved transfer is modeled. The learned entropic relaxation supplies a local dissipative channel for this missing transfer.

Panel of Figure 2 — Figure 2: Architecture of entropically-stabilized NeurDE . a The pipeline decouples feature extraction from the iterative DEQ solver. Context $\mathbf{z}$ is encoded once and injected into every iteration. b The neural update block processes the concatenated context, state, and physics hints through a convolutional network to refine the stability parameter $\alpha$ . c Physics hints include: entropy residual $G=H(\alpha)-H_{\mathrm{pre}}$ , gradients $\Delta=H^{\prime}(\alpha)$ , and instability indicator. The physics hints are computed outside of the computational graph, ensuring gradients flow only through $g_{\theta}$ (implicit differentiation).

Panel of Figure 5 — Figure 5: Long-rollout comparison on the doubly periodic shear layer at Re=100,000. Vorticity forecasts for the box-filtered high-resolution reference (top), entropy-stabilized NeurDE (middle), and an underresolved ELBM baseline (bottom) on (a) a $64\times 64$ grid and (b) a $16\times 16$ grid.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	4/4	$\bf{0.0807\pm 6.9{\times}10^{-5}}$	$\bf{0.846\pm 2.5{\times}10^{-4}}$	$\bf{0.536\pm 3.8{\times}10^{-4}}$	$3.28\pm 2.4{\times}10^{-3}$	$0.0454\pm 2.2{\times}10^{-4}$	$0.104\pm 1.4{\times}10^{-4}$
$128^{2}$	ELBM	0/1	$0.281$	$0.546$	$0.981$	$\bf{0.869}$	$\bf{0.684}$	$\bf{0.647}$
$64^{2}$	NeurDE	4/4	$\bf{0.154\pm 5.7{\times}10^{-4}}$	$\bf{0.771\pm 3.9{\times}10^{-4}}$	$\bf{0.641\pm 3.7{\times}10^{-5}}$	$3.60\pm 9.6{\times}10^{-2}$	$\bf{0.0283\pm 0.0015}$	$0.0553\pm 0.0016$
$64^{2}$	ELBM	0/1	$0.392$	$0.542$	$1.03$	$\bf{0.438}$	$1.98$	$\bf{1.49}$
$32^{2}$	NeurDE	4/4	$\bf{0.304\pm 0.045}$	$\bf{0.684\pm 0.032}$	$\bf{0.730\pm 0.032}$	$3.72\pm 0.70$	$\bf{0.050\pm 0.046}$	$\bf{0.060\pm 0.029}$

Table 1: Doubly periodic shear layer at

\mathrm{Re}=100{,}000

: forecast metrics from the training data (

t_{0}=0

) at the final lead time. Entropically-stabilized NeurDE results are the mean

\pm

standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative

L^{2}

error of velocity and vorticity

\omega

, and vorticity correlation. Spectrum diagnostics: RMSE of

\log_{10}E(k)

over active shells, and high-

k

energy and palinstrophy relative to the reference.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	4/4	$0.0519\pm 3.3{\times}10^{-6}$	$\bf{0.937\pm 1.6{\times}10^{-5}}$	$\bf{0.353\pm 4.2{\times}10^{-5}}$	$4.34\pm 3.5{\times}10^{-3}$	$0.00635\pm 2.3{\times}10^{-5}$	$0.0868\pm 5.0{\times}10^{-5}$
$128^{2}$	ELBM	1/1	$\bf{0.0448}$	$0.920$	$0.401$	$\bf{0.257}$	$\bf{1.08}$	$\bf{1.02}$
$64^{2}$	NeurDE	4/4	$0.0956\pm 5.9{\times}10^{-4}$	$0.916\pm 3.3{\times}10^{-4}$	$0.421\pm 8.5{\times}10^{-4}$	$4.58\pm 0.11$	$0.00309\pm 0.00047$	$0.0780\pm 4.0{\times}10^{-4}$
$64^{2}$	ELBM	1/1	$\bf{0.0555}$	$\bf{0.920}$	$\bf{0.402}$	$\bf{0.346}$	$\bf{1.56}$	$\bf{1.28}$
$32^{2}$	NeurDE	4/4	$0.175\pm 0.019$	$0.870\pm 0.016$	$0.519\pm 0.031$	$3.75\pm 0.62$	$0.0040\pm 0.0044$	$0.081\pm 0.018$

Table 2: Doubly periodic shear layer at

\mathrm{Re}=100{,}000

: forecast metrics from the testing data (

t_{0}=T/2

) at the final lead time; columns as in Table 1 .

Figure 8: Long-rollout comparison on the Taylor Green Vortex at Re=50,000. Vorticity forecasts for the box-filtered high-resolution reference (top), entropy-stabilized NeurDE (middle), and an underresolved ELBM baseline (bottom) on a $64\times 64$ grid. This demonstrates that NeurDE does not smooth where it is unnecessary.

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9: Phase-space representation of a kinetic distribution. a Phase space $(\mathbf{x},\mathbf{v})$ at $t=t_{0}$ . The green region marks a spatial volume element containing particles; red particles indicate those whose velocities lie in a selected interval. b The corresponding velocity distribution function $f$ . The red area represents the particle count $N_{p}$ in the selected velocity interval, connecting microscopic particle states to the macroscopic distribution. Inspired by Boyd & Schunk Boyd and Schwartzentruber [2017] .

Layer	Input Dim	Output Dim	Kernel	Activation
Input GN	$d_{h}+4$	$d_{h}+4$	–	–
Conv1	$d_{h}+4$	$d_{h}$	$3\times 3$	–
GN1	$d_{h}$	$d_{h}$	–	–
Act1	$d_{h}$	$d_{h}$	–	GELU/ReLU
Conv2	$d_{h}$	$d_{h}$	$3\times 3$	–
GN2	$d_{h}$	$d_{h}$	–	–

Appendix

Table 3: Update Block layer specifications

Component	Hyperparameter	Value
Inputs	Channels	14: $f_{i}$ (9, D2Q9), $\lVert f^{\mathrm{neq}}\rVert_{2}$ (1), $\rho$ (1), $\mathbf{u}$ (2), $\omega$ (1)
	Preprocessing	$\operatorname{sign}(x)\,\log(1+10^{5}\lvert x\rvert)$
Encoder	Layers	2 conv $3\times 3$ : $14\to 64\to 64$ , ReLU
	Evaluations	once per collision step
Update block	Layers	5 conv: $3\times 3$ $68\to 64$ ; $2\times$ $3\times 3$ $64\to 64$ ; $3\times 3$ $64\to 32$ ; $1\times 1$ $32\to 1$ ; ReLU between
	Input	64 encoder + 1 $\tanh(y_{k})$ + 3 physics hints

Appendix

Table 4: NeurDE architecture hyperparameters, shared by every case, grid, and seed. The equilibrium is the analytic entropic one; the network predicts the ELBM relaxation parameter

\alpha

at each lattice site as the fixed point

y^{\star}

of a deep equilibrium (DEQ) model. All convolutions use zero padding of width one.

Panel of Figure 10 — Figure 10: Computational setup for Kelvin–Helmholtz instability simulation. The domain $[0,1]^{2}$ contains two counter-rotating shear layers at $y=0.25$ and $y=0.75$ with hyperbolic tangent velocity profiles (blue regions). A sinusoidal perturbation in the cross-stream velocity $u_{y}$ (red curves) seeds the instability. Green arrows indicate the streamwise velocity direction in each layer. Periodic boundary conditions are applied in both directions (gray dashed arrows).

		Field fidelity			Spectrum diagnostics
Grid	Method	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE $\downarrow$	High- $k$ /ref. $\to 1$
$64^{2}$	NeurDE	$0.0806$	$0.9387$	$2.383$	$3.794$	$0.0077$
	NeurDE shell	$\mathbf{0.0462}$	$\mathbf{0.9614}$	$\mathbf{0.275}$	$0.540$	$0.422$
	ELBM	$0.0544$	$0.9374$	$0.355$	$\mathbf{0.302}$	$\mathbf{1.415}$
$32^{2}$	NeurDE	$0.0686$	$\mathbf{0.9551}$	$\mathbf{0.2977}$	$\mathbf{0.2593}$	$0.6226$
	NeurDE shell	$\mathbf{0.0676}$	$0.9534$	$0.3021$	$0.2613$	$0.7478$

Appendix

Table 5: Final metrics of the fine-tuning ablation. Metrics at the final saved horizon of the

64^{2}

(

t_{\mathrm{index}}=2500

32^{2}

(

t_{\mathrm{index}}=1249

), and

16^{2}

(

t_{\mathrm{index}}=624

) rollouts, against the box-filtered reference. The first three columns measure field fidelity; the last two diagnose spectral behavior. Bold marks the best value within each grid block. For High-

k

/ref., best means closest to one, not smallest.

Figure 11: Core field-level fidelity for

64^{2}

forecasts. Entropically-stabilized NeurDE , NeurDE shell , and ELBM are compared against the box-filtered high-resolution reference. (a) Relative velocity error, (b) vorticity correlation, and (c) relative vorticity error over rollout time. Fine-tuning substantially reduces the velocity and vorticity errors of the base model and increases vorticity correlation over the later rollout.

Figure 12: Field snapshots for the $64^{2}$ rollout at $t_{\mathrm{index}}=2500$ . Columns compare the box-filtered reference, entropically-stabilized NeurDE , NeurDE shell , and ELBM; rows show density $\rho$ , speed $|\mathbf{u}|$ , and vorticity $\omega$ . The base entropically-stabilized NeurDE forecast is visibly over-smoothed in vorticity, while fine-tuning restores sharper vortex structure and better phase alignment with the reference. ELBM preserves the main vortices but shows excess small-scale texture in the fields.

Figure 13: Energy and enstrophy spectra at

t_{\mathrm{index}}=2500

. The base entropically-stabilized NeurDE forecast loses high-wavenumber energy and enstrophy, while NeurDE shell restores much of the missing spectral content. ELBM remains strongest on the final energy-spectrum log-RMSE, so the improvement should be read as improved field-level fidelity and reduced spectral collapse rather than better spectral accuracy on every metric.

Figure 14: Two-point correlations at

t_{\mathrm{index}}=2500

. Velocity correlations are similar across methods, but vorticity correlations expose the main failure of the base model: it preserves overly smooth long-range structure and does not reproduce the reference-like small-scale decorrelation. NeurDE shell and ELBM track the reference vorticity correlation much more closely.

Figure 15: Supplementary fidelity metrics for the

64^{2}

fine-tuning ablation. This figure reports the full collection of time-resolved fidelity metrics used to compare entropically-stabilized NeurDE , NeurDE shell , and ELBM against the filtered reference.

Figure 16: Supplementary flow statistics for the

64^{2}

fine-tuning ablation. The statistics summarize density, kinetic, vorticity, divergence, and derivative-shape diagnostics for the reference and each rollout.

Figure 17: Supplementary artifact diagnostics for the $64^{2}$ fine-tuning ablation. Fourier-space and artifact-oriented diagnostics expose how the base forecast loses high-wavenumber content and how fine-tuning recovers a larger fraction of the resolved reference spectral structure.

Figure 18: Supplementary two-point error evolution for the $64^{2}$ fine-tuning ablation. The curves track correlation-error evolution over rollout time and separate velocity-correlation errors from the more discriminating vorticity-correlation errors.

Panel of Figure 19 — Figure 19: Gradient accumulation in the Incremental Progress Training Algorithm (IPTA). By computing gradients over a sparse subset of timesteps, either deterministically (denser near the end) or stochastically (biased toward distant steps), IPTA removes the recency bias of truncated backpropagation and enables memory-efficient long-horizon training.

	Stability	$384^{2}$	$256^{2}$	$128^{2}$	$64^{2}$	$32^{2}$	$16^{2}$
TGV-50K	374.3	–	–	51.3	80.0	80	80
DPSL-50K	32.5	–	14.5	39.5	56.1	80	80
DPSL-100K	16.2	8.6	16.5	44.8	68.7	80	80

Appendix

Table 6: Epochs completed, mean over four seeds. Rollout training on a grid is complete at 80 epochs; lower values were stopped by the wall-time limit. Epochs shrink as the horizon grows, so counts are not comparable across stages.

Case	Stage	Wall time	Node-h	GCD-h	Steps
TGV-50K	Stability	1:57:19	1.96	15.64	69,611
	Solver $128^{2}$	6:00:34	6.01	48.08	3,657
	Solver $64^{2}$	1:50:48	1.85	14.78	1,736
	Solver $32^{2}$	0:39:30	0.66	5.27	787
	Solver $16^{2}$	0:11:14	0.19	1.50	373
	Total	10:39:27	10.66	85.26

Appendix

Table 7: Training budget of the base models, averaged over four seeds. All stages ran on one Frontier node (8 MI250X GCDs) with data-parallel training.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	3.5/4	$\bf{0.109\pm 0.044}$	$\bf{0.806\pm 0.083}$	$\bf{0.59\pm 0.12}$	$3.37\pm 0.88$	$0.051\pm 0.062$	$0.133\pm 0.080$
$128^{2}$	ELBM	0/1	$0.259$	$0.600$	$0.92$	$\bf{0.967}$	$\bf{0.48}$	$\bf{0.540}$
$64^{2}$	NeurDE	4/4	$\bf{0.144\pm 4.9{\times}10^{-3}}$	$\bf{0.793\pm 8.8{\times}10^{-3}}$	$\bf{0.614\pm 0.011}$	$3.15\pm 0.23$	$0.041\pm 0.012$	$0.0716\pm 0.0084$
$64^{2}$	ELBM	0/1	$0.381$	$0.582$	$0.961$	$\bf{0.479}$	$\bf{1.5}$	$\bf{1.20}$
$32^{2}$	NeurDE	4/4	$\bf{0.256\pm 0.019}$	$\bf{0.725\pm 0.017}$	$\bf{0.690\pm 0.018}$	$2.74\pm 0.20$	$\bf{0.098\pm 0.046}$	$\bf{0.093\pm 0.025}$

Appendix

Table 8: Real-space and spectral metrics for doubly periodic shear layer at

\mathrm{Re}=50{,}000

. We compare emulation metrics from the training data (

t_{0}=0

) at the final lead time against the downsampled

512^{2}

reference. Entropically-stabilized NeurDE results are the mean

\pm

standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative

L^{2}

error of velocity and vorticity

\omega

, and vorticity correlation. Spectrum diagnostics: RMSE of

\log_{10}E(k)

over active shells, and high-

k

energy and palinstrophy relative to the reference. Bold marks the better method per grid.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	4/4	$0.045\pm 0.011$	$0.959\pm 0.010$	$0.288\pm 0.039$	$3.64\pm 0.70$	$0.015\pm 0.024$	$0.141\pm 0.054$
$128^{2}$	ELBM	1/1	$\bf{0.021}$	$\bf{0.970}$	$\bf{0.244}$	$\bf{0.286}$	$\bf{0.96}$	$\bf{0.921}$
$64^{2}$	NeurDE	4/4	$0.0875\pm 0.0015$	$0.935\pm 9.9{\times}10^{-4}$	$0.375\pm 3.0{\times}10^{-3}$	$3.56\pm 0.12$	$0.0051\pm 0.0018$	$0.107\pm 2.4{\times}10^{-3}$
$64^{2}$	ELBM	1/1	$\bf{0.0548}$	$\bf{0.939}$	$\bf{0.349}$	$\bf{0.308}$	$\bf{1.4}$	$\bf{1.17}$
$32^{2}$	NeurDE	4/4	$0.146\pm 9.1{\times}10^{-3}$	$0.904\pm 6.8{\times}10^{-3}$	$0.456\pm 0.016$	$2.89\pm 0.23$	$0.0125\pm 0.0081$	$0.122\pm 0.013$

Appendix

Table 9: Real-space and spectral metrics for doubly periodic shear layer at

\mathrm{Re}=50{,}000

. We compare emulation metrics from the testing data (

t_{0}=T/2

) at the final lead time, against the downsampled

512^{2}

reference. Entropically-stabilized NeurDE results are the mean

\pm

standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative

L^{2}

error of velocity and vorticity

\omega

, and vorticity correlation. Spectrum diagnostics: RMSE of

\log_{10}E(k)

over active shells, and high-

k

energy and palinstrophy relative to the reference. Bold marks the better method per grid.

Panel of Figure 20 — Figure 20: Doubly periodic shear layer at $\mathrm{Re}=50{,}000$ : vorticity forecasts on grids (a) $128^{2}$ , (b) $64^{2}$ , (c) $32^{2}$ , and (d) $16^{2}$ ; the grid size is also marked to the left of each panel. In each panel, the rows are the downsampled $512^{2}$ reference (top), entropically-stabilized NeurDE (middle), and numerical ELBM run at that grid (bottom). The left block starts from the training data ( $t_{0}=0$ ), the right block from the testing data ( $t_{0}=T/2$ ); columns are lead times $\Delta t$ in solver steps.

Panel of Figure 21 — Figure 21: Doubly periodic shear layer at $\mathrm{Re}=50{,}000$ : shell-averaged spectra at the final lead time on grids (a) $128^{2}$ , (b) $64^{2}$ , (c) $32^{2}$ , and (d) $16^{2}$ . Columns are the energy $E(k)$ , enstrophy $Z(k)=k^{2}E(k)$ , and palinstrophy $P(k)=k^{4}E(k)$ spectra, where $k$ is the wavenumber shell; rows are forecasts from the training (top) and testing (bottom) data. Curves compare the downsampled $512^{2}$ reference, numerical ELBM, and entropically-stabilized NeurDE .

Figure 22: The percentage of pixels that violate the second law of thermodynamics over the NeurdE emulation of the doubly periodic shear layer at

\mathrm{Re}=50{,}000

. We show the training and testing datasets for all grids. The mean and standard deviation are taken over all seeds.

Panel of Figure 23 — Figure 23: Doubly periodic shear layer at $\mathrm{Re}=50{,}000$ , seed 576268 on the $128^{2}$ grid: (a) vorticity forecasts and (b) shell-averaged spectra at the final lead time, laid out as in Figs. 20 and 21 . The rollout stays finite, but its forecast from the training data departs from the other three seeds: at $\Delta t=2500$ its velocity error is $0.174$ (other seeds $0.080$ – $0.096$ ) and its vorticity correlation is $0.683$ ( $0.829$ – $0.862$ ). It also keeps more energy at high $k$ than the other seeds (high- $k$ /ref. $0.142$ against $0.007$ – $0.034$ ). Its forecast from the testing data is in line with, and on the field metrics slightly better than, the other seeds.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	4/4	$\bf{0.0807\pm 6.9{\times}10^{-5}}$	$\bf{0.846\pm 2.5{\times}10^{-4}}$	$\bf{0.536\pm 3.8{\times}10^{-4}}$	$3.28\pm 2.4{\times}10^{-3}$	$0.0454\pm 2.2{\times}10^{-4}$	$0.104\pm 1.4{\times}10^{-4}$
$128^{2}$	ELBM	0/1	$0.281$	$0.546$	$0.981$	$\bf{0.869}$	$\bf{0.684}$	$\bf{0.647}$
$64^{2}$	NeurDE	4/4	$\bf{0.154\pm 5.7{\times}10^{-4}}$	$\bf{0.771\pm 3.9{\times}10^{-4}}$	$\bf{0.641\pm 3.7{\times}10^{-5}}$	$3.60\pm 9.6{\times}10^{-2}$	$\bf{0.0283\pm 0.0015}$	$0.0553\pm 0.0016$
$64^{2}$	ELBM	0/1	$0.392$	$0.542$	$1.03$	$\bf{0.438}$	$1.98$	$\bf{1.49}$
$32^{2}$	NeurDE	4/4	$\bf{0.304\pm 0.045}$	$\bf{0.684\pm 0.032}$	$\bf{0.730\pm 0.032}$	$3.72\pm 0.70$	$\bf{0.050\pm 0.046}$	$\bf{0.060\pm 0.029}$

Appendix

Table 10: Real-space and spectral metrics for doubly periodic shear layer at

\mathrm{Re}=100{,}000

. We compare emulation metrics from the training data (

t_{0}=0

) at the final lead time against the downsampled

512^{2}

reference. Entropically-stabilized NeurDE results are the mean

\pm

standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative

L^{2}

error of velocity and vorticity

\omega

, and vorticity correlation. Spectrum diagnostics: RMSE of

\log_{10}E(k)

over active shells, and high-

k

energy and palinstrophy relative to the reference. Bold marks the better method per grid.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	4/4	$0.0519\pm 3.3{\times}10^{-6}$	$\bf{0.937\pm 1.6{\times}10^{-5}}$	$\bf{0.353\pm 4.2{\times}10^{-5}}$	$4.34\pm 3.5{\times}10^{-3}$	$0.00635\pm 2.3{\times}10^{-5}$	$0.0868\pm 5.0{\times}10^{-5}$
$128^{2}$	ELBM	1/1	$\bf{0.0448}$	$0.920$	$0.401$	$\bf{0.257}$	$\bf{1.08}$	$\bf{1.02}$
$64^{2}$	NeurDE	4/4	$0.0956\pm 5.9{\times}10^{-4}$	$0.916\pm 3.3{\times}10^{-4}$	$0.421\pm 8.5{\times}10^{-4}$	$4.58\pm 0.11$	$0.00309\pm 0.00047$	$0.0780\pm 4.0{\times}10^{-4}$
$64^{2}$	ELBM	1/1	$\bf{0.0555}$	$\bf{0.920}$	$\bf{0.402}$	$\bf{0.346}$	$\bf{1.56}$	$\bf{1.28}$
$32^{2}$	NeurDE	4/4	$0.175\pm 0.019$	$0.870\pm 0.016$	$0.519\pm 0.031$	$3.75\pm 0.62$	$0.0040\pm 0.0044$	$0.081\pm 0.018$

Appendix

Table 11: Real-space and spectral metrics for doubly periodic shear layer at

\mathrm{Re}=100{,}000

. We compare emulation metrics from the testing data (

t_{0}=T/2

) at the final lead time against the downsampled

512^{2}

reference. Entropically-stabilized NeurDE results are the mean

\pm

standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative

L^{2}

error of velocity and vorticity

\omega

, and vorticity correlation. Spectrum diagnostics: RMSE of

\log_{10}E(k)

over active shells, and high-

k

energy and palinstrophy relative to the reference. Bold marks the better method per grid.

Panel of Figure 24 — Figure 24: Doubly periodic shear layer at $\mathrm{Re}=100{,}000$ : vorticity forecasts on grids (a) $128^{2}$ , (b) $64^{2}$ , (c) $32^{2}$ , and (d) $16^{2}$ ; the grid size is also marked to the left of each panel. In each panel, the rows are the downsampled $512^{2}$ reference (top), entropically-stabilized NeurDE (middle), and numerical ELBM run at that grid (bottom). The left block starts from the training data ( $t_{0}=0$ ), the right block from the testing data ( $t_{0}=T/2$ ); columns are lead times $\Delta t$ in solver steps.

Panel of Figure 25 — Figure 25: Doubly periodic shear layer at $\mathrm{Re}=100{,}000$ : shell-averaged spectra at the final lead time on grids (a) $128^{2}$ , (b) $64^{2}$ , (c) $32^{2}$ , and (d) $16^{2}$ . Columns are the energy $E(k)$ , enstrophy $Z(k)=k^{2}E(k)$ , and palinstrophy $P(k)=k^{4}E(k)$ spectra, where $k$ is the wavenumber shell; rows are forecasts from the training (top) and testing (bottom) data. Curves compare the downsampled $512^{2}$ reference, numerical ELBM, and entropically-stabilized NeurDE .

Figure 26: The percentage of pixels that violate the second law of thermodynamics over the NeurdE emulation of the doubly periodic shear layer at

\mathrm{Re}=100{,}000

. We show the training and testing datasets for all grids. The mean and standard deviation are taken over all seeds.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	4/4	$(6.14\pm 0.81){\times}10^{-4}$	$1.00\pm 1.4{\times}10^{-9}$	$(6.23\pm 0.82){\times}10^{-4}$	–	–	$1.00\pm 1.7{\times}10^{-4}$
$128^{2}$	ELBM	1/1	$\bf{2.53{\times}10^{-4}}$	$1.00$	$\bf{2.58{\times}10^{-4}}$	–	–	$1.00$
$64^{2}$	NeurDE	4/4	$0.00209\pm 0.00049$	$1.00\pm 3.5{\times}10^{-8}$	$0.00213\pm 0.00046$	–	–	$0.996\pm 1.0{\times}10^{-3}$
$64^{2}$	ELBM	1/1	$\bf{0.00162}$	$1.00$	$\bf{0.00179}$	–	–	$\bf{0.997}$
$32^{2}$	NeurDE	4/4	$0.00954\pm 6.8{\times}10^{-5}$	$1.00\pm 3.7{\times}10^{-9}$	$0.00951\pm 6.7{\times}10^{-5}$	–	–	$0.981\pm 1.4{\times}10^{-4}$

Appendix

Table 12: Real-space and spectral metrics for TGV at

\mathrm{Re}=50{,}000

. We compare emulation metrics from the training data (

t_{0}=0

) at the final lead time against the downsampled

512^{2}

reference. Entropically-stabilized NeurDE results are the mean

\pm

standard deviation over four seeds, ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative

L^{2}

error of velocity and vorticity

\omega

, and vorticity correlation. Spectrum diagnostics: RMSE of

\log_{10}E(k)

over active shells, and high-

k

energy and palinstrophy relative to the reference. Bold marks the better method per grid.

Grid	Method	Faithful Emulation	Field Fidelity			Spectrum Diagnostics
Grid	Method	Faithful Emulation	Vel. err $\downarrow$	$\omega$ corr. $\uparrow$	$\omega$ err $\downarrow$	Spec. RMSE † $\downarrow$	High- $k$ /ref. $\to 1$	$P$ /ref. $\to 1$
$128^{2}$	NeurDE	4/4	$(5.62\pm 0.79){\times}10^{-4}$	$1.00\pm 2.5{\times}10^{-9}$	$(5.76\pm 0.81){\times}10^{-4}$	–	–	$1.00\pm 1.7{\times}10^{-4}$
$128^{2}$	ELBM	1/1	$\bf{3.14{\times}10^{-4}}$	$1.00$	$\bf{3.18{\times}10^{-4}}$	–	–	$\bf{0.999}$
$64^{2}$	NeurDE	4/4	$0.00225\pm 0.00048$	$1.00\pm 3.9{\times}10^{-8}$	$0.00227\pm 0.00046$	–	–	$0.996\pm 1.0{\times}10^{-3}$
$64^{2}$	ELBM	1/1	$\bf{0.00180}$	$1.00$	$\bf{0.00197}$	–	–	$\bf{0.997}$
$32^{2}$	NeurDE	4/4	$0.00970\pm 6.8{\times}10^{-5}$	$1.00\pm 3.4{\times}10^{-9}$	$0.00971\pm 6.8{\times}10^{-5}$	–	–	$0.981\pm 1.4{\times}10^{-4}$

Appendix

Table 13: Real-space and spectral metrics for TGV at

\mathrm{Re}=50{,}000

. We compare emulation metrics from the testing data (

t_{0}=T/2

) at the final lead time against the downsampled

512^{2}

reference. Entropically-stabilized NeurDE results are the mean

\pm

standard deviation over four seeds, ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative

L^{2}

error of velocity and vorticity

\omega

, and vorticity correlation. Spectrum diagnostics: RMSE of

\log_{10}E(k)

over active shells, and high-

k

energy and palinstrophy relative to the reference. Bold marks the better method per grid.

Panel of Figure 27 — Figure 27: TGV velocity streamline emulations at $\mathrm{Re}=50{,}000$ , colored by speed on grids (a) $128^{2}$ , (b) $64^{2}$ , (c) $32^{2}$ , and (d) $16^{2}$ ; the grid size is also marked to the left of each panel. In each panel, the rows are the downsampled $512^{2}$ reference (top), entropically-stabilized NeurDE (middle), and numerical ELBM run at that grid (bottom). The left block starts from the training data ( $t_{0}=0$ ), the right block from the testing data ( $t_{0}=T/2$ ); columns are lead times $\Delta t$ in solver steps.

Panel of Figure 28 — Figure 28: TGV at $\mathrm{Re}=50{,}000$ shell-averaged spectra at the final lead time on grids (a) $128^{2}$ , (b) $64^{2}$ , (c) $32^{2}$ , and (d) $16^{2}$ . Columns are the energy $E(k)$ , enstrophy $Z(k)=k^{2}E(k)$ , and palinstrophy $P(k)=k^{4}E(k)$ spectra, where $k$ is the wavenumber shell; rows are forecasts from the training (top) and testing (bottom) data. Curves compare the downsampled $512^{2}$ reference, numerical ELBM, and entropically-stabilized NeurDE . The flow stays in a few low- $k$ shells, so the high- $k$ tail is at roundoff level.

Figure 29: The percentage of pixels that violate the second law of thermodynamics over the NeurdE emulation of the Taylor Green vortex experiment at

\mathrm{Re}=50{,}000

. We show the training and testing datasets for all grids. The mean and standard deviation are taken over all seeds.

Hyperparameter	Value
Spatial dimensions	2 ( $256\times 256$ periodic grid)
Input / output channels	4 (density, $u_{x}$ , $u_{y}$ , vorticity)
Fourier modes retained	64 per dimension ( $64\times 33$ after the real FFT)
Hidden channels (width)	128
Fourier layers	4
Lifting / projection channels	256 / 256

Appendix

Table 14: FNO architecture.

Hyperparameter	Values swept	Final run
Hidden channels	128, 256	128
Global batch size	16, 32, 64	16
OneCycle initial divisor ( div_factor )	5, 25, 100	25
OneCycle final divisor ( final_div_factor )	100, 1000	100

Appendix

Table 15: Sweep grid and the values used in the final run.

Data
Hyperparameter	Value
Simulation	D2Q9 weakly compressible isothermal LBM, $\mathrm{Re}=5\times 10^{4}$ , $\mathrm{Ma}=0.1$
Trajectory	one trajectory, $256\times 256$ grid, $10^{4}$ saved steps
Train / validation / test split	50 % / 10 % / 40 %, contiguous in time
Inputs and targets	density, velocity, vorticity
Loss and optimizer

Appendix

Table 16: FNO training setup.

Quantity	Value
System	NERSC Perlmutter, 1 GPU node
GPU type	NVIDIA A100-SXM4, 40 GB
Number of GPUs	4
CPU cores allocated	64 physical (128 hardware threads)
Job wall-clock time	23.96 h (24 h limit)
Training time (epoch 0 to epoch 182)	23.67 h

Appendix

Table 17: Compute used to train the final FNO.

Rollout length	Epochs	Wall-clock (h)	Minutes per epoch
1	25	0.30	0.7
2	25	0.42	1.0
3	25	0.53	1.3
5	22	0.71	1.9
10	16	1.01	3.8
25	40	10.02	15.0

Appendix

Table 18: Wall-clock time per curriculum stage (4 GPUs).

Figure 30: Diverging vorticity decay from DNS as forecast length increases.

Quantity	Value
System	NERSC Perlmutter, 2 GPU nodes
GPU type	NVIDIA A100-SXM4, 80 GB
Number of GPUs	8 (4 per node)
CPU cores allocated	128 physical (256 hardware threads)
Total wall-clock time	38.11 h
GPU-hours	304.8

Appendix

Table 19: Compute used to fine-tune Walrus.

PDE Surrogate Modeling

PDE surrogate modeling builds fast approximations of partial differential equation solutions for simulation, prediction, and design.

See all 347 papers

Oct 30, 2023

Gradient-free online learning of subgrid-scale dynamics with neural emulators

In this paper, we propose a generic algorithm to train machine learning-based subgrid parametrizations online, i.e., with

\textit{a posteriori}

loss functions, but for non-differentiable numerical solvers. The proposed approach leverages a neural emulator to approximate the reduced state-space solver, which is then used to allow gradient propagation through temporal integration steps. We apply this methodology on a chaotic two-timescales Lorenz-96 system and a single layer quasi-geostrophic system with zonal dynamics, known to be highly unstable with offline strategies. Using our algorithm, we are able to train a parametrization that recovers most of the benefits of online strategies without having to compute the gradient of the original solver. We found that training the neural emulator and parametrization components separately with different loss quantities is necessary in order to minimize the propagation of approximation biases. Experiments on emulator architectures with different complexities also indicates that emulator performance is key in order to learn an accurate parametrization. This work is a step towards learning parametrization with online strategies for climate models.

Hugo Frezat, Ronan Fablet, Guillaume Balarac +1

Univ. Grenoble-Alpes, CNRS, Grenoble INP, LEGI, Grenoble, France · Univ. Grenoble-Alpes, CNRS, INRAE, IRD, Grenoble INP, IGE, Grenoble, France · IMT Atlantique, CNRS, Lab-STICC, Brest, France +2

Climate Modeling Gradient-Free Optimization Computational Fluid Dynamics

Ask PDF

Jul 16, 2026

Subgrid-Scale Parameterization in Burgers' Equation Using Structure-Preserving Neural Networks and Entropy Variables

We present a machine learning approach for developing subgrid-scale (SGS) parametrizations in coarse simulations of partial differential equations. We utilize structure-preserving neural networks and entropy variables to learn subgrid fluxes in coarse simulations of the Burgers' equation. In particular, we employ a decoupled neural network architecture explicitly separating the subgrid corrections into two distinct components: a conservative Flux Potential network and an Eddy Viscosity network. We demonstrate that this reduced-order framework maintains high physical fidelity, accurately reproducing the energy spectrum, spatial and temporal correlation functions, and dynamical characteristics of the full-scale system. Furthermore, we show that our approach is robust and applicable to parameters outside the training regime.

Aijaz Nazir, Ilya Timofeyev

Dept. of Mathematics, University of Houston, Houston, TX 77204

Neural Surrogate Modeling PDE Solving Structure-Preserving Neural Networks Computational Fluid Dynamics

Ask PDF

Oct 14, 2026

The Physics Is the Network: Fast, Training-Free Neural Emulation

Neural emulation need not require learning a substitute for known physics. We investigate PolyNets that couple a continuous-time neural representation of polynomial dynamics with a simple recurrent network for discrete-time integration. When the governing coefficients are known, the resulting tensor graph requires no training and preserves the prescribed numerical scheme while enabling efficient hardware execution and parallel simulation. Using the one-dimensional Korteweg-de Vries equation, we compare PolyNet with CPU and GPU finite-difference Runge-Kutta solvers and three trained neural models: a state-map emulator, a physics-informed predictor, and a learned-tendency integrator. Analytical soliton solutions provide a reference for amplitude and phase errors across grid resolutions, 10^6 dispersively scaled time steps, ensembles of up to 10^4 trajectories, and varying initial amplitudes. PolyNet achieves accuracy comparable to the numerical solvers, including at amplitudes outside the learned models' training conditions. It combines this fidelity with high GPU throughput, approximately constant over the tested resolution range, and favorable ensemble scaling.

Luka Todorovic, Kaushal Gianchandani, Sai Ravela

Department of Mathematics and Department of Statistics, University of Michigan, Ann Arbor, MI · NASA Postdoctoral Program Fellow, NASA Goddard Institute for Space Studies, New York, NY · Department of Earth, Atmospheric, and Planetary Sciences, Massachusetts Institute of Technology, Cambridge, MA

Neural PDE Solvers Neural Surrogate Modeling Physics-Based Simulation Polynomial Neural Networks Efficient Neural Network Inference

Ask PDF

Abstract

PDE Surrogate Modeling

Explore similar work

Gradient-free online learning of subgrid-scale dynamics with neural emulators

Subgrid-Scale Parameterization in Burgers' Equation Using Structure-Preserving Neural Networks and Entropy Variables

The Physics Is the Network: Fast, Training-Free Neural Emulation