cs.LGNov 2, 2026

Learning the Entropic Relaxation: Stable Coarse-Grid Emulation with Neural Discrete Equilibrium

Authors: Jose Antonio Lara Benitez, Kareem Hegazy, Anastasis Kratsios, Ivan Dokmanić, Maarten V. de Hoop, Michael W. Mahoney

Organizations: Rice University and LBNL · ICSI and University of California at Berkeley · McMaster University and the Vector Institute · University of Basel · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The ability to emulate complicated and computationally-expensive physical systems in an efficient and stable manner is a popular motivation for scientific machine learning (SciML). Stable numerical solvers tend to be computationally expensive, as they attempt to resolve small (spatial and temporal) scales that receive and dissipate spectral energy. Learned coarse-grid forecasters remove the dissipative channel, allowing unresolved frequencies to re-emerge as grid-scale noise (aliasing). In this paper, we use the Neural Discrete Equilibrium (NeurDE) method to explore stable emulation at extremely coarse grids, demonstrating physically-principled control over entropy dynamics that reinforces the physical system's dissipative mechanism. Entropically-stabilized NeurDE produces stable long-time rollouts for thousands of forecasting steps on large Reynolds number turbulent flows (Re=100,000\mathrm{Re}=100{,}000) with very coarse grids (down to R16×16\mathbb{R}^{16\times16}). At these very coarse grid scales, entropically-stabilized NeurDE can be more stable than even our stabilized data-generation numerical method.

Figures & tables

Figure 1: Coarse discretization interrupts scale-to-scale energy transfer. Solid curves show the resolved spectral power at successive stages of a rollout; dashed lines indicate an idealized inertial-range decay. As turbulence develops, a coarse representation loses the high-wavenumber dissipative tail and can accumulate grid-scale content, unless the unresolved transfer is modeled. The learned entropic relaxation supplies a local dissipative channel for this missing transfer.
Figure 2: Architecture of entropically-stabilized NeurDE . a The pipeline decouples feature extraction from the iterative DEQ solver. Context z\mathbf{z} is encoded once and injected into every iteration. b The neural update block processes the concatenated context, state, and physics hints through a convolutional network to refine the stability parameter α\alpha . c Physics hints include: entropy residual G=H(α)−HpreG=H(\alpha)-H_{\mathrm{pre}} , gradients Δ=H′(α)\Delta=H^{\prime}(\alpha) , and instability indicator. The physics hints are computed outside of the computational graph, ensuring gradients flow only through gθg_{\theta} (implicit differentiation).
Figure 3
Figure 5: Long-rollout comparison on the doubly periodic shear layer at Re=100,000. Vorticity forecasts for the box-filtered high-resolution reference (top), entropy-stabilized NeurDE (middle), and an underresolved ELBM baseline (bottom) on (a) a 64×6464\times 64 grid and (b) a 16×1616\times 16 grid.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE4/40.0807±6.9×10−5\bf{0.0807\pm 6.9{\times}10^{-5}}0.846±2.5×10−4\bf{0.846\pm 2.5{\times}10^{-4}}0.536±3.8×10−4\bf{0.536\pm 3.8{\times}10^{-4}}3.28±2.4×10−33.28\pm 2.4{\times}10^{-3}0.0454±2.2×10−40.0454\pm 2.2{\times}10^{-4}0.104±1.4×10−40.104\pm 1.4{\times}10^{-4}
ELBM0/10.2810.2810.5460.5460.9810.9810.869\bf{0.869}0.684\bf{0.684}0.647\bf{0.647}
64264^{2}NeurDE4/40.154±5.7×10−4\bf{0.154\pm 5.7{\times}10^{-4}}0.771±3.9×10−4\bf{0.771\pm 3.9{\times}10^{-4}}0.641±3.7×10−5\bf{0.641\pm 3.7{\times}10^{-5}}3.60±9.6×10−23.60\pm 9.6{\times}10^{-2}0.0283±0.0015\bf{0.0283\pm 0.0015}0.0553±0.00160.0553\pm 0.0016
ELBM0/10.3920.3920.5420.5421.031.030.438\bf{0.438}1.981.981.49\bf{1.49}
32232^{2}NeurDE4/40.304±0.045\bf{0.304\pm 0.045}0.684±0.032\bf{0.684\pm 0.032}0.730±0.032\bf{0.730\pm 0.032}3.72±0.703.72\pm 0.700.050±0.046\bf{0.050\pm 0.046}0.060±0.029\bf{0.060\pm 0.029}
Table 1: Doubly periodic shear layer at Re=100,000\mathrm{Re}=100{,}000 : forecast metrics from the training data ( t0=0t_{0}=0 ) at the final lead time. Entropically-stabilized NeurDE results are the mean ±\pm standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative L2L^{2} error of velocity and vorticity ω\omega , and vorticity correlation. Spectrum diagnostics: RMSE of log⁡10E(k)\log_{10}E(k) over active shells, and high- kk energy and palinstrophy relative to the reference.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE4/40.0519±3.3×10−60.0519\pm 3.3{\times}10^{-6}0.937±1.6×10−5\bf{0.937\pm 1.6{\times}10^{-5}}0.353±4.2×10−5\bf{0.353\pm 4.2{\times}10^{-5}}4.34±3.5×10−34.34\pm 3.5{\times}10^{-3}0.00635±2.3×10−50.00635\pm 2.3{\times}10^{-5}0.0868±5.0×10−50.0868\pm 5.0{\times}10^{-5}
ELBM1/10.0448\bf{0.0448}0.9200.9200.4010.4010.257\bf{0.257}1.08\bf{1.08}1.02\bf{1.02}
64264^{2}NeurDE4/40.0956±5.9×10−40.0956\pm 5.9{\times}10^{-4}0.916±3.3×10−40.916\pm 3.3{\times}10^{-4}0.421±8.5×10−40.421\pm 8.5{\times}10^{-4}4.58±0.114.58\pm 0.110.00309±0.000470.00309\pm 0.000470.0780±4.0×10−40.0780\pm 4.0{\times}10^{-4}
ELBM1/10.0555\bf{0.0555}0.920\bf{0.920}0.402\bf{0.402}0.346\bf{0.346}1.56\bf{1.56}1.28\bf{1.28}
32232^{2}NeurDE4/40.175±0.0190.175\pm 0.0190.870±0.0160.870\pm 0.0160.519±0.0310.519\pm 0.0313.75±0.623.75\pm 0.620.0040±0.00440.0040\pm 0.00440.081±0.0180.081\pm 0.018
Table 2: Doubly periodic shear layer at Re=100,000\mathrm{Re}=100{,}000 : forecast metrics from the testing data ( t0=T/2t_{0}=T/2 ) at the final lead time; columns as in Table 1 .
Figure 7
Figure 8: Long-rollout comparison on the Taylor Green Vortex at Re=50,000. Vorticity forecasts for the box-filtered high-resolution reference (top), entropy-stabilized NeurDE (middle), and an underresolved ELBM baseline (bottom) on a 64×6464\times 64 grid. This demonstrates that NeurDE does not smooth where it is unnecessary.
Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Phase-space representation of a kinetic distribution. a Phase space (x,v)(\mathbf{x},\mathbf{v}) at t=t0t=t_{0} . The green region marks a spatial volume element containing particles; red particles indicate those whose velocities lie in a selected interval. b The corresponding velocity distribution function ff . The red area represents the particle count NpN_{p} in the selected velocity interval, connecting microscopic particle states to the macroscopic distribution. Inspired by Boyd & Schunk Boyd and Schwartzentruber [2017] .
LayerInput DimOutput DimKernelActivation
Input GNdh+4d_{h}+4dh+4d_{h}+4––
Conv1dh+4d_{h}+4dhd_{h}3×33\times 3–
GN1dhd_{h}dhd_{h}––
Act1dhd_{h}dhd_{h}–GELU/ReLU
Conv2dhd_{h}dhd_{h}3×33\times 3–
GN2dhd_{h}dhd_{h}––
Appendix
Table 3: Update Block layer specifications
ComponentHyperparameterValue
InputsChannels14: fif_{i} (9, D2Q9), ∥fneq∥2\lVert f^{\mathrm{neq}}\rVert_{2} (1), ρ\rho (1), u\mathbf{u} (2), ω\omega (1)
 Preprocessingsign⁡(x) log⁡(1+105∣x∣)\operatorname{sign}(x)\,\log(1+10^{5}\lvert x\rvert)
EncoderLayers2 conv 3×33\times 3 : 14→64→6414\to 64\to 64 , ReLU
 Evaluationsonce per collision step
Update blockLayers5 conv: 3×33\times 3 68→6468\to 64 ; 2×2\times 3×33\times 3 64→6464\to 64 ; 3×33\times 3 64→3264\to 32 ; 1×11\times 1 32→132\to 1 ; ReLU between
 Input64 encoder + 1 tanh⁡(yk)\tanh(y_{k}) + 3 physics hints
Appendix
Table 4: NeurDE architecture hyperparameters, shared by every case, grid, and seed. The equilibrium is the analytic entropic one; the network predicts the ELBM relaxation parameter α\alpha at each lattice site as the fixed point y⋆y^{\star} of a deep equilibrium (DEQ) model. All convolutions use zero padding of width one.
Figure 10: Computational setup for Kelvin–Helmholtz instability simulation. The domain [0,1]2[0,1]^{2} contains two counter-rotating shear layers at y=0.25y=0.25 and y=0.75y=0.75 with hyperbolic tangent velocity profiles (blue regions). A sinusoidal perturbation in the cross-stream velocity uyu_{y} (red curves) seeds the instability. Green arrows indicate the streamwise velocity direction in each layer. Periodic boundary conditions are applied in both directions (gray dashed arrows).
  Field fidelitySpectrum diagnostics
GridMethodVel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE ↓\downarrowHigh- kk /ref. →1\to 1
64264^{2}NeurDE0.08060.08060.93870.93872.3832.3833.7943.7940.00770.0077
NeurDE shell0.0462\mathbf{0.0462}0.9614\mathbf{0.9614}0.275\mathbf{0.275}0.5400.5400.4220.422
ELBM0.05440.05440.93740.93740.3550.3550.302\mathbf{0.302}1.415\mathbf{1.415}
32232^{2}NeurDE0.06860.06860.9551\mathbf{0.9551}0.2977\mathbf{0.2977}0.2593\mathbf{0.2593}0.62260.6226
NeurDE shell0.0676\mathbf{0.0676}0.95340.95340.30210.30210.26130.26130.74780.7478
Appendix
Table 5: Final metrics of the fine-tuning ablation. Metrics at the final saved horizon of the 64264^{2} ( tindex=2500t_{\mathrm{index}}=2500 ), 32232^{2} ( tindex=1249t_{\mathrm{index}}=1249 ), and 16216^{2} ( tindex=624t_{\mathrm{index}}=624 ) rollouts, against the box-filtered reference. The first three columns measure field fidelity; the last two diagnose spectral behavior. Bold marks the best value within each grid block. For High- kk /ref., best means closest to one, not smallest.
Figure 11: Core field-level fidelity for 64264^{2} forecasts. Entropically-stabilized NeurDE , NeurDE shell , and ELBM are compared against the box-filtered high-resolution reference. (a) Relative velocity error, (b) vorticity correlation, and (c) relative vorticity error over rollout time. Fine-tuning substantially reduces the velocity and vorticity errors of the base model and increases vorticity correlation over the later rollout.
Figure 12: Field snapshots for the 64264^{2} rollout at tindex=2500t_{\mathrm{index}}=2500 . Columns compare the box-filtered reference, entropically-stabilized NeurDE , NeurDE shell , and ELBM; rows show density ρ\rho , speed ∣u∣|\mathbf{u}| , and vorticity ω\omega . The base entropically-stabilized NeurDE forecast is visibly over-smoothed in vorticity, while fine-tuning restores sharper vortex structure and better phase alignment with the reference. ELBM preserves the main vortices but shows excess small-scale texture in the fields.
Figure 13: Energy and enstrophy spectra at tindex=2500t_{\mathrm{index}}=2500 . The base entropically-stabilized NeurDE forecast loses high-wavenumber energy and enstrophy, while NeurDE shell restores much of the missing spectral content. ELBM remains strongest on the final energy-spectrum log-RMSE, so the improvement should be read as improved field-level fidelity and reduced spectral collapse rather than better spectral accuracy on every metric.
Figure 14: Two-point correlations at tindex=2500t_{\mathrm{index}}=2500 . Velocity correlations are similar across methods, but vorticity correlations expose the main failure of the base model: it preserves overly smooth long-range structure and does not reproduce the reference-like small-scale decorrelation. NeurDE shell and ELBM track the reference vorticity correlation much more closely.
Figure 15: Supplementary fidelity metrics for the 64264^{2} fine-tuning ablation. This figure reports the full collection of time-resolved fidelity metrics used to compare entropically-stabilized NeurDE , NeurDE shell , and ELBM against the filtered reference.
Figure 16: Supplementary flow statistics for the 64264^{2} fine-tuning ablation. The statistics summarize density, kinetic, vorticity, divergence, and derivative-shape diagnostics for the reference and each rollout.
Figure 17: Supplementary artifact diagnostics for the 64264^{2} fine-tuning ablation. Fourier-space and artifact-oriented diagnostics expose how the base forecast loses high-wavenumber content and how fine-tuning recovers a larger fraction of the resolved reference spectral structure.
Figure 18: Supplementary two-point error evolution for the 64264^{2} fine-tuning ablation. The curves track correlation-error evolution over rollout time and separate velocity-correlation errors from the more discriminating vorticity-correlation errors.
Figure 19: Gradient accumulation in the Incremental Progress Training Algorithm (IPTA). By computing gradients over a sparse subset of timesteps, either deterministically (denser near the end) or stochastically (biased toward distant steps), IPTA removes the recency bias of truncated backpropagation and enables memory-efficient long-horizon training.
 Stability3842384^{2}2562256^{2}1282128^{2}64264^{2}32232^{2}16216^{2}
TGV-50K374.3––51.380.08080
DPSL-50K32.5–14.539.556.18080
DPSL-100K16.28.616.544.868.78080
Appendix
Table 6: Epochs completed, mean over four seeds. Rollout training on a grid is complete at 80 epochs; lower values were stopped by the wall-time limit. Epochs shrink as the horizon grows, so counts are not comparable across stages.
CaseStageWall timeNode-hGCD-hSteps
TGV-50KStability1:57:191.9615.6469,611
Solver 1282128^{2}6:00:346.0148.083,657
Solver 64264^{2}1:50:481.8514.781,736
Solver 32232^{2}0:39:300.665.27787
Solver 16216^{2}0:11:140.191.50373
Total10:39:2710.6685.26 
Appendix
Table 7: Training budget of the base models, averaged over four seeds. All stages ran on one Frontier node (8 MI250X GCDs) with data-parallel training.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE3.5/40.109±0.044\bf{0.109\pm 0.044}0.806±0.083\bf{0.806\pm 0.083}0.59±0.12\bf{0.59\pm 0.12}3.37±0.883.37\pm 0.880.051±0.0620.051\pm 0.0620.133±0.0800.133\pm 0.080
ELBM0/10.2590.2590.6000.6000.920.920.967\bf{0.967}0.48\bf{0.48}0.540\bf{0.540}
64264^{2}NeurDE4/40.144±4.9×10−3\bf{0.144\pm 4.9{\times}10^{-3}}0.793±8.8×10−3\bf{0.793\pm 8.8{\times}10^{-3}}0.614±0.011\bf{0.614\pm 0.011}3.15±0.233.15\pm 0.230.041±0.0120.041\pm 0.0120.0716±0.00840.0716\pm 0.0084
ELBM0/10.3810.3810.5820.5820.9610.9610.479\bf{0.479}1.5\bf{1.5}1.20\bf{1.20}
32232^{2}NeurDE4/40.256±0.019\bf{0.256\pm 0.019}0.725±0.017\bf{0.725\pm 0.017}0.690±0.018\bf{0.690\pm 0.018}2.74±0.202.74\pm 0.200.098±0.046\bf{0.098\pm 0.046}0.093±0.025\bf{0.093\pm 0.025}
Appendix
Table 8: Real-space and spectral metrics for doubly periodic shear layer at Re=50,000\mathrm{Re}=50{,}000 . We compare emulation metrics from the training data ( t0=0t_{0}=0 ) at the final lead time against the downsampled 5122512^{2} reference. Entropically-stabilized NeurDE results are the mean ±\pm standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative L2L^{2} error of velocity and vorticity ω\omega , and vorticity correlation. Spectrum diagnostics: RMSE of log⁡10E(k)\log_{10}E(k) over active shells, and high- kk energy and palinstrophy relative to the reference. Bold marks the better method per grid.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE4/40.045±0.0110.045\pm 0.0110.959±0.0100.959\pm 0.0100.288±0.0390.288\pm 0.0393.64±0.703.64\pm 0.700.015±0.0240.015\pm 0.0240.141±0.0540.141\pm 0.054
ELBM1/10.021\bf{0.021}0.970\bf{0.970}0.244\bf{0.244}0.286\bf{0.286}0.96\bf{0.96}0.921\bf{0.921}
64264^{2}NeurDE4/40.0875±0.00150.0875\pm 0.00150.935±9.9×10−40.935\pm 9.9{\times}10^{-4}0.375±3.0×10−30.375\pm 3.0{\times}10^{-3}3.56±0.123.56\pm 0.120.0051±0.00180.0051\pm 0.00180.107±2.4×10−30.107\pm 2.4{\times}10^{-3}
ELBM1/10.0548\bf{0.0548}0.939\bf{0.939}0.349\bf{0.349}0.308\bf{0.308}1.4\bf{1.4}1.17\bf{1.17}
32232^{2}NeurDE4/40.146±9.1×10−30.146\pm 9.1{\times}10^{-3}0.904±6.8×10−30.904\pm 6.8{\times}10^{-3}0.456±0.0160.456\pm 0.0162.89±0.232.89\pm 0.230.0125±0.00810.0125\pm 0.00810.122±0.0130.122\pm 0.013
Appendix
Table 9: Real-space and spectral metrics for doubly periodic shear layer at Re=50,000\mathrm{Re}=50{,}000 . We compare emulation metrics from the testing data ( t0=T/2t_{0}=T/2 ) at the final lead time, against the downsampled 5122512^{2} reference. Entropically-stabilized NeurDE results are the mean ±\pm standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative L2L^{2} error of velocity and vorticity ω\omega , and vorticity correlation. Spectrum diagnostics: RMSE of log⁡10E(k)\log_{10}E(k) over active shells, and high- kk energy and palinstrophy relative to the reference. Bold marks the better method per grid.
Figure 20: Doubly periodic shear layer at Re=50,000\mathrm{Re}=50{,}000 : vorticity forecasts on grids (a) 1282128^{2} , (b) 64264^{2} , (c) 32232^{2} , and (d) 16216^{2} ; the grid size is also marked to the left of each panel. In each panel, the rows are the downsampled 5122512^{2} reference (top), entropically-stabilized NeurDE (middle), and numerical ELBM run at that grid (bottom). The left block starts from the training data ( t0=0t_{0}=0 ), the right block from the testing data ( t0=T/2t_{0}=T/2 ); columns are lead times Δt\Delta t in solver steps.
Figure 21: Doubly periodic shear layer at Re=50,000\mathrm{Re}=50{,}000 : shell-averaged spectra at the final lead time on grids (a) 1282128^{2} , (b) 64264^{2} , (c) 32232^{2} , and (d) 16216^{2} . Columns are the energy E(k)E(k) , enstrophy Z(k)=k2E(k)Z(k)=k^{2}E(k) , and palinstrophy P(k)=k4E(k)P(k)=k^{4}E(k) spectra, where kk is the wavenumber shell; rows are forecasts from the training (top) and testing (bottom) data. Curves compare the downsampled 5122512^{2} reference, numerical ELBM, and entropically-stabilized NeurDE .
Figure 22: The percentage of pixels that violate the second law of thermodynamics over the NeurdE emulation of the doubly periodic shear layer at Re=50,000\mathrm{Re}=50{,}000 . We show the training and testing datasets for all grids. The mean and standard deviation are taken over all seeds.
Figure 23: Doubly periodic shear layer at Re=50,000\mathrm{Re}=50{,}000 , seed 576268 on the 1282128^{2} grid: (a) vorticity forecasts and (b) shell-averaged spectra at the final lead time, laid out as in Figs. 20 and 21 . The rollout stays finite, but its forecast from the training data departs from the other three seeds: at Δt=2500\Delta t=2500 its velocity error is 0.1740.174 (other seeds 0.0800.080 – 0.0960.096 ) and its vorticity correlation is 0.6830.683 ( 0.8290.829 – 0.8620.862 ). It also keeps more energy at high kk than the other seeds (high- kk /ref. 0.1420.142 against 0.0070.007 – 0.0340.034 ). Its forecast from the testing data is in line with, and on the field metrics slightly better than, the other seeds.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE4/40.0807±6.9×10−5\bf{0.0807\pm 6.9{\times}10^{-5}}0.846±2.5×10−4\bf{0.846\pm 2.5{\times}10^{-4}}0.536±3.8×10−4\bf{0.536\pm 3.8{\times}10^{-4}}3.28±2.4×10−33.28\pm 2.4{\times}10^{-3}0.0454±2.2×10−40.0454\pm 2.2{\times}10^{-4}0.104±1.4×10−40.104\pm 1.4{\times}10^{-4}
ELBM0/10.2810.2810.5460.5460.9810.9810.869\bf{0.869}0.684\bf{0.684}0.647\bf{0.647}
64264^{2}NeurDE4/40.154±5.7×10−4\bf{0.154\pm 5.7{\times}10^{-4}}0.771±3.9×10−4\bf{0.771\pm 3.9{\times}10^{-4}}0.641±3.7×10−5\bf{0.641\pm 3.7{\times}10^{-5}}3.60±9.6×10−23.60\pm 9.6{\times}10^{-2}0.0283±0.0015\bf{0.0283\pm 0.0015}0.0553±0.00160.0553\pm 0.0016
ELBM0/10.3920.3920.5420.5421.031.030.438\bf{0.438}1.981.981.49\bf{1.49}
32232^{2}NeurDE4/40.304±0.045\bf{0.304\pm 0.045}0.684±0.032\bf{0.684\pm 0.032}0.730±0.032\bf{0.730\pm 0.032}3.72±0.703.72\pm 0.700.050±0.046\bf{0.050\pm 0.046}0.060±0.029\bf{0.060\pm 0.029}
Appendix
Table 10: Real-space and spectral metrics for doubly periodic shear layer at Re=100,000\mathrm{Re}=100{,}000 . We compare emulation metrics from the training data ( t0=0t_{0}=0 ) at the final lead time against the downsampled 5122512^{2} reference. Entropically-stabilized NeurDE results are the mean ±\pm standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative L2L^{2} error of velocity and vorticity ω\omega , and vorticity correlation. Spectrum diagnostics: RMSE of log⁡10E(k)\log_{10}E(k) over active shells, and high- kk energy and palinstrophy relative to the reference. Bold marks the better method per grid.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE4/40.0519±3.3×10−60.0519\pm 3.3{\times}10^{-6}0.937±1.6×10−5\bf{0.937\pm 1.6{\times}10^{-5}}0.353±4.2×10−5\bf{0.353\pm 4.2{\times}10^{-5}}4.34±3.5×10−34.34\pm 3.5{\times}10^{-3}0.00635±2.3×10−50.00635\pm 2.3{\times}10^{-5}0.0868±5.0×10−50.0868\pm 5.0{\times}10^{-5}
ELBM1/10.0448\bf{0.0448}0.9200.9200.4010.4010.257\bf{0.257}1.08\bf{1.08}1.02\bf{1.02}
64264^{2}NeurDE4/40.0956±5.9×10−40.0956\pm 5.9{\times}10^{-4}0.916±3.3×10−40.916\pm 3.3{\times}10^{-4}0.421±8.5×10−40.421\pm 8.5{\times}10^{-4}4.58±0.114.58\pm 0.110.00309±0.000470.00309\pm 0.000470.0780±4.0×10−40.0780\pm 4.0{\times}10^{-4}
ELBM1/10.0555\bf{0.0555}0.920\bf{0.920}0.402\bf{0.402}0.346\bf{0.346}1.56\bf{1.56}1.28\bf{1.28}
32232^{2}NeurDE4/40.175±0.0190.175\pm 0.0190.870±0.0160.870\pm 0.0160.519±0.0310.519\pm 0.0313.75±0.623.75\pm 0.620.0040±0.00440.0040\pm 0.00440.081±0.0180.081\pm 0.018
Appendix
Table 11: Real-space and spectral metrics for doubly periodic shear layer at Re=100,000\mathrm{Re}=100{,}000 . We compare emulation metrics from the testing data ( t0=T/2t_{0}=T/2 ) at the final lead time against the downsampled 5122512^{2} reference. Entropically-stabilized NeurDE results are the mean ±\pm standard deviation over four seeds; ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative L2L^{2} error of velocity and vorticity ω\omega , and vorticity correlation. Spectrum diagnostics: RMSE of log⁡10E(k)\log_{10}E(k) over active shells, and high- kk energy and palinstrophy relative to the reference. Bold marks the better method per grid.
Figure 24: Doubly periodic shear layer at Re=100,000\mathrm{Re}=100{,}000 : vorticity forecasts on grids (a) 1282128^{2} , (b) 64264^{2} , (c) 32232^{2} , and (d) 16216^{2} ; the grid size is also marked to the left of each panel. In each panel, the rows are the downsampled 5122512^{2} reference (top), entropically-stabilized NeurDE (middle), and numerical ELBM run at that grid (bottom). The left block starts from the training data ( t0=0t_{0}=0 ), the right block from the testing data ( t0=T/2t_{0}=T/2 ); columns are lead times Δt\Delta t in solver steps.
Figure 25: Doubly periodic shear layer at Re=100,000\mathrm{Re}=100{,}000 : shell-averaged spectra at the final lead time on grids (a) 1282128^{2} , (b) 64264^{2} , (c) 32232^{2} , and (d) 16216^{2} . Columns are the energy E(k)E(k) , enstrophy Z(k)=k2E(k)Z(k)=k^{2}E(k) , and palinstrophy P(k)=k4E(k)P(k)=k^{4}E(k) spectra, where kk is the wavenumber shell; rows are forecasts from the training (top) and testing (bottom) data. Curves compare the downsampled 5122512^{2} reference, numerical ELBM, and entropically-stabilized NeurDE .
Figure 26: The percentage of pixels that violate the second law of thermodynamics over the NeurdE emulation of the doubly periodic shear layer at Re=100,000\mathrm{Re}=100{,}000 . We show the training and testing datasets for all grids. The mean and standard deviation are taken over all seeds.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE4/4(6.14±0.81)×10−4(6.14\pm 0.81){\times}10^{-4}1.00±1.4×10−91.00\pm 1.4{\times}10^{-9}(6.23±0.82)×10−4(6.23\pm 0.82){\times}10^{-4}––1.00±1.7×10−41.00\pm 1.7{\times}10^{-4}
ELBM1/12.53×10−4\bf{2.53{\times}10^{-4}}1.001.002.58×10−4\bf{2.58{\times}10^{-4}}––1.001.00
64264^{2}NeurDE4/40.00209±0.000490.00209\pm 0.000491.00±3.5×10−81.00\pm 3.5{\times}10^{-8}0.00213±0.000460.00213\pm 0.00046––0.996±1.0×10−30.996\pm 1.0{\times}10^{-3}
ELBM1/10.00162\bf{0.00162}1.001.000.00179\bf{0.00179}––0.997\bf{0.997}
32232^{2}NeurDE4/40.00954±6.8×10−50.00954\pm 6.8{\times}10^{-5}1.00±3.7×10−91.00\pm 3.7{\times}10^{-9}0.00951±6.7×10−50.00951\pm 6.7{\times}10^{-5}––0.981±1.4×10−40.981\pm 1.4{\times}10^{-4}
Appendix
Table 12: Real-space and spectral metrics for TGV at Re=50,000\mathrm{Re}=50{,}000 . We compare emulation metrics from the training data ( t0=0t_{0}=0 ) at the final lead time against the downsampled 5122512^{2} reference. Entropically-stabilized NeurDE results are the mean ±\pm standard deviation over four seeds, ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative L2L^{2} error of velocity and vorticity ω\omega , and vorticity correlation. Spectrum diagnostics: RMSE of log⁡10E(k)\log_{10}E(k) over active shells, and high- kk energy and palinstrophy relative to the reference. Bold marks the better method per grid.
GridMethodFaithful EmulationField FidelitySpectrum Diagnostics
Vel. err ↓\downarrowω\omega corr. ↑\uparrowω\omega err ↓\downarrowSpec. RMSE † ↓\downarrowHigh- kk /ref. →1\to 1PP /ref. →1\to 1
1282128^{2}NeurDE4/4(5.62±0.79)×10−4(5.62\pm 0.79){\times}10^{-4}1.00±2.5×10−91.00\pm 2.5{\times}10^{-9}(5.76±0.81)×10−4(5.76\pm 0.81){\times}10^{-4}––1.00±1.7×10−41.00\pm 1.7{\times}10^{-4}
ELBM1/13.14×10−4\bf{3.14{\times}10^{-4}}1.001.003.18×10−4\bf{3.18{\times}10^{-4}}––0.999\bf{0.999}
64264^{2}NeurDE4/40.00225±0.000480.00225\pm 0.000481.00±3.9×10−81.00\pm 3.9{\times}10^{-8}0.00227±0.000460.00227\pm 0.00046––0.996±1.0×10−30.996\pm 1.0{\times}10^{-3}
ELBM1/10.00180\bf{0.00180}1.001.000.00197\bf{0.00197}––0.997\bf{0.997}
32232^{2}NeurDE4/40.00970±6.8×10−50.00970\pm 6.8{\times}10^{-5}1.00±3.4×10−91.00\pm 3.4{\times}10^{-9}0.00971±6.8×10−50.00971\pm 6.8{\times}10^{-5}––0.981±1.4×10−40.981\pm 1.4{\times}10^{-4}
Appendix
Table 13: Real-space and spectral metrics for TGV at Re=50,000\mathrm{Re}=50{,}000 . We compare emulation metrics from the testing data ( t0=T/2t_{0}=T/2 ) at the final lead time against the downsampled 5122512^{2} reference. Entropically-stabilized NeurDE results are the mean ±\pm standard deviation over four seeds, ELBM results are from one deterministic run. The "Faithful Emulation" reports whether the rollout reproduces the expected structures. Field fidelity: relative L2L^{2} error of velocity and vorticity ω\omega , and vorticity correlation. Spectrum diagnostics: RMSE of log⁡10E(k)\log_{10}E(k) over active shells, and high- kk energy and palinstrophy relative to the reference. Bold marks the better method per grid.
Figure 27: TGV velocity streamline emulations at Re=50,000\mathrm{Re}=50{,}000 , colored by speed on grids (a) 1282128^{2} , (b) 64264^{2} , (c) 32232^{2} , and (d) 16216^{2} ; the grid size is also marked to the left of each panel. In each panel, the rows are the downsampled 5122512^{2} reference (top), entropically-stabilized NeurDE (middle), and numerical ELBM run at that grid (bottom). The left block starts from the training data ( t0=0t_{0}=0 ), the right block from the testing data ( t0=T/2t_{0}=T/2 ); columns are lead times Δt\Delta t in solver steps.
Figure 28: TGV at Re=50,000\mathrm{Re}=50{,}000 shell-averaged spectra at the final lead time on grids (a) 1282128^{2} , (b) 64264^{2} , (c) 32232^{2} , and (d) 16216^{2} . Columns are the energy E(k)E(k) , enstrophy Z(k)=k2E(k)Z(k)=k^{2}E(k) , and palinstrophy P(k)=k4E(k)P(k)=k^{4}E(k) spectra, where kk is the wavenumber shell; rows are forecasts from the training (top) and testing (bottom) data. Curves compare the downsampled 5122512^{2} reference, numerical ELBM, and entropically-stabilized NeurDE . The flow stays in a few low- kk shells, so the high- kk tail is at roundoff level.
Figure 29: The percentage of pixels that violate the second law of thermodynamics over the NeurdE emulation of the Taylor Green vortex experiment at Re=50,000\mathrm{Re}=50{,}000 . We show the training and testing datasets for all grids. The mean and standard deviation are taken over all seeds.
HyperparameterValue
Spatial dimensions2 ( 256×256256\times 256 periodic grid)
Input / output channels4 (density, uxu_{x} , uyu_{y} , vorticity)
Fourier modes retained64 per dimension ( 64×3364\times 33 after the real FFT)
Hidden channels (width)128
Fourier layers4
Lifting / projection channels256 / 256
Appendix
Table 14: FNO architecture.
HyperparameterValues sweptFinal run
Hidden channels128, 256128
Global batch size16, 32, 6416
OneCycle initial divisor ( div_factor )5, 25, 10025
OneCycle final divisor ( final_div_factor )100, 1000100
Appendix
Table 15: Sweep grid and the values used in the final run.
HyperparameterValue
Data
SimulationD2Q9 weakly compressible isothermal LBM, Re=5×104\mathrm{Re}=5\times 10^{4} , Ma=0.1\mathrm{Ma}=0.1
Trajectoryone trajectory, 256×256256\times 256 grid, 10410^{4} saved steps
Train / validation / test split50 % / 10 % / 40 %, contiguous in time
Inputs and targetsdensity, velocity, vorticity
Loss and optimizer
Appendix
Table 16: FNO training setup.
QuantityValue
SystemNERSC Perlmutter, 1 GPU node
GPU typeNVIDIA A100-SXM4, 40 GB
Number of GPUs4
CPU cores allocated64 physical (128 hardware threads)
Job wall-clock time23.96 h (24 h limit)
Training time (epoch 0 to epoch 182)23.67 h
Appendix
Table 17: Compute used to train the final FNO.
Rollout lengthEpochsWall-clock (h)Minutes per epoch
1250.300.7
2250.421.0
3250.531.3
5220.711.9
10161.013.8
254010.0215.0
Appendix
Table 18: Wall-clock time per curriculum stage (4 GPUs).
Figure 30: Diverging vorticity decay from DNS as forecast length increases.
QuantityValue
SystemNERSC Perlmutter, 2 GPU nodes
GPU typeNVIDIA A100-SXM4, 80 GB
Number of GPUs8 (4 per node)
CPU cores allocated128 physical (256 hardware threads)
Total wall-clock time38.11 h
GPU-hours304.8
Appendix
Table 19: Compute used to fine-tune Walrus.
More in this topic

PDE Surrogate Modeling

PDE surrogate modeling builds fast approximations of partial differential equation solutions for simulation, prediction, and design.

See all 347 papers

Explore similar work

CardsList

  1. Gradient-free online learning of subgrid-scale dynamics with neural emulators

    Oct 30, 2023Hugo Frezat, Ronan Fablet, Guillaume Balarac +1Climate ModelingGradient-Free Optimization

    Univ. Grenoble-Alpes, CNRS, Grenoble INP, LEGI, Grenoble, France · Univ. Grenoble-Alpes, CNRS, INRAE, IRD, Grenoble INP, IGE, Grenoble, France · IMT Atlantique, CNRS, Lab-STICC, Brest, France +2

  2. Subgrid-Scale Parameterization in Burgers' Equation Using Structure-Preserving Neural Networks and Entropy Variables

    Jul 16, 2026Aijaz Nazir, Ilya TimofeyevNeural Surrogate ModelingPDE Solving

    Dept. of Mathematics, University of Houston, Houston, TX 77204

  3. The Physics Is the Network: Fast, Training-Free Neural Emulation

    Oct 14, 2026Luka Todorovic, Kaushal Gianchandani, Sai RavelaNeural PDE SolversNeural Surrogate Modeling

    Department of Mathematics and Department of Statistics, University of Michigan, Ann Arbor, MI · NASA Postdoctoral Program Fellow, NASA Goddard Institute for Space Studies, New York, NY · Department of Earth, Atmospheric, and Planetary Sciences, Massachusetts Institute of Technology, Cambridge, MA