Staying on the Attractor: Supervising Neural Surrogates of 3D Turbulence Where They Leave It

Yilong Dai; Shaswata Mitra; Raj Patel; Yiming Sun; Shengyu Chen; Jiaqi Gong; Sudip Mittal; Shahram Rahimi

cs.LGSep 26, 2026

Staying on the Attractor: Supervising Neural Surrogates of 3D Turbulence Where They Leave It

Authors: Yilong Dai, Shaswata Mitra, Raj Patel, Yiming Sun, Shengyu Chen, Jiaqi Gong, Sudip Mittal, Shahram Rahimi,

Organizations: The University of Alabama · Rutgers University · University of Pittsburgh

Abstract

Neural surrogates are trained to predict 3D turbulent flows in place of direct numerical simulation (DNS). For chaotic flows, the goal is short-term pointwise accuracy followed by long-term physical and statistical fidelity. However, small prediction errors can carry a surrogate away from the flow's attractor. Off-attractor states are poorly represented in training data, leaving their evolution weakly constrained. The learned dynamics can then amplify deviations and lead to blow-up, freezing, or statistical drift. We propose off-attractor supervision (OAS) to supervise neural surrogates where they leave the attractor. OAS teaches the model how the true Navier-Stokes dynamics would evolve from these states. Each selected state is paired with its own future computed by DNS. Three generators select a few hundred states for relabeling. The first collects states from the surrogate's own rollouts. The second uses surrogate attacks to target freezing, excessive amplification, and violations of incompressibility and energy balance. The third perturbs training states along an amplified direction and a strongly damped random direction of the dynamics. All attacks run on the surrogate alone, and DNS relabeling is performed offline once per selected state. Experiments on $128^3$ turbulence show that OAS increases the median time to failure from 21 to 721 steps. The compared baselines achieve medians of at most 110 steps, and the advantage holds across training seeds. OAS also achieves the lowest pointwise error at step 15 and the best long-horizon statistics among the compared methods. OAS integrates physical models into neural simulation by extending supervision from fixed reference trajectories to states where the surrogate is likely to fail. This principle can guide the development of more reliable scientific surrogates when deployment takes models beyond the coverage of their training data.

Figures & tables

Figure 1: The three failure modes, and the one-step response behind them (Sections 2.1 – 2.3 ). (a) Mid-plane $u_{x}$ of a held-out DNS trajectory and of three rollouts of models trained with clean-future labels, one column each and one row per rollout step. (b) A rollout leaving the attractor. (c) One-step amplification in the dynamics and in the learned map. (d) Divergence of two nearby DNS trajectories. (e) Total energy of the columns of (a). Dotted: the steps of (a).

Figure 2: (a) The three state generators, drawn on the attractor. (b) The pipeline, from a generated state to a labeled pair in fine-tuning (Section 3.4 ).

	nRMSE@15 $\downarrow$	band exit $\uparrow$	strict $\uparrow$	enstrophy $\to\!1$	spec. dist. $\downarrow$	spread $\to\!1$
Baseline U-Net	3.50	21	21	932 $\times$	0.95	0.03
Noise injection	1.28	29	29	576 $\times$	0.85	0.01
Pushforward trick	1.14	24	24	1771 $\times$	0.97	0.02
Unrolled training	0.83	89	89	32.6 $\times$	0.50	0.45
Denoising refinement	1.03	110	88	17.7 $\times$	0.48	0.12
OAS (ours)	0.74	721	721	7.4 $\times$	0.22	0.91

Table 1: Main comparison. Three held-out initial conditions, medians, over rollouts continued to failure. The baseline U-Net, unrolled training, and OAS are seed medians (Table 16 ). The other three methods are single runs. The three statistics are read over the first 200 steps (

10\,T_{L}

), the spread as a single reading at its end. Criteria in Section 4.1 .

Figure 3: (a) Mid-plane $u_{x}$ of OAS and of the five other methods of Table 1 , one column each and one row per rollout step, from one held-out initial state on the color scale of Fig. 1 a. (b) Total kinetic energy $K_{t}/K_{0}$ , one color per method as in (a), the rollouts of (a) (thick) and two further held-out initial conditions (thin). Dotted: the steps of (a). Shaded: the band.

Figure 4: (a) The eight generator combinations, the matrix below marking which generators each pool holds. (b) The three attack objectives and their combinations, on attack pairs alone. (c) Maximum on-policy depth at matched budget.

Figure 5: Step-20 nRMSE versus (a) DNS correlation and (b) log-spectral distance for the six methods in Table 1 (dashed: nRMSE of 1). (c) Band-exit step versus worst-direction amplification for seven deterministic configurations (dashed: the largest amplification found in the DNS).

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

family	states	label	solver
One-step supervised	attractor only	clean future	offline, dataset only
Noise injection, pushforward	randomly perturbed	clean future $S(x)$	none
Unrolled training	on-policy, short window	clean future $S(x)$	none
Solver-in-the-loop	on-policy, solver-corrected	clean future $S(x)$	differentiable, in the training loop
Diverted chain	on-policy	own future $S(\tilde{x})$	differentiable, in the training loop
Solver-integrated attacks	adversarial	own future $S(\tilde{x})$	in the attack loop, differentiated through

Appendix

Table 2: Where off-attractor supervision sits. States : which inputs the model is trained on. Label : which future each is paired with,

S(x)

being the future of the clean state it was displaced from and

S(\tilde{x})

its own. Solver : where the reference solver runs, if at all. Rows are families rather than individual papers, and the text qualifies them. The own-future label appears in four of them, and the second and fourth columns are where those four differ.

Input: $\mathcal{C}$ , $S$ , $f_{\theta_{0}}$ , $p$ , $K$ ; Output: $\theta_{3}$
1	$\mathcal{G}\leftarrow\varnothing$
2	for stage $m=1,2,3$ do
3	if $m\leq 2$ : $\mathcal{G}\leftarrow\big(\mathcal{G}\setminus\text{on-policy}\big)\cup\{f_{\theta_{m-1}}^{k}(x)\,:\,x\in\mathcal{C},\ k\in\{4,8,16,32\}\}$
4	if $m=2$ : $\mathcal{G}\leftarrow\mathcal{G}\cup\{x+\delta^{\star}(x)\}$ , $\delta^{\star}$ from PGD on frozen $f_{\theta_{m-1}}$ , one objective per state
5	if $m=3$ : $\mathcal{G}\leftarrow\mathcal{G}\cup\{x+\epsilon v\}$ , $v$ from power iteration, or least amplified of 12 random directions
6	for each state $\tilde{x}$ newly added to $\mathcal{G}$ do

Appendix

Table 3: Off-attractor supervision: pool assembly and staged fine-tuning. Inputs are the clean training states

\mathcal{C}

, the solver

S

with its recorded forcing realizations, the baseline model

f_{\theta_{0}}

trained on

\mathcal{L}_{\mathrm{clean}}

, the pair fraction

p

, and the label horizon

K

. The three stages of Section 3.4 differ in which generators have been added, and the first also in its horizon and learning rate (Appendix D ). The on-policy states are re-collected once, against the model that the first stage produces.

Figure 6: (a–c) Methods of Table 1 over steps 1 to 20, medians over three initial conditions (shaded: min to max): nRMSE against the DNS (dashed: 1), Pearson correlation, and log-spectral distance, with denoising refinement omitted from (c). (d) DNS energy spectrum $E(k)$ (black, time average), shell energy of the baseline U-Net’s one-step error (filled), and their ratio (dashed, right axis).

Figure 7: Onset of failure. Mid-plane $u_{x}$ of the six methods of Table 1 and of the clean-trained FNO of Table 13 at thirteen rollout steps, six of them in the first ten, one column each and one row per step, from one held-out initial condition and on the color scale of Fig. 1 a. Framed: the first step shown after the rollout fails, with its failure mode and failure step.

pairs	one-step val. loss	band exit $\uparrow$	strict $\uparrow$	per condition
0 60	$9.7\times 10^{-5}$	388	368	[278, 368, 371]
120	$1.05\times 10^{-4}$	317	236	[219, 236, 448]
242	$1.21\times 10^{-4}$	650	319	[140, 319, 431]
296	$1.21\times 10^{-4}$	661	569	[558, 569, 613]

Appendix

Table 4: Pool size at a matched fine-tuning budget: the one-step baseline, eight epochs, and random subsets of the 296-pair pool that carries depths 64 and 96. Medians over three initial conditions with the per-condition strict values in brackets. Single runs.

on-policy	surrogate-attack	DNS-probe	band exit	freeze	strict	failure mode
✓	✓	✓	524	173	173	freeze
✓	–	✓	639	636	636	freeze (late)
✓ ( $-$ depths 16, 32)	✓	✓	534	294	294	freeze
–	✓	✓	54	—	54	blow-up

Appendix

Table 5: One-shot family. Every configuration is fine-tuned from the baseline model on a reduced pool (rollouts to failure, three initial conditions, medians). The full-pool row is the seed whose band exit equals the five-seed median.

on-policy	surrogate-attack	DNS-probe	base	band exit	freeze	strict	failure mode
–	–	–	one-step	13	—	13	blow-up
✓	–	–	row 1	674	86	86	freeze
✓	✓	–	row 2	377	139	139	freeze
✓	✓	✓	row 3	721	—	721	band exit (late)
✓ ( $+$ depths 64, 96)	✓	✓	row 3	796	635	635	freeze (late)
–	✓	–	row 1	76	—	76	blow-up

Appendix

Table 6: Component ablation, all eight combinations of the three generators, plus the main pool extended to depths 64 and 96 in row 5 (rollouts to failure, three initial conditions, medians). Row 1 is the single-seed baseline run, the bold row and row 5 are five-seed medians (Table 16 ), and the rest are single runs. The base column gives the model each row was fine-tuned from, since the pool is assembled in stages and the leave-one-out rows below the rule therefore do not share one: rows 2 to 4 are those stages in order, the row without DNS-probe states is row 3 itself, and the two remaining leave-one-out rows retrain the last stage on the reduced pool. Table 5 repeats the comparison from a single fixed base.

objectives	band exit (3 ICs)	median
freeze-inducing	[9, 11, 10]	10
amplification-inducing	[51, 53, 57]	53
physics-violating	[50, 53, 58]	53
freeze + amplification †	[63]	63
freeze + violation	[52, 53, 63]	53
amplification + violation	[59, 61, 63]	61

Appendix

Table 7: Attack objectives (

2^{3}

grid, 66 pairs per cell) and the solver-in-the-loop control. Band-exit step over a 200-step window. † single initial condition.

max. on-policy rollout depth	band exit	strict	freeze events (of 3)
$\leq 8$	180	180	0
$\leq 16$	406	330	1
$\leq 32$	384	355	2
$\leq 64$	573	308	3
$\leq 96$ (all depths)	438	180	3
16 and 32 only	421	358	1

Appendix

Table 8: On-policy depth ablation. Top: matched budget, only the maximum rollout depth of the on-policy states varies. Bottom: full budget. Single runs, except the two main-model rows (five-seed medians, Table 16 ).

hyperparameter	default	variant	band exit	strict
learning rate	$10^{-4}$	$5\times 10^{-5}$	436	379
pair-batch ratio	0.3	0.2	613	218
fine-tuning epochs	24	4	465	86

Appendix

Table 9: Training hyperparameters of the staged main model (721 on both criteria), one varied at a time (rollouts to failure, three initial conditions, medians). Single runs.

states	labels	loss protocol	band exit	strict
off-attractor (OAS staged)	own future $S(\tilde{x})$	4-step unrolled on pairs	721	721
off-attractor (OAS one-shot) †	own future $S(\tilde{x})$	4-step unrolled on pairs	524	173
clean windows, fine-tuning schedule †	clean future $S(x)$	4-step unrolled	38	38
clean windows, $984$ new DNS frames ‡	clean future $S(x)$	4-step unrolled on pairs	47	47
clean windows, unrolled schedule	clean future $S(x)$	4-step unrolled	259	259
unrolled base $+$ attack pairs	mixed	4-step unrolled	99	99

Appendix

Table 10: Label control (rollouts to failure, three initial conditions, medians). † same baseline model and fine-tuning schedule, only the training pairs differ. ‡ matched DNS budget, on the staged schedule of row 1. Row 1 is the five-seed median of Table 16 . The rest are single runs.

direction	$\lambda$
worst (power iteration)	1.036
random isotropic	0.417
model’s actual error direction	0.94
spectrum-shaped	1.016

Appendix

Table 11: One-step amplification

\lambda

of the dynamics along four directions, measured with the DNS at relative amplitude

10^{-3}

configuration	worst dir.	random dir.	band exit	strict
DNS-probe states only	3.17	1.03	18	18
Baseline U-Net	2.86	1.04	13	13
Noise injection	2.61	1.03	29	29
Pushforward trick	2.28	1.03	24	24
Surrogate-attack states only	2.10	1.03	76	76
Unrolled training	1.16	1.03	259	259

Appendix

Table 12: One-step amplification of the learned map along its worst direction and along a random one, with the failure times of the same run, for the seven configurations of Fig. 5 c. Each is the single run whose failure times the table reports. Both directions are probed at relative amplitude

10^{-3}

, as in Table 11 . The random-direction column is not bolded: it is flat across configurations and far from the

0.417

of the dynamics, so it distinguishes nothing (Section 4.4 ).

backbone	nRMSE@1	nRMSE@15	band exit	strict
U-Net (baseline)	0.049	3.498	0 13 [ 0 12, 0 13, 0 15]	13
FNO ( Li et al., 2021 )	0.102	2.120	408 [352, 408, 466]	17 [11, 17, 18]
TFNO ( Kossaifi et al., 2024 )	0.128	2.941	0 33 [ 0 31, 0 33, 0 34]	19 [18, 19, 23]
DPOT ( Hao et al., 2024 )	0.164	5.831	0 11 [ 00 7, 0 11, 0 12]	11
FactFormer ( Li et al., 2023 )	0.119	5.274	00 6 [ 00 5, 00 6, 00 7]	0 6

Appendix

Table 13: Clean-trained backbones. Rollouts to failure from three initial conditions, medians with the per-condition values in brackets. Single runs.

backbone		nRMSE@15 $\downarrow$	strict $\uparrow$
FNO	clean-trained	2.120	00 17
	$+$ OAS (3 seeds)	1.141 [1.077, 1.141, 1.232]	00 35 [34, 35, 37]
	gain	$1.9\times$	$2.1\times$
U-Net	clean-trained	3.498	00 21
	$+$ OAS (5 seeds)	0.740	721 [705–784]
	gain	$\mathbf{4.7\times}$	$\mathbf{34\times}$

Appendix

Table 14: Off-attractor supervision on the two candidate backbones, at rollout step 15 and on the strict criterion, over rollouts to failure from three initial conditions. The clean-trained rows are the single run of Table 13 (FNO) and the three-seed median of Table 1 (U-Net). Fine-tuned rows are seed medians. Brackets show individual seed results or their range.

	pairs	epochs	band exit $\uparrow$	strict $\uparrow$
Baseline U-Net	0	24	77 [59, 77, 84]	77
OAS (one-shot)	132 ∗	24	583 [503, 583, 589]	540 [485, 540, 545]
gain over baseline			$7.6\times$	$7.0\times$
∗ 54 on-policy, 54 surrogate-attack, 24 DNS-probe pairs.

Appendix

Table 15: Second regime,

\nu=0.010

(

\mathrm{Re}_{\lambda}\!\approx\!35

), with its own dataset and pool. Rollouts to failure from three initial conditions, medians with the per-condition values in brackets. Single runs.

	seeds	band exit $\uparrow$	strict $\uparrow$
Baseline U-Net	3	21 [13, 22, 21]	21
Unrolled training	3	89 [259, 52, 89]	89 [259, 52, 89]
OAS one-shot	5	524 [524, 399, 181, 774, 617]	247 [173, 399, 181, 247, 617]
OAS staged (main)	5	721 [705, 719, 721, 723, 784]	721 [705, 719, 721, 723, 784]
OAS staged $+$ depths 64, 96	5	796 [685, 794, 796, 820, 833]	635 [583, 593, 635, 647, 718]

Appendix

Table 16: Seed robustness. Rollouts continued to failure, from three initial conditions. Median over seeds, with the per-seed medians in brackets.

Neural Surrogate Modeling

Neural surrogate modeling uses learned models to approximate costly simulations, enabling faster prediction, design, control, and inference.

See all 412 papers

A Validated LBM Dataset and Pipeline for Surrogate Modeling of Turbulent 3D Obstructed Channel Flows
Jun 15, 2026Lukas Schröder, Shubham Kavane, Harald KöstlerNeural Surrogate Modeling Synthetic Benchmark Generation
Chair of Computer Science 10 (System Simulation) Friedrich-Alexander-Universität Erlangen-Nürnberg
Scale-Split Neural Operator for Memory- and Data-Efficient 3D Turbulence Prediction
Sep 30, 2026Shaoxiang Qin, Yucheng Zhao, Zongyi Li +2PDE Surrogate Modeling Neural Operators
University of Tennessee, Knoxville · McGill University · New York University +1
FSAN: Flow State Attention Network for Aerodynamic Prediction
Sep 6, 2026Wenxuan Jin, Jianguo Yao, Haibing Guan +1Neural Surrogate Modeling PDE Surrogate Modeling
Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University

Jun 15, 2026J/K move · Enter open · S save

A Validated LBM Dataset and Pipeline for Surrogate Modeling of Turbulent 3D Obstructed Channel Flows

Lukas Schröder, Shubham Kavane, Harald Köstler

Chair of Computer Science 10 (System Simulation) Friedrich-Alexander-Universität Erlangen-Nürnberg

Neural Surrogate Modeling Synthetic Benchmark Generation Computational Fluid Dynamics

Open paper AskPDF arXiv

Evaluating neural operators for 3D turbulent flow requires validated datasets with physical benchmarks. We present a reproducible pipeline generating training data for 3D channel flows around generated geometries at Re=1,000-10,000. Our lattice Boltzmann solver with cumulant collision operators is rigorously verified against experimental measurements (Strouhal number, drag coefficients, turbulent fluctuations) with comprehensive grid convergence studies at resolution 1024x512x512. Building upon an established framework, this validated pipeline enables standardized surrogate model comparison. We outline planned systematic evaluation of Fourier Neural Operator and U-Net variants on forecasting, super-resolution, and error correction tasks, using physics-informed metrics to assess turbulent energy cascade representation. Future work will compare computational efficiency between numerical solvers and neural surrogates, exploring practical application. We seek community feedback on our validation approach, planned benchmark methodology, and evaluation priorities for neural operators in turbulent flows.

Abstract

Neural Surrogate Modeling

Explore similar work

A Validated LBM Dataset and Pipeline for Surrogate Modeling of Turbulent 3D Obstructed Channel Flows

Scale-Split Neural Operator for Memory- and Data-Efficient 3D Turbulence Prediction

FSAN: Flow State Attention Network for Aerodynamic Prediction