Neural surrogates are trained to predict 3D turbulent flows in place of direct numerical simulation (DNS). For chaotic flows, the goal is short-term pointwise accuracy followed by long-term physical and statistical fidelity. However, small prediction errors can carry a surrogate away from the flow's attractor. Off-attractor states are poorly represented in training data, leaving their evolution weakly constrained. The learned dynamics can then amplify deviations and lead to blow-up, freezing, or statistical drift. We propose off-attractor supervision (OAS) to supervise neural surrogates where they leave the attractor. OAS teaches the model how the true Navier-Stokes dynamics would evolve from these states. Each selected state is paired with its own future computed by DNS. Three generators select a few hundred states for relabeling. The first collects states from the surrogate's own rollouts. The second uses surrogate attacks to target freezing, excessive amplification, and violations of incompressibility and energy balance. The third perturbs training states along an amplified direction and a strongly damped random direction of the dynamics. All attacks run on the surrogate alone, and DNS relabeling is performed offline once per selected state. Experiments on 1283 turbulence show that OAS increases the median time to failure from 21 to 721 steps. The compared baselines achieve medians of at most 110 steps, and the advantage holds across training seeds. OAS also achieves the lowest pointwise error at step 15 and the best long-horizon statistics among the compared methods. OAS integrates physical models into neural simulation by extending supervision from fixed reference trajectories to states where the surrogate is likely to fail. This principle can guide the development of more reliable scientific surrogates when deployment takes models beyond the coverage of their training data.
Figures & tables
Figure 1: The three failure modes, and the one-step response behind them (Sections 2.1 – 2.3 ). (a) Mid-plane ux of a held-out DNS trajectory and of three rollouts of models trained with clean-future labels, one column each and one row per rollout step. (b) A rollout leaving the attractor. (c) One-step amplification in the dynamics and in the learned map. (d) Divergence of two nearby DNS trajectories. (e) Total energy of the columns of (a). Dotted: the steps of (a).
Figure 2: (a) The three state generators, drawn on the attractor. (b) The pipeline, from a generated state to a labeled pair in fine-tuning (Section 3.4 ).
nRMSE@15 ↓
band exit ↑
strict ↑
enstrophy →1
spec. dist. ↓
spread →1
Baseline U-Net
3.50
21
21
932 ×
0.95
0.03
Noise injection
1.28
29
29
576 ×
0.85
0.01
Pushforward trick
1.14
24
24
1771 ×
0.97
0.02
Unrolled training
0.83
89
89
32.6 ×
0.50
0.45
Denoising refinement
1.03
110
88
17.7 ×
0.48
0.12
OAS (ours)
0.74
721
721
7.4 ×
0.22
0.91
Table 1: Main comparison. Three held-out initial conditions, medians, over rollouts continued to failure. The baseline U-Net, unrolled training, and OAS are seed medians (Table 16 ). The other three methods are single runs. The three statistics are read over the first 200 steps ( 10TL ), the spread as a single reading at its end. Criteria in Section 4.1 .
Figure 3: (a) Mid-plane ux of OAS and of the five other methods of Table 1 , one column each and one row per rollout step, from one held-out initial state on the color scale of Fig. 1 a. (b) Total kinetic energy Kt/K0 , one color per method as in (a), the rollouts of (a) (thick) and two further held-out initial conditions (thin). Dotted: the steps of (a). Shaded: the band.
Figure 4: (a) The eight generator combinations, the matrix below marking which generators each pool holds. (b) The three attack objectives and their combinations, on attack pairs alone. (c) Maximum on-policy depth at matched budget.
Figure 5: Step-20 nRMSE versus (a) DNS correlation and (b) log-spectral distance for the six methods in Table 1 (dashed: nRMSE of 1). (c) Band-exit step versus worst-direction amplification for seven deterministic configurations (dashed: the largest amplification found in the DNS).
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
family
states
label
solver
One-step supervised
attractor only
clean future
offline, dataset only
Noise injection, pushforward
randomly perturbed
clean future S(x)
none
Unrolled training
on-policy, short window
clean future S(x)
none
Solver-in-the-loop
on-policy, solver-corrected
clean future S(x)
differentiable, in the training loop
Diverted chain
on-policy
own future S(x~)
differentiable, in the training loop
Solver-integrated attacks
adversarial
own future S(x~)
in the attack loop, differentiated through
Appendix
Table 2: Where off-attractor supervision sits. States : which inputs the model is trained on. Label : which future each is paired with, S(x) being the future of the clean state it was displaced from and S(x~) its own. Solver : where the reference solver runs, if at all. Rows are families rather than individual papers, and the text qualifies them. The own-future label appears in four of them, and the second and fourth columns are where those four differ.
Input: C , S , fθ0 , p , K ; Output: θ3
1
G←∅
2
for stage m=1,2,3 do
3
if m≤2 : G←(G∖on-policy)∪{fθm−1k(x):x∈C,k∈{4,8,16,32}}
4
if m=2 : G←G∪{x+δ⋆(x)} , δ⋆ from PGD on frozen fθm−1 , one objective per state
5
if m=3 : G←G∪{x+ϵv} , v from power iteration, or least amplified of 12 random directions
6
for each state x~ newly added to G do
Appendix
Table 3: Off-attractor supervision: pool assembly and staged fine-tuning. Inputs are the clean training states C , the solver S with its recorded forcing realizations, the baseline model fθ0 trained on Lclean , the pair fraction p , and the label horizon K . The three stages of Section 3.4 differ in which generators have been added, and the first also in its horizon and learning rate (Appendix D ). The on-policy states are re-collected once, against the model that the first stage produces.
Figure 6: (a–c) Methods of Table 1 over steps 1 to 20, medians over three initial conditions (shaded: min to max): nRMSE against the DNS (dashed: 1), Pearson correlation, and log-spectral distance, with denoising refinement omitted from (c). (d) DNS energy spectrum E(k) (black, time average), shell energy of the baseline U-Net’s one-step error (filled), and their ratio (dashed, right axis).
Figure 7: Onset of failure. Mid-plane ux of the six methods of Table 1 and of the clean-trained FNO of Table 13 at thirteen rollout steps, six of them in the first ten, one column each and one row per step, from one held-out initial condition and on the color scale of Fig. 1 a. Framed: the first step shown after the rollout fails, with its failure mode and failure step.
pairs
one-step val. loss
band exit ↑
strict ↑
per condition
0 60
9.7×10−5
388
368
[278, 368, 371]
120
1.05×10−4
317
236
[219, 236, 448]
242
1.21×10−4
650
319
[140, 319, 431]
296
1.21×10−4
661
569
[558, 569, 613]
Appendix
Table 4: Pool size at a matched fine-tuning budget: the one-step baseline, eight epochs, and random subsets of the 296-pair pool that carries depths 64 and 96. Medians over three initial conditions with the per-condition strict values in brackets. Single runs.
on-policy
surrogate-attack
DNS-probe
band exit
freeze
strict
failure mode
✓
✓
✓
524
173
173
freeze
✓
–
✓
639
636
636
freeze (late)
✓ ( − depths 16, 32)
✓
✓
534
294
294
freeze
–
✓
✓
54
—
54
blow-up
Appendix
Table 5: One-shot family. Every configuration is fine-tuned from the baseline model on a reduced pool (rollouts to failure, three initial conditions, medians). The full-pool row is the seed whose band exit equals the five-seed median.
on-policy
surrogate-attack
DNS-probe
base
band exit
freeze
strict
failure mode
–
–
–
one-step
13
—
13
blow-up
✓
–
–
row 1
674
86
86
freeze
✓
✓
–
row 2
377
139
139
freeze
✓
✓
✓
row 3
721
—
721
band exit (late)
✓ ( + depths 64, 96)
✓
✓
row 3
796
635
635
freeze (late)
–
✓
–
row 1
76
—
76
blow-up
Appendix
Table 6: Component ablation, all eight combinations of the three generators, plus the main pool extended to depths 64 and 96 in row 5 (rollouts to failure, three initial conditions, medians). Row 1 is the single-seed baseline run, the bold row and row 5 are five-seed medians (Table 16 ), and the rest are single runs. The base column gives the model each row was fine-tuned from, since the pool is assembled in stages and the leave-one-out rows below the rule therefore do not share one: rows 2 to 4 are those stages in order, the row without DNS-probe states is row 3 itself, and the two remaining leave-one-out rows retrain the last stage on the reduced pool. Table 5 repeats the comparison from a single fixed base.
objectives
band exit (3 ICs)
median
freeze-inducing
[9, 11, 10]
10
amplification-inducing
[51, 53, 57]
53
physics-violating
[50, 53, 58]
53
freeze + amplification †
[63]
63
freeze + violation
[52, 53, 63]
53
amplification + violation
[59, 61, 63]
61
Appendix
Table 7: Attack objectives ( 23 grid, 66 pairs per cell) and the solver-in-the-loop control. Band-exit step over a 200-step window. † single initial condition.
max. on-policy rollout depth
band exit
strict
freeze events (of 3)
≤8
180
180
0
≤16
406
330
1
≤32
384
355
2
≤64
573
308
3
≤96 (all depths)
438
180
3
16 and 32 only
421
358
1
Appendix
Table 8: On-policy depth ablation. Top: matched budget, only the maximum rollout depth of the on-policy states varies. Bottom: full budget. Single runs, except the two main-model rows (five-seed medians, Table 16 ).
hyperparameter
default
variant
band exit
strict
learning rate
10−4
5×10−5
436
379
pair-batch ratio
0.3
0.2
613
218
fine-tuning epochs
24
4
465
86
Appendix
Table 9: Training hyperparameters of the staged main model (721 on both criteria), one varied at a time (rollouts to failure, three initial conditions, medians). Single runs.
states
labels
loss protocol
band exit
strict
off-attractor (OAS staged)
own future S(x~)
4-step unrolled on pairs
721
721
off-attractor (OAS one-shot) †
own future S(x~)
4-step unrolled on pairs
524
173
clean windows, fine-tuning schedule †
clean future S(x)
4-step unrolled
38
38
clean windows, 984 new DNS frames ‡
clean future S(x)
4-step unrolled on pairs
47
47
clean windows, unrolled schedule
clean future S(x)
4-step unrolled
259
259
unrolled base + attack pairs
mixed
4-step unrolled
99
99
Appendix
Table 10: Label control (rollouts to failure, three initial conditions, medians). † same baseline model and fine-tuning schedule, only the training pairs differ. ‡ matched DNS budget, on the staged schedule of row 1. Row 1 is the five-seed median of Table 16 . The rest are single runs.
direction
λ
worst (power iteration)
1.036
random isotropic
0.417
model’s actual error direction
0.94
spectrum-shaped
1.016
Appendix
Table 11: One-step amplification λ of the dynamics along four directions, measured with the DNS at relative amplitude 10−3 .
configuration
worst dir.
random dir.
band exit
strict
DNS-probe states only
3.17
1.03
18
18
Baseline U-Net
2.86
1.04
13
13
Noise injection
2.61
1.03
29
29
Pushforward trick
2.28
1.03
24
24
Surrogate-attack states only
2.10
1.03
76
76
Unrolled training
1.16
1.03
259
259
Appendix
Table 12: One-step amplification of the learned map along its worst direction and along a random one, with the failure times of the same run, for the seven configurations of Fig. 5 c. Each is the single run whose failure times the table reports. Both directions are probed at relative amplitude 10−3 , as in Table 11 . The random-direction column is not bolded: it is flat across configurations and far from the 0.417 of the dynamics, so it distinguishes nothing (Section 4.4 ).
backbone
nRMSE@1
nRMSE@15
band exit
strict
U-Net (baseline)
0.049
3.498
0 13 [ 0 12, 0 13, 0 15]
13
FNO ( Li et al., 2021 )
0.102
2.120
408 [352, 408, 466]
17 [11, 17, 18]
TFNO ( Kossaifi et al., 2024 )
0.128
2.941
0 33 [ 0 31, 0 33, 0 34]
19 [18, 19, 23]
DPOT ( Hao et al., 2024 )
0.164
5.831
0 11 [ 00 7, 0 11, 0 12]
11
FactFormer ( Li et al., 2023 )
0.119
5.274
00 6 [ 00 5, 00 6, 00 7]
0 6
Appendix
Table 13: Clean-trained backbones. Rollouts to failure from three initial conditions, medians with the per-condition values in brackets. Single runs.
backbone
nRMSE@15 ↓
strict ↑
FNO
clean-trained
2.120
00 17
+ OAS (3 seeds)
1.141 [1.077, 1.141, 1.232]
00 35 [34, 35, 37]
gain
1.9×
2.1×
U-Net
clean-trained
3.498
00 21
+ OAS (5 seeds)
0.740
721 [705–784]
gain
4.7×
34×
Appendix
Table 14: Off-attractor supervision on the two candidate backbones, at rollout step 15 and on the strict criterion, over rollouts to failure from three initial conditions. The clean-trained rows are the single run of Table 13 (FNO) and the three-seed median of Table 1 (U-Net). Fine-tuned rows are seed medians. Brackets show individual seed results or their range.
Table 15: Second regime, ν=0.010 ( Reλ≈35 ), with its own dataset and pool. Rollouts to failure from three initial conditions, medians with the per-condition values in brackets. Single runs.
seeds
band exit ↑
strict ↑
Baseline U-Net
3
21 [13, 22, 21]
21
Unrolled training
3
89 [259, 52, 89]
89 [259, 52, 89]
OAS one-shot
5
524 [524, 399, 181, 774, 617]
247 [173, 399, 181, 247, 617]
OAS staged (main)
5
721 [705, 719, 721, 723, 784]
721 [705, 719, 721, 723, 784]
OAS staged + depths 64, 96
5
796 [685, 794, 796, 820, 833]
635 [583, 593, 635, 647, 718]
Appendix
Table 16: Seed robustness. Rollouts continued to failure, from three initial conditions. Median over seeds, with the per-seed medians in brackets.
Evaluating neural operators for 3D turbulent flow requires validated datasets with physical benchmarks. We present a reproducible pipeline generating training data for 3D channel flows around generated geometries at Re=1,000-10,000. Our lattice Boltzmann solver with cumulant collision operators is rigorously verified against experimental measurements (Strouhal number, drag coefficients, turbulent fluctuations) with comprehensive grid convergence studies at resolution 1024x512x512. Building upon an established framework, this validated pipeline enables standardized surrogate model comparison. We outline planned systematic evaluation of Fourier Neural Operator and U-Net variants on forecasting, super-resolution, and error correction tasks, using physics-informed metrics to assess turbulent energy cascade representation. Future work will compare computational efficiency between numerical solvers and neural surrogates, exploring practical application. We seek community feedback on our validation approach, planned benchmark methodology, and evaluation priorities for neural operators in turbulent flows.
Lukas Schröder, Shubham Kavane, Harald Köstler
Chair of Computer Science 10 (System Simulation) Friedrich-Alexander-Universität Erlangen-Nürnberg
Neural surrogates have emerged as fast alternatives to the numerical simulation of three-dimensional turbulence. However, training them at high resolution remains challenging, since the memory of full-field models grows with the resolution. In addition, full-resolution training data are expensive to simulate and store, and therefore scarce. We introduce ScaleSplit-NO (Scale-Split Neural Operator), which exploits the scale structure of turbulence with two neural operators: a Parent predicts the global coarse field at the next time step, and a Child predicts full-resolution local patches conditioned on this prediction. Neither model operates on the full-resolution field. The Child is pretrained alone and then attached to the Parent's coarse prediction through zero-initialized connections. On two complex high-resolution turbulence benchmarks, ScaleSplit-NO surpasses all competing baselines in both prediction accuracy and data efficiency. On the higher-resolution dataset JHTDB256 (2563), its normalized mean squared error (NMSE) is 53% lower than that of the strongest baseline, and its training memory is 79% lower than that of the most memory-efficient baseline. We further demonstrate its effectiveness for urban wind prediction in a real district of Montreal on a 500×150×500 grid, reducing one-step NMSE by 65.8% relative to the baseline. Moreover, swapping in a Parent trained on additional coarse fields improves prediction without retraining the Child, providing further accuracy gains at a small storage cost.
Shaoxiang Qin, Yucheng Zhao, Zongyi Li +2
University of Tennessee, Knoxville · McGill University · New York University +1
Accurate aerodynamic prediction is critical for designing fuel-efficient and safe transportation systems such as aircraft and automobiles, yet traditional computational fluid dynamics (CFD) simulations remain computationally expensive and expertise-intensive, severely limiting their use in iterative design and real-time analysis. Existing deep learning surrogates suffer from two major limitations: (i) they are evaluated on datasets with narrow flow-condition ranges, leaving their performance under complex flow conditions undemonstrated; (ii) they treat global flow conditions as a single vector injected uniformly across all surface points, ignoring that different geometric regions experience distinct local flow phenomena, which degrades prediction accuracy under complex flow conditions. To address these limitations, we propose the Flow State Attention Network (FSAN). FSAN separately encodes point cloud and flow conditions, then partitions the geometry into multiple flow states via learnable soft assignments, and uses flow features to update these state representations, which in turn influence point cloud features through state changes. This enables fine-grained, state-specific interaction between geometry and flow information. Extensive experiments on two well-recognized aerodynamic benchmarks demonstrate that FSAN achieves the highest accuracy among the methods compared in this work at a higher computational cost. On Emmi-Wing, FSAN reduces the Relative L2 (REL-L2) error by over 20% compared to the strongest baseline (Transolver), and on DrivAerNet++, it achieves a 10% reduction compared to the strongest baseline (AdaField). These results establish FSAN as a promising neural surrogate on public benchmarks with diverse flow conditions and geometries.
Wenxuan Jin, Jianguo Yao, Haibing Guan +1
Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University