Organizations: CEREA, ENPC, EDF R&D, Institut Polytechnique de Paris, Île-de-France, France · EDF R&D, Chatou, France · Research Division, FM, 1165 Boston-Providence Turnpike, Norwood, MA 02062, USA · Aix-Marseille Université, CNRS, IUSTI UMR 7343, Marseille, France
Radiative heat transfer plays a dominant role in fire simulations, where solving the radiative transfer equation (RTE) is a computational bottleneck. Neural operators can accelerate these computations by learning solution operators from offline simulation data. However, conventional DeepONet surrogates rely on fixed-grid inputs and affine reconstruction, limiting their flexibility across resolutions and their approximation accuracy at a fixed latent dimension. In this paper, we develop a transformer-based DeepONet for multi-resolution RTE surrogate modelling. The proposed approach combines a Perceiver encoder, which maps temperature and absorption coefficient fields at different spatial resolutions into a fixed-size latent representation, with a cross-attention decoder that enables non-affine reconstruction of the radiative intensity. This architecture allows a single model to process inputs at varying resolutions without resampling them onto a common grid. We formulate a computable empirical version of a previously derived theoretical lower bound for affine reconstruction, enabling direct comparison with model errors on the same dataset. Experiments on a two-dimensional pool-fire dataset demonstrate that the proposed approach outperforms autoencoder-based DeepONets in predicting both radiative intensity and incident radiation, with fewer trainable parameters. Power spectral density analysis shows improved preservation of small-scale structures. Both affine baselines remain above the corresponding lower bound at all evaluated ranks, whereas the proposed model achieves a training error below it at one matched latent dimension, demonstrating the potential of non-affine reconstruction to overcome this representational limitation. Experiments on a dataset combining three spatial resolutions confirm the accuracy advantage of the approach at each input resolution.
Figures & tables
Figure 1: Comparison of encoding and reconstruction architectures. (a): In PCA- and AE-DeepONet baselines, the input sensor grid and the branch/trunk output widths ( p ) are strictly fixed. Consequently, multi-resolution inputs require explicit resampling, and the final inner product yields an affine reconstruction constrained by a theoretical error bound. (b): The proposed transformer-based DeepONet dynamically adapts the number of patch tokens ( L ) to the input resolution while maintaining a fixed-size latent state ( z∈RN×D ), natively handling varying resolutions without resampling. Furthermore, its cross-attention reconstructor evaluates the query as a non-affine function of the latent state, overcoming the affine representational bottleneck. Both models evaluate the intensity pointwise given the same query ξ=(x,y,φ) .
Figure 2: The two baselines. (a) PCA-DeepONet. Each input field is projected on a PCA basis learned beforehand on the training snapshots, which reduces it from 22500 values to r coefficients, r being set by an energy criterion (Sec. IV.1.1 ). Two branch nets and a Hadamard product turn these coefficients into a coefficient vector β∈Rp , paired with the trunk output by an inner product and a scalar bias. (b) AE-DeepONet. Same architecture, with the PCA replaced by the encoders of two convolutional autoencoders pretrained on T and κ . Only the encoding column changes; the branch–trunk junction is left untouched. In both models the final junction is affine in β and of rank p .
Figure 3: Transformer-based DeepONet. The input pair (T,κ) , given on any H×W grid, is cut into patches and embedded into L tokens, L following the resolution; N trainable latent queries zq collect them in one cross-attention pass, and J Transformer blocks refine the result into the latent state z∈RN×D , whose size is the only one frozen at training time. The reconstructor embeds a query ξ with random Fourier features and lets it attend to z through C cross-attention blocks, so that the prediction is a nonaffine function of z and of ξ . The two heads correspond to the two uses of the network: (a) reconstruction of the input pair, ξ=(x,y) , used as a pre-training, and (b) prediction of the intensity, ξ=(x,y,φ) , which reuses the encoder weights optimised in (a).
Reconstruction relative error (L²) (%)
Encoder
d ( T / κ )
T
κ
PCA
500 / 500
5.46
11.86
Convolutional autoencoder
500 / 500
1.32
3.24
Table 1: Relative L2 error of the reconstruction of the input pair, on the test set. Both encoders are given the same compression dimension, d=500 per field.
Figure 4: Reconstruction of one test snapshot, T on top and κ below. From left to right: the solver field, the PCA reconstruction ( d=500 ), the autoencoder reconstruction ( d=500 ), and the absolute error of each reconstruction at every grid point, as a percentage of the maximum of the solver field on this snapshot.
Relative L2 error (%)
Model
Reconstructor
I
G
Parameters
PCA-DeepONet ( p=128 )
affine + softplus
7.31
5.43
3 075 969
PCA-DeepONet ( p=128 )
affine
7.62
5.73
3 075 969
PCA-DeepONet ( p=256 )
affine + softplus
6.07
4.64
3 272 961
PCA-DeepONet ( p=256 )
affine
6.20
4.81
3 272 961
PCA-DeepONet ( p=512 )
affine + softplus
5.62
4.34
3 666 945
Table 2: Relative L2 error of the prediction of the intensity I and of the incident radiation G , on the test set. The last column gives the number of parameters optimised during the prediction training. The frozen PCA basis is therefore not counted, whereas the fine-tuned encoders of the AE-DeepONet and of the transformer-based DeepONet are counted.
Figure 5: Incident radiation G on one test snapshot: solver field, prediction, and pointwise relative error. (a) PCA-DeepONet, p=512 , affine reconstructor. (b) AE-DeepONet, p=512 , affine reconstructor. (c) Transformer DeepONet, N=4 latent tokens of width D=128 , nonaffine reconstructor.
Figure 6: Power spectral density Eq. ( 21 ) of the incident radiation G on the test set. From left to right: the solver (FVDOM), the PCA-DeepONet ( p=512 ), the AE-DeepONet ( p=512 ) and the transformer-based DeepONet ( N=4 , D=128 ). Thin lines are 120 test snapshots drawn at random, the thick line is the mean over the 901 test snapshots, and the dashed line repeats the solver mean on every panel.
Figure 7: Lower bound Eq. ( 17 ) on the error of an affine reconstructor of rank p , on the training set (left, M=4200 snapshots) and on the test set (right, M=901 ). The shaded region is out of reach of any affine reconstructor. The markers are the models of Table 2 , placed at p for the two baselines and at N×D for the transformer-based DeepONet. The open circle marks p⋆ , the rank at which the bound comes down to the error of the best transformer-based DeepONet.
Relative L2 error (%)
Model
Encoder
Reconstructor
I
G
E/bound (train / test)
PCA-DeepONet ( p=128 )
affine
affine
7.62
5.73
1.15 / 1.35
AE-DeepONet ( p=128 )
nonlinear
affine
7.70
5.79
1.14 / 1.36
Perceiver ( p=128 )
Perceiver
affine
7.14
5.34
1.05 / 1.29
Transformer DeepONet
Perceiver
cross-attention
5.79
4.05
0.80 / 1.03
Table 3: Effect of the encoder at fixed reconstructor. The three affine models share the reconstruction Eq. ( 5e ) of rank p=128 and differ only in the way the input pair is encoded. The last column gives the ratio of the measured error to the bound Eq. ( 20 ) at the latent dimension of each model.
Figure 8: Temperature (top) and absorption coefficient (bottom) of one snapshot at the three resolutions of the dataset, 252 , 752 and 1502 from left to right. The same block averaging is applied to the intensity.
Input resolution
εL2(I) (%)
252
752
1502
All
PCA-DeepONet @ 1502
15.56
9.25
9.13
11.22
PCA-DeepONet @ 502
15.92
9.98
9.70
11.78
AE-DeepONet @ 1502
15.70
9.45
9.40
11.42
AE-DeepONet @ 502
15.90
10.13
9.87
11.88
Transformer DeepONet
8.81
6.54
6.86
7.37
Table 4: Relative L2 error on the intensity, for models trained on the multi-resolution dataset, as a function of the resolution at which the test input pair is supplied; the reference is the 1502 solver field in every case. @ denotes the working grid of the fixed-size encoders. Both baselines use the pure affine reconstructor of Fig. 2 .
Figure 9: Test snapshot whose input pair is supplied at 252 , the prediction being requested on the 1502 solver mesh. Top row: incident radiation G predicted by the model named above each panel. Bottom row: absolute error against the reference, as a percentage of ∥G∥∞ . Each row shares one colour scale.
Figure 10: Same as Fig. 9 , for a test snapshot supplied at 752 .
Figure 11: Same as Fig. 9 , for a test snapshot supplied at 1502 , the resolution of the solver mesh.
Computational fluid dynamics (CFD) has become an essential tool for predicting fire behavior, yet maintaining both efficiency and accuracy remains challenging. A major source of computational cost in fire simulations is the modeling of radiation transfer, which is usually the dominant heat transfer mechanism in fires. Solving the high-dimensional radiative transfer equation (RTE) with traditional numerical methods can be a performance bottleneck. Here, we present a machine learning framework based on Fourier-enhanced multiple-input neural operators (Fourier-MIONet) as an efficient alternative to direct numerical integration of the RTE. We first investigate the performance of neural operator architectures for a small-scale 2D pool fire and find that Fourier-MIONet provides the most accurate radiative solution predictions. The approach is then extended to 3D CFD fire simulations, where the computational mesh is locally refined across multiple levels. In these high-resolution settings, monolithic surrogate models for direct field-to-field mapping become difficult to train and computationally inefficient. To address this issue, a nested Fourier-MIONet is proposed to predict radiation solutions across multiple mesh-refinement levels. We validate the approach on 3D McCaffrey pool fires simulated with FireFOAM, including fixed fire sizes and a unified model trained over a continuous range of heat release rates (HRRs). The proposed method achieves global relative errors of 2-4% for 3D varying-HRR scenarios while providing faster inference than the estimated cost of one finite-volume radiation solve in FireFOAM for the 16-solid-angle case. With fast and accurate inference, the surrogate makes higher-fidelity radiation treatments practical and enables the incorporation of more spectrally resolved radiation models into CFD fire simulations for engineering applications.
Anran Jiao, Wengyao Jiang, Xiaoyi Lu +2
Department of Statistics and Data Science, Yale University, New Haven, CT 06511, USA · Department of Biostatistics, Yale University, New Haven, CT 06511, USA · FM, Research Division, Norwood, MA 02062, USA +1
Linear radiation transport equations (RTEs) form the simulation foundations underpinning design and analysis tasks in nuclear engineering, inertial confinement fusion, medical imaging, and astrophysics, but resolving the high-dimensional phase space at engineering fidelity remains expensive enough that outer-loop workflows, such as design optimization, uncertainty quantification, and parameter sweeps, are routinely budget-bound on traditional solvers. Neural surrogates promise to relax this bottleneck by amortizing simulation cost across thousands of downstream queries, but the architectural choices and engineered inductive biases that make a surrogate accurate on one transport problem do not transfer straightforwardly across model families. We benchmark two parameter-matched neural surrogate architectures, the physics-attention Transolver and the multi-scale graph network Bi-Stride Multi-Scale MeshGraphNet (BSMS-MGN), as end-to-end approximations of the final-time particle concentration for the two-dimensional linear RTE on the canonical Lattice and Hohlraum benchmarks. An ablation across Fourier features and region-weighted training loss exposes strongly architecture-dependent inductive-bias preferences, indicating that design choices common to physics-informed surrogate workflows must be revisited per architecture rather than imported across model families, and that downstream utility depends on per-QoI sensitivity rather than a single field-level score. The model training recipe, training data, and evaluation pipeline are released alongside this paper to support reproduction, transfer to related transport problems, and evaluation as amortized forward-model components in larger outer-loop simulation workflows.
Carmelo Gonzales, Steffen Schotthöfer, Cory D. Hauck
NVIDIA, Santa Clara, US · Seamless Labs, Mountain View, US · Oak Ridge National Laboratory
We study the problem of architecture selection for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators. We introduce the Multi-Scale Attention Transformer (\msat{}), a deep learning architecture that encodes spatiotemporal solution histories as token sequences and trains end-to-end via a composite supervised objective with optional physics-informed regularization terms. We conduct a comprehensive empirical evaluation against nine baselines -- including physics-informed neural networks (PINNs), neural operators (FNO, DeepONet, GNOT), and state-space models (Mamba-NO) -- across five benchmark problems from the PINNacle suite, using identical train/test splits and reference data for all methods. \msat{} achieves state-of-the-art generalization on complex geometry problems (Lrel2=0.0101 on Heat2D-CG, a 3.7× improvement over FNO) at 34s total inference vs.\ 120,812s for Mamba-NO. Ablation studies over the physics regularization component reveal a precise inductive bias tradeoff: physics priors reduce test error on diffusion-dominated problems but degrade generalization on chaotic and recirculating-flow regimes, directly characterizing the prior misspecification boundary. Approximation error bounds as a function of domain boundary complexity κ provide a theoretical basis for these empirical findings and a principled rule for architecture selection.