Beyond Affine Reconstruction: Multi-Resolution Transformer DeepONets for the Radiative Transfer Equation

Tanguy Azéma; Marc Bocquet; Fatiha Nmira; Xiaoyi Lu; Jean-Louis Consalvi; Sibo Cheng

cs.LGOct 24, 2026

Beyond Affine Reconstruction: Multi-Resolution Transformer DeepONets for the Radiative Transfer Equation

Authors: Tanguy Azéma, Marc Bocquet, Fatiha Nmira, Xiaoyi Lu, Jean-Louis Consalvi, Sibo Cheng

Organizations: CEREA, ENPC, EDF R&D, Institut Polytechnique de Paris, Île-de-France, France · EDF R&D, Chatou, France · Research Division, FM, 1165 Boston-Providence Turnpike, Norwood, MA 02062, USA · Aix-Marseille Université, CNRS, IUSTI UMR 7343, Marseille, France

Abstract

Radiative heat transfer plays a dominant role in fire simulations, where solving the radiative transfer equation (RTE) is a computational bottleneck. Neural operators can accelerate these computations by learning solution operators from offline simulation data. However, conventional DeepONet surrogates rely on fixed-grid inputs and affine reconstruction, limiting their flexibility across resolutions and their approximation accuracy at a fixed latent dimension. In this paper, we develop a transformer-based DeepONet for multi-resolution RTE surrogate modelling. The proposed approach combines a Perceiver encoder, which maps temperature and absorption coefficient fields at different spatial resolutions into a fixed-size latent representation, with a cross-attention decoder that enables non-affine reconstruction of the radiative intensity. This architecture allows a single model to process inputs at varying resolutions without resampling them onto a common grid. We formulate a computable empirical version of a previously derived theoretical lower bound for affine reconstruction, enabling direct comparison with model errors on the same dataset. Experiments on a two-dimensional pool-fire dataset demonstrate that the proposed approach outperforms autoencoder-based DeepONets in predicting both radiative intensity and incident radiation, with fewer trainable parameters. Power spectral density analysis shows improved preservation of small-scale structures. Both affine baselines remain above the corresponding lower bound at all evaluated ranks, whereas the proposed model achieves a training error below it at one matched latent dimension, demonstrating the potential of non-affine reconstruction to overcome this representational limitation. Experiments on a dataset combining three spatial resolutions confirm the accuracy advantage of the approach at each input resolution.

Figures & tables

Figure 1: Comparison of encoding and reconstruction architectures. (a): In PCA- and AE-DeepONet baselines, the input sensor grid and the branch/trunk output widths (

p

) are strictly fixed. Consequently, multi-resolution inputs require explicit resampling, and the final inner product yields an affine reconstruction constrained by a theoretical error bound. (b): The proposed transformer-based DeepONet dynamically adapts the number of patch tokens (

L

) to the input resolution while maintaining a fixed-size latent state (

z\in\mathbb{R}^{N\times D}

), natively handling varying resolutions without resampling. Furthermore, its cross-attention reconstructor evaluates the query as a non-affine function of the latent state, overcoming the affine representational bottleneck. Both models evaluate the intensity pointwise given the same query

\xi=(x,y,\varphi)

Panel of Figure 2 — Figure 2: The two baselines. (a) PCA-DeepONet. Each input field is projected on a PCA basis learned beforehand on the training snapshots, which reduces it from $22\,500$ values to $r$ coefficients, $r$ being set by an energy criterion (Sec. IV.1.1 ). Two branch nets and a Hadamard product turn these coefficients into a coefficient vector $\bm{\beta}\in\mathbb{R}^{p}$ , paired with the trunk output by an inner product and a scalar bias. (b) AE-DeepONet. Same architecture, with the PCA replaced by the encoders of two convolutional autoencoders pretrained on $T$ and $\kappa$ . Only the encoding column changes; the branch–trunk junction is left untouched. In both models the final junction is affine in $\bm{\beta}$ and of rank $p$ .

Figure 3: Transformer-based DeepONet. The input pair

(T,\kappa)

, given on any

H\times W

grid, is cut into patches and embedded into

L

tokens,

L

following the resolution;

N

trainable latent queries

\bm{z}_{q}

collect them in one cross-attention pass, and

J

Transformer blocks refine the result into the latent state

\bm{z}\in\mathbb{R}^{N\times D}

, whose size is the only one frozen at training time. The reconstructor embeds a query

\xi

with random Fourier features and lets it attend to

\bm{z}

through

C

cross-attention blocks, so that the prediction is a nonaffine function of

\bm{z}

and of

\xi

. The two heads correspond to the two uses of the network: (a) reconstruction of the input pair,

\xi=(x,y)

, used as a pre-training, and (b) prediction of the intensity,

\xi=(x,y,\varphi)

, which reuses the encoder weights optimised in (a).

		Reconstruction relative error (L²) (%)
Encoder	$d$ ( $T$ / $\kappa$ )	$T$	$\kappa$
PCA	500 / 500	5.46	11.86
Convolutional autoencoder	500 / 500	1.32	3.24

Table 1: Relative

L^{2}

error of the reconstruction of the input pair, on the test set. Both encoders are given the same compression dimension,

d=500

per field.

Figure 4: Reconstruction of one test snapshot, $T$ on top and $\kappa$ below. From left to right: the solver field, the PCA reconstruction ( $d=500$ ), the autoencoder reconstruction ( $d=500$ ), and the absolute error of each reconstruction at every grid point, as a percentage of the maximum of the solver field on this snapshot.

Model	Reconstructor	$I$	$G$	Parameters
		Relative $L^{2}$ error (%)
PCA-DeepONet ( $p=128$ )	affine + softplus	7.31	5.43	3 075 969
PCA-DeepONet ( $p=128$ )	affine	7.62	5.73	3 075 969
PCA-DeepONet ( $p=256$ )	affine + softplus	6.07	4.64	3 272 961
PCA-DeepONet ( $p=256$ )	affine	6.20	4.81	3 272 961
PCA-DeepONet ( $p=512$ )	affine + softplus	5.62	4.34	3 666 945

Table 2: Relative

L^{2}

error of the prediction of the intensity

I

and of the incident radiation

G

, on the test set. The last column gives the number of parameters optimised during the prediction training. The frozen PCA basis is therefore not counted, whereas the fine-tuned encoders of the AE-DeepONet and of the transformer-based DeepONet are counted.

Panel of Figure 5 — Figure 5: Incident radiation $G$ on one test snapshot: solver field, prediction, and pointwise relative error. (a) PCA-DeepONet, $p=512$ , affine reconstructor. (b) AE-DeepONet, $p=512$ , affine reconstructor. (c) Transformer DeepONet, $N=4$ latent tokens of width $D=128$ , nonaffine reconstructor.

Figure 6: Power spectral density Eq. ( 21 ) of the incident radiation

G

on the test set. From left to right: the solver (FVDOM), the PCA-DeepONet (

p=512

), the AE-DeepONet (

p=512

) and the transformer-based DeepONet (

N=4

D=128

). Thin lines are

120

test snapshots drawn at random, the thick line is the mean over the

901

test snapshots, and the dashed line repeats the solver mean on every panel.

Figure 7: Lower bound Eq. ( 17 ) on the error of an affine reconstructor of rank

p

, on the training set (left,

M=4200

snapshots) and on the test set (right,

M=901

). The shaded region is out of reach of any affine reconstructor. The markers are the models of Table 2 , placed at

p

for the two baselines and at

N\times D

for the transformer-based DeepONet. The open circle marks

p^{\star}

, the rank at which the bound comes down to the error of the best transformer-based DeepONet.

Model	Encoder	Reconstructor	$I$	$G$	$\widehat{\mathcal{E}}/\text{bound}$ (train / test)
			Relative $L^{2}$ error (%)
PCA-DeepONet ( $p=128$ )	affine	affine	7.62	5.73	1.15 / 1.35
AE-DeepONet ( $p=128$ )	nonlinear	affine	7.70	5.79	1.14 / 1.36
Perceiver ( $p=128$ )	Perceiver	affine	7.14	5.34	1.05 / 1.29
Transformer DeepONet	Perceiver	cross-attention	5.79	4.05	0.80 / 1.03

Table 3: Effect of the encoder at fixed reconstructor. The three affine models share the reconstruction Eq. ( 5e ) of rank

p=128

and differ only in the way the input pair is encoded. The last column gives the ratio of the measured error to the bound Eq. ( 20 ) at the latent dimension of each model.

Figure 8: Temperature (top) and absorption coefficient (bottom) of one snapshot at the three resolutions of the dataset, $25^{2}$ , $75^{2}$ and $150^{2}$ from left to right. The same block averaging is applied to the intensity.

$\varepsilon_{L^{2}}(I)$ (%)	$25^{2}$	$75^{2}$	$150^{2}$	All
	Input resolution
PCA-DeepONet @ $150^{2}$	15.56	9.25	9.13	11.22
PCA-DeepONet @ $50^{2}$	15.92	9.98	9.70	11.78
AE-DeepONet @ $150^{2}$	15.70	9.45	9.40	11.42
AE-DeepONet @ $50^{2}$	15.90	10.13	9.87	11.88
Transformer DeepONet	8.81	6.54	6.86	7.37

Table 4: Relative

L^{2}

error on the intensity, for models trained on the multi-resolution dataset, as a function of the resolution at which the test input pair is supplied; the reference is the

150^{2}

solver field in every case. @ denotes the working grid of the fixed-size encoders. Both baselines use the pure affine reconstructor of Fig. 2 .

Figure 9: Test snapshot whose input pair is supplied at $25^{2}$ , the prediction being requested on the $150^{2}$ solver mesh. Top row: incident radiation $G$ predicted by the model named above each panel. Bottom row: absolute error against the reference, as a percentage of $\|G\|_{\infty}$ . Each row shares one colour scale.

Figure 10: Same as Fig. 9 , for a test snapshot supplied at $75^{2}$ .

Figure 11: Same as Fig. 9 , for a test snapshot supplied at $150^{2}$ , the resolution of the solver mesh.

PDE Surrogate Modeling

PDE surrogate modeling builds fast approximations of partial differential equation solutions for simulation, prediction, and design.

See all 325 papers

Nested Fourier-enhanced neural operator for efficient modeling of radiation transfer in fires
Apr 15, 2026Anran Jiao, Wengyao Jiang, Xiaoyi Lu +2Radiative Transfer PDE Surrogate Modeling
Department of Statistics and Data Science, Yale University, New Haven, CT 06511, USA · Department of Biostatistics, Yale University, New Haven, CT 06511, USA · FM, Research Division, Norwood, MA 02062, USA +1
Efficient Neural Surrogates for Linear Radiation Transport on the Lattice and Hohlraum benchmarks
Oct 3, 2026Carmelo Gonzales, Steffen Schotthöfer, Cory D. HauckNeural Surrogate Modeling PDE Surrogate Modeling
NVIDIA, Santa Clara, US · Seamless Labs, Mountain View, US · Oak Ridge National Laboratory
When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains
May 8, 2026Brandon Yee, Pairie Koh, Jack Rodriguez +1Neural PDE Solvers PDE Solving
Physics Lab, Yee Collins Research Group

Apr 15, 2026J/K move · Enter open · S save

Nested Fourier-enhanced neural operator for efficient modeling of radiation transfer in fires

Anran Jiao, Wengyao Jiang, Xiaoyi Lu, Yi Wang+1

Department of Statistics and Data Science, Yale University, New Haven, CT 06511, USA · Department of Biostatistics, Yale University, New Haven, CT 06511, USA · FM, Research Division, Norwood, MA 02062, USA+1

Radiative Transfer PDE Surrogate Modeling Neural Operators Fourier Neural Operator Computational Fluid Dynamics

Open paper AskPDF arXiv

Computational fluid dynamics (CFD) has become an essential tool for predicting fire behavior, yet maintaining both efficiency and accuracy remains challenging. A major source of computational cost in fire simulations is the modeling of radiation transfer, which is usually the dominant heat transfer mechanism in fires. Solving the high-dimensional radiative transfer equation (RTE) with traditional numerical methods can be a performance bottleneck. Here, we present a machine learning framework based on Fourier-enhanced multiple-input neural operators (Fourier-MIONet) as an efficient alternative to direct numerical integration of the RTE. We first investigate the performance of neural operator architectures for a small-scale 2D pool fire and find that Fourier-MIONet provides the most accurate radiative solution predictions. The approach is then extended to 3D CFD fire simulations, where the computational mesh is locally refined across multiple levels. In these high-resolution settings, monolithic surrogate models for direct field-to-field mapping become difficult to train and computationally inefficient. To address this issue, a nested Fourier-MIONet is proposed to predict radiation solutions across multiple mesh-refinement levels. We validate the approach on 3D McCaffrey pool fires simulated with FireFOAM, including fixed fire sizes and a unified model trained over a continuous range of heat release rates (HRRs). The proposed method achieves global relative errors of 2-4% for 3D varying-HRR scenarios while providing faster inference than the estimated cost of one finite-volume radiation solve in FireFOAM for the 16-solid-angle case. With fast and accurate inference, the surrogate makes higher-fidelity radiation treatments practical and enables the incorporation of more spectrally resolved radiation models into CFD fire simulations for engineering applications.

Abstract

PDE Surrogate Modeling

Explore similar work

Nested Fourier-enhanced neural operator for efficient modeling of radiation transfer in fires

Efficient Neural Surrogates for Linear Radiation Transport on the Lattice and Hohlraum benchmarks

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains