CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition

Hyungjoon Juen; Minwoo Shin

cs.LGOct 7, 2026

CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition

Authors: Hyungjoon Juen, Minwoo Shin

Organizations: Department of Software, Yonsei University (Mirae Campus), Wonju 26493, Republic of Korea

Abstract

The Fourier Neural Operator (FNO) learns solution operators of partial differential equations (PDEs) through Fourier-space kernel parameterization, but frequency truncation can limit the learning of high-frequency variations. AM-FNO and SirenFNO generate kernels for all grid modes from spectral coordinates using shared networks, making coordinate encoding and generator design important. Recent work on implicit neural representations (INRs) has proposed constructing frequency interactions through explicit feature composition rather than relying on subsequent MLPs to form them implicitly. Building on this approach, we propose CAFE+FNO, which incorporates Content-Aware Frequency Encoding+ (CAFE+) into Fourier kernel generation. CAFE+ combines Fourier--Chebyshev features through parallel affine branches and a Hadamard product, forming interactions within and across the two feature families. A kernel MLP maps the resulting representation of each normalized spectral coordinate to a complex channel-mixing matrix. Each layer shares its generator across all stored modes, making the number of trainable parameters independent of the number of modes for a fixed architecture. We compare CAFE+FNO with existing FNO variants on five PDE benchmarks and conduct ablation studies on basis configuration, multiplicative composition, and bandwidth learnability. Code and experimental configurations are available at https://github.com/fabsk101/CAFEPlusFNO.git.

Figures & tables

Figure 1: Architecture of CAFE+FNO in two dimensions. The input is lifted by

P

, processed by

L

operator layers, and projected by

Q

to the predicted solution. Within each layer, random Fourier features and Chebyshev features of the normalized spectral coordinate

\bm{\xi}_{k}

are concatenated and passed through parallel linear branches. Their outputs are combined via a Hadamard product, and a kernel MLP generates the mode-specific complex channel-mixing matrix used in the spectral convolution.

Dataset	Input	Target	Resolution	Train/test	Source
Darcy	$a(x)$	$u(x)$	$128\!\times\!128$	1000/200	NeuralOperator
NS	$u(x,t_{\mathrm{in}})$	$u(x,t_{\mathrm{out}})$	$128\!\times\!128$	1000/200	NeuralOperator
Burgers	$[u(x,t_{j})]_{j=0}^{9}$	$[u(x,t_{j})]_{j=10}^{19}$	$1024$	1000/200	PDEBench
Airfoil	$(X,Y)$	$u(x)$	$221\!\times\!51$	1000/200	Geo-FNO
ReacDiff	$[u(x,t_{j})]_{j=0}^{9}$	$[u(x,t_{j})]_{j=10}^{19}$	$1024$	1000/200	PDEBench

Table 1: Description of the benchmark datasets.

Model	Darcy		NS		Burgers		Airfoil		ReacDiff
	$128\times 128$		$128\times 128$		$1024$		$221\times 51$		$1024$
FNO	65.40 $\pm$	4.18	56.29 $\pm$	0.93	11.30 $\pm$	0.08	6.17 $\pm$	0.17	4.18 $\pm$	0.15
U-FNO	47.05 $\pm$	1.83	57.69 $\pm$	0.78	10.02 $\pm$	0.21	—		9.13 $\pm$	5.77
TFNO-CP	47.34 $\pm$	1.28	35.14 $\pm$	0.96	10.27 $\pm$	0.49	7.06 $\pm$	0.48	4.07 $\pm$	0.26
AM-FNO(MLP)	62.02 $\pm$	12.90	29.94 $\pm$	1.83	18.73 $\pm$	3.20	6.29 $\pm$	0.68	1758.09 $\pm$	3843.63
SirenFNO	24.34 $\pm$	1.24	43.24 $\pm$	2.02	5.86 $\pm$	0.41	5.76 $\pm$	0.24	2.36 $\pm$	0.78

Table 2: Final test relative

L_{2}

errors (

\times 10^{-3}

) at epoch 500: mean

\pm

sample standard deviation over five seeds. Bold marks the lowest mean per dataset. AM-FNO uses the MLP variant; U-FNO was excluded a priori on Airfoil (—). Evaluation details appear in Appendix A .

Figure 2: Comparison of FNO, AM-FNO, SirenFNO, and CAFE+FNO on a Navier–Stokes (NS) sample using trained checkpoints. The top row shows the ground-truth vorticity and model predictions; the bottom row shows pointwise absolute errors. Relative $L_{2}$ errors are reported below the predictions.

Model	Darcy		NS		Burgers		Airfoil		ReacDiff
	$128\times 128$		$128\times 128$		$1024$		$221\times 51$		$1024$
CAFE+FNO	13.63 $\pm$	0.12	17.25 $\pm$	0.29	4.11 $\pm$	0.37	4.69 $\pm$	0.10	3.80 $\pm$	0.27
CAFE+FNO ∗	27.28 $\pm$	2.70	38.53 $\pm$	1.46	5.14 $\pm$	0.34	17.58 $\pm$	11.38	3.87 $\pm$	0.28
CAFE+FNO RFF	15.07 $\pm$	0.71	19.76 $\pm$	0.20	4.91 $\pm$	0.39	5.36 $\pm$	0.22	4.48 $\pm$	1.14
CAFE+FNO Cheb	14.46 $\pm$	0.71	17.81 $\pm$	0.14	4.16 $\pm$	0.24	4.87 $\pm$	0.30	4.00 $\pm$	0.24
CAFE+FNO †	13.69 $\pm$	0.14	17.22 $\pm$	0.26	4.13 $\pm$	0.37	4.69 $\pm$	0.12	3.65 $\pm$	0.29

Table 3: CAFE+FNO ablations: relative

L_{2}

errors (

\times 10^{-3}

). Evaluation and reporting conventions follow Table 2 . Bold marks the lowest unrounded mean per dataset. ∗ : no composition block, with the kernel MLP input dimension matched to the default model (16 for ReacDiff and 32 otherwise). RFF/Cheb: single-basis variants retaining the composition block. † : learnable bandwidth with a fixed Gaussian matrix.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Batch size	Prediction protocol	Evaluation space
Darcy	32	Direct prediction of the provided input–output field pairs	After output denormalization
Navier–Stokes	32	Direct prediction of the provided single input–output field pairs	After output denormalization
Burgers	32	Autoregressive prediction of 10 future snapshots from 10 input snapshots	Normalized space
Airfoil	8	Direct prediction of the target field from coordinates	Original value space
ReacDiff	32	Autoregressive prediction of 10 future snapshots from 10 input snapshots	Original value space

Appendix

Table 4: Training and evaluation settings.

Dataset	Dense branch output dimension	Dense MLP hidden width	CP rank	TT rank	Tucker per-axis rank
Darcy	32	32	8	8	10
Navier–Stokes	32	64	16	16	16
Burgers	32	32	8	8	8
Airfoil	32	32	8	8	8
ReacDiff	16	32	8	8	8

Appendix

Table 5: Dataset-specific architectural settings for CAFE+FNO. The branch output dimension and MLP hidden width refer to the dense model; the common settings for the decomposed models are described in the text.

Model	Darcy	NS	Burgers	Airfoil	ReacDiff
Resolution	$128\times 128$	$128\times 128$	$1024$	$221\times 51$	$1024$
FNO	1,192,801	4,469,601	4,216,161	2,372,513	4,216,161
U-FNO	5,931,201	990,753	1,883,073	—	1,892,161
TFNO-CP	72,913	237,505	226,369	131,993	226,369
AM-FNO (MLP)	385,473	4,443,137	823,073	1,136,673	823,073
SirenFNO	308,865	579,457	308,993	308,897	304,833

Appendix

Table 6: Trainable parameter counts in real scalars, including lifting, operator layers, and projection. Each complex parameter counts as two real scalars; shared parameters are counted once. Fixed buffers are excluded, and kernel-generator outputs are not counted as additional parameters. The dash denotes an omitted model–dataset combination.

Model	Darcy	NS	Burgers	Airfoil	ReacDiff
Resolution	$128\!\times\!128$	$128\!\times\!128$	$1024$	$221\!\times\!51$	$1024$
FNO	$6.200\mathrm{e}{-2}$	$5.610\mathrm{e}{-2}$	$1.135\mathrm{e}{-2}$	$6.047\mathrm{e}{-3}$	$4.252\mathrm{e}{-3}$
U-FNO	$4.790\mathrm{e}{-2}$	$5.824\mathrm{e}{-2}$	$1.005\mathrm{e}{-2}$	—	$7.578\mathrm{e}{-3}$
TFNO-CP	$4.928\mathrm{e}{-2}$	$3.394\mathrm{e}{-2}$	$9.423\mathrm{e}{-3}$	$7.026\mathrm{e}{-3}$	$4.185\mathrm{e}{-3}$
AM-FNO (MLP)	$7.497\mathrm{e}{-2}$	$3.113\mathrm{e}{-2}$	$2.338\mathrm{e}{-2}$	$6.519\mathrm{e}{-3}$	$1.001\mathrm{e}{-2}$
SirenFNO	$2.587\mathrm{e}{-2}$	$4.460\mathrm{e}{-2}$	$5.392\mathrm{e}{-3}$	$5.691\mathrm{e}{-3}$	$2.148\mathrm{e}{-3}$

Appendix

Table 7: Test relative

L_{2}

errors for seed

0

after 500 training epochs. The dash denotes an omitted model–dataset combination.

Model	Darcy	NS	Burgers	Airfoil	ReacDiff
Resolution	$128\!\times\!128$	$128\!\times\!128$	$1024$	$221\!\times\!51$	$1024$
FNO	$7.201\mathrm{e}{-2}$	$5.585\mathrm{e}{-2}$	$1.121\mathrm{e}{-2}$	$6.130\mathrm{e}{-3}$	$4.125\mathrm{e}{-3}$
U-FNO	$4.904\mathrm{e}{-2}$	$5.822\mathrm{e}{-2}$	$9.668\mathrm{e}{-3}$	—	$4.914\mathrm{e}{-3}$
TFNO-CP	$4.734\mathrm{e}{-2}$	$3.525\mathrm{e}{-2}$	$1.054\mathrm{e}{-2}$	$6.670\mathrm{e}{-3}$	$3.930\mathrm{e}{-3}$
AM-FNO (MLP)	$4.344\mathrm{e}{-2}$	$2.858\mathrm{e}{-2}$	$2.010\mathrm{e}{-2}$	$5.792\mathrm{e}{-3}$	$1.052\mathrm{e}{-1}$
SirenFNO	$2.285\mathrm{e}{-2}$	$4.578\mathrm{e}{-2}$	$5.617\mathrm{e}{-3}$	$6.068\mathrm{e}{-3}$	$2.384\mathrm{e}{-3}$

Appendix

Table 8: Test relative

L_{2}

errors for seed

42

after 500 training epochs. The dash denotes an omitted model–dataset combination.

Model	Darcy	NS	Burgers	Airfoil	ReacDiff
Resolution	$128\!\times\!128$	$128\!\times\!128$	$1024$	$221\!\times\!51$	$1024$
FNO	$6.389\mathrm{e}{-2}$	$5.573\mathrm{e}{-2}$	$1.134\mathrm{e}{-2}$	$6.310\mathrm{e}{-3}$	$3.944\mathrm{e}{-3}$
U-FNO	$4.521\mathrm{e}{-2}$	$5.775\mathrm{e}{-2}$	$1.011\mathrm{e}{-2}$	—	$1.927\mathrm{e}{-2}$
TFNO-CP	$4.770\mathrm{e}{-2}$	$3.488\mathrm{e}{-2}$	$1.030\mathrm{e}{-2}$	$6.828\mathrm{e}{-3}$	$3.715\mathrm{e}{-3}$
AM-FNO (MLP)	$6.956\mathrm{e}{-2}$	$2.806\mathrm{e}{-2}$	$1.490\mathrm{e}{-2}$	$7.247\mathrm{e}{-3}$	$8.633\mathrm{e}{0}$
SirenFNO	$2.517\mathrm{e}{-2}$	$4.055\mathrm{e}{-2}$	$6.347\mathrm{e}{-3}$	$5.532\mathrm{e}{-3}$	$3.682\mathrm{e}{-3}$

Appendix

Table 9: Test relative

L_{2}

errors for seed

73

after 500 training epochs. The dash denotes an omitted model–dataset combination.

Model	Darcy	NS	Burgers	Airfoil	ReacDiff
Resolution	$128\!\times\!128$	$128\!\times\!128$	$1024$	$221\!\times\!51$	$1024$
FNO	$6.690\mathrm{e}{-2}$	$5.584\mathrm{e}{-2}$	$1.122\mathrm{e}{-2}$	$6.000\mathrm{e}{-3}$	$4.239\mathrm{e}{-3}$
U-FNO	$4.812\mathrm{e}{-2}$	$5.636\mathrm{e}{-2}$	$1.008\mathrm{e}{-2}$	—	$7.520\mathrm{e}{-3}$
TFNO-CP	$4.607\mathrm{e}{-2}$	$3.504\mathrm{e}{-2}$	$1.046\mathrm{e}{-2}$	$7.891\mathrm{e}{-3}$	$4.407\mathrm{e}{-3}$
AM-FNO (MLP)	$5.421\mathrm{e}{-2}$	$3.245\mathrm{e}{-2}$	$1.719\mathrm{e}{-2}$	$5.511\mathrm{e}{-3}$	$1.133\mathrm{e}{-2}$
SirenFNO	$2.442\mathrm{e}{-2}$	$4.266\mathrm{e}{-2}$	$5.710\mathrm{e}{-3}$	$5.936\mathrm{e}{-3}$	$1.789\mathrm{e}{-3}$

Appendix

Table 10: Test relative

L_{2}

errors for seed

108

after 500 training epochs. The dash denotes an omitted model–dataset combination.

Model	Darcy	NS	Burgers	Airfoil	ReacDiff
Resolution	$128\!\times\!128$	$128\!\times\!128$	$1024$	$221\!\times\!51$	$1024$
FNO	$6.220\mathrm{e}{-2}$	$5.794\mathrm{e}{-2}$	$1.138\mathrm{e}{-2}$	$6.387\mathrm{e}{-3}$	$4.347\mathrm{e}{-3}$
U-FNO	$4.499\mathrm{e}{-2}$	$5.789\mathrm{e}{-2}$	$1.020\mathrm{e}{-2}$	—	$6.383\mathrm{e}{-3}$
TFNO-CP	$4.631\mathrm{e}{-2}$	$3.660\mathrm{e}{-2}$	$1.064\mathrm{e}{-2}$	$6.896\mathrm{e}{-3}$	$4.123\mathrm{e}{-3}$
AM-FNO (MLP)	$6.792\mathrm{e}{-2}$	$2.946\mathrm{e}{-2}$	$1.809\mathrm{e}{-2}$	$6.375\mathrm{e}{-3}$	$3.048\mathrm{e}{-2}$
SirenFNO	$2.341\mathrm{e}{-2}$	$4.262\mathrm{e}{-2}$	$6.246\mathrm{e}{-3}$	$5.552\mathrm{e}{-3}$	$1.774\mathrm{e}{-3}$

Appendix

Table 11: Test relative

L_{2}

errors for seed

202

after 500 training epochs. The dash denotes an omitted model–dataset combination.

Figure 3: Ground truth and model predictions for random test sample set 1.

Figure 4: Absolute error maps for random test sample set 1.

Figure 5: Ground truth and model predictions for random test sample set 2.

Figure 6: Absolute error maps for random test sample set 2.

Figure 7: Ground truth and model predictions for random test sample set 3.

Figure 8: Absolute error maps for random test sample set 3.

Explore similar work

SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operators
Jun 9, 2026Pengqing Shi, Jie Yin, Stephen Tierney +1Fourier Neural Operator Spectral Bias
The University of Sydney, Australia
Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs
Jun 26, 2026Alex Colagrande, Paul Caillon, Eva Feillet +1Fourier Neural Operator Neural PDE Solvers
Miles Team, LAMSADE, Université Paris Dauphine-PSL, Paris, France · Université Paris-Saclay, CNRS, LISN, 91400, Orsay, France · ESPCI PSL, Paris, France
ADEx-FNO: A Unified Ambient-Domain Framework for Fourier Neural Operators on Varying Geometries
Aug 9, 2026Roberto Nuca, Giovanni Testa, Luca Galimberti +1Fourier Neural Operator PDE Surrogate Modeling
Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia · Dipartimento di Scienze e Tecnologie Aerospaziali, Politecnico di Milano, Milan, Italy · Physical Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia

Jun 9, 2026, cs.LGJ/K move · Enter open · S save

SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operators

Pengqing Shi, Jie Yin, Stephen Tierney, Junbin Gao

The University of Sydney, Australia

Fourier Neural Operator Spectral Bias PDE Operator Learning

Open paper AskPDF arXiv

Fourier neural operators (FNOs) are effective and efficient surrogates for approximating solutions of PDEs and generalize across discretizations. However, owing to the reliance on frequency truncation to maintain learning efficiency of FNOs, empirical studies suggest that FNOs exhibit spectral bias toward low-frequency information, which may hinder the learning capability especially for certain PDEs with strong high-frequency oscillations. To address this limitation, we propose SirenFNO, a novel framework that leverages sinusoidal representation networks (SIRENs) to learn implicit neural representations and performs mode-wise kernel parameterization. Our SIREN parameterization learns a full-grid spectrum with a constant and discretization-independent parameter count, thereby eliminating the need for frequency truncation. We further extend SirenFNO with functional tensor decompositions to enhance parameter and learning efficiency. Empirical results show that our SirenFNO consistently outperforms FNO with approximately

4

15

times parameter reductions with preserved discretization invariance, and our functional decomposition variants obtain performance improvements with a maximum of

73

times fewer parameters across multiple PDE benchmarks.

Abstract

Explore similar work

SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operators

Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs

ADEx-FNO: A Unified Ambient-Domain Framework for Fourier Neural Operators on Varying Geometries