Multi-stage neural operator learning with application for convolutions

Zhiping Mao; Zhenye Wen; Yong Zhang; Xiaofei Zhao

cs.LGAug 19, 2026

Multi-stage neural operator learning with application for convolutions

Authors: Zhiping Mao, Zhenye Wen, Yong Zhang, Xiaofei Zhao

Organizations: School of Mathematical Sciences, Eastern Institute of Technology, Ningbo, 315200 Ningbo, China · School of Mathematics and Statistics, Wuhan University, 430072 Wuhan, China · Center for Applied Mathematics and KL-AAGDM, Tianjin University, 300072 Tianjin, China · State Key Laboratory of Synthetic Biology, Tianjin University, Tianjin, 300072, China · Computational Sciences Hubei Key Laboratory, Wuhan University, 430072 Wuhan, China

Abstract

Convolution integrals widely exist in applications, and to enable fast and accurate computations, this paper introduces two general multi-stage neural operator learning frameworks. The first, Deep Collocation Neural Operator (DCNO), is a supervised approach that iteratively refines the operator approximation by learning residuals from input-output data pairs. The second, Deep Galerkin Neural Operator (DGNO), is an unsupervised framework applicable when the target operator can be represented by a PDE, leveraging the weak form of the PDE residual for training. Both methods progressively construct basis operators through multiple training stages to enrich the approximation space, leading to significantly improved accuracy over standard one-shot operator learning. We provide theoretical analysis for their approximation capabilities and implement them for learning convolutions. Extensive numerical experiments demonstrate that both DCNO and DGNO achieve high accuracy, approaching machine precision under single float for convolution problems, and offer substantial efficiency gains for numerous queries or parametric variations compared to traditional solvers. We also extend these frameworks to handle multi-input operator learning scenarios involving variations in both the density and kernel of a convolution.

Figures & tables

Panel of Figure 1 — Figure 1: Results of DCNO in Algorithm 1 for Example 5.1 : (a) Change of Loss $=\beta_{s-1}\sqrt{\text{Loss}_{s}}$ during the training; (b) Mean relative $L^{2}$ error with respect to the number of stages on test set.

	DCNO				DGNO
	$W_{0}=10$	$W_{0}=30$	$W_{0}=50$	$W_{0}=70$	$W_{0}=10$	$W_{0}=30$	$W_{0}=50$	$W_{0}=70$
$L=1$	1.75 $\,e\!-\!3$	1.81 $\,e\!-\!3$	1.20 $\,e\!-\!3$	5.37 $\,e\!-\!4$	2.99 $\,e\!-\!3$	2.15 $\,e\!-\!3$	1.27 $\,e\!-\!3$	6.70 $\,e\!-\!4$
$L=2$	1.94 $\,e\!-\!5$	1.80 $\,e\!-\!5$	1.15 $\,e\!-\!5$	9.46 $\,e\!-\!6$	5.80 $\,e\!-\!5$	3.17 $\,e\!-\!5$	2.38 $\,e\!-\!5$	1.62 $\,e\!-\!5$
$L=3$	4.01 $\,e\!-\!6$	3.17 $\,e\!-\!6$	1.48 $\,e\!-\!6$	8.29 $\,e\!-\!7$	1.68 $\,e\!-\!5$	6.03 $\,e\!-\!6$	3.47 $\,e\!-\!6$	9.20 $\,e\!-\!7$
$L=4$	1.61 $\,e\!-\!6$	1.24 $\,e\!-\!6$	7.04 $\,e\!-\!7$	5.47 $\,e\!-\!7$	1.43 $\,e\!-\!5$	3.10 $\,e\!-\!6$	1.89 $\,e\!-\!6$	6.17 $\,e\!-\!7$

Table 1: Relative

L^{2}

-errors obtained by DCNO and DGNO under different network architectures.

Panel of Figure 2 — Figure 2: Numerical solution (1st row) and pointwise error (2nd row) of DCNO for Example 5.1 on three test samples (a) $\alpha=1$ ; (b) $\alpha=3$ ; (c) $\alpha=5$ .

Panel of Figure 3 — Figure 3: Basis functions generated by the basis operators of DCNO method.

Panel of Figure 4 — Figure 4: Relative $L^{2}$ error of DCNO under varying noise levels (left); A Gaussian noise sample with noise level $0.001$ (middle); Pointwise error of DCNO for Example 5.1 on a test sample with $\alpha=1$ (right).

Panel of Figure 5 — Figure 5: Results of DGNO in Algorithm 2 for Example 5.1 : (a) Change of Loss during the iterations; (b) Mean relative $L^{2}$ error with respect to the number of stages.

	Stages	Epochs per stage	Training time(s)	$\text{Mean relative }L^{2}\,\text{error}$
supervised DCNO	5	3000	251.23s	1.31E-6
unsupervised DGNO	5	3000	491.60s	6.17E-7

Table 2: Number of stages, epochs per stage, training time and error for DCNO and DGNO.

Panel of Figure 6 — Figure 6: Solutions for different values of the kernel parameter $a$ (left); Mean relative $L^{2}$ error with respect to the number of stages for $a=0.0001$ (right).

Figure 7: Results of DCNO for Example 5.2 : (a) Exact and numerical solutions; (b) Error of the numerical solution; (c) Computational time of the traditional solver and DCNO against the number of computational cases.

$a$	$0.0001$	$0.001$	$0.001$	$0.1$
Error	1.20 $\,e\!-\!6$	3.08 $\,e\!-\!6$	2.71 $\,e\!-\!6$	3.48 $\,e\!-\!6$

Table 3: Mean relative

L^{2}

error of numerical solutions from DCNO for varying

a

Method	Evaluation time (s)	Training time (s)
Traditional method	670.44	—
DCNO method	0.021	202.14

Table 4: Comparison between DCNO and the traditional method SOE on test set.

Panel of Figure 8 — Figure 8: Results of Example 5.3 : (a) Relative $L^{2}$ error of $\psi$ over time; (b) Total evaluation time of the traditional solver and DGNO for solving the Poisson equation ( 5.3b ) during the simulation; (c) Evolution of the mass over time.

Panel of Figure 9 — Figure 9: $|\psi(\cdot,t=0)|^{2}$ (left); $|\psi(\cdot,t=1)|^{2}$ by DGNO (middle); Pointwise absolute error at $t=1$ (right) for Example 5.3 .

Panel of Figure 10 — Figure 10: Mean relative $L^{2}$ error with respect to the number of stages: (a) Example 5.4 ; (b) Example 5.5 .

Panel of Figure 11 — Figure 11: Numerical solution (1st row) and pointwise error (2nd row) of DCNO for Example 5.4 on three test samples: (a) $\alpha=\tau=2$ ; (b) $\alpha=\tau=1.5$ ; (c) $\alpha=\tau=1$ .

Panel of Figure 12 — Figure 12: Numerical solution (1st row) and pointwise error (2nd row) of DGNO for Example 5.5 on three test samples: (a) $\alpha=0.5,\,\gamma=2$ ; (b) $\alpha=0.6,\,\gamma=3$ ; (c) $\alpha=0.7,\,\gamma=4$ .

Explore similar work

Kernel Methods for Learning Operators with Multiple Inputs and Outputs
Aug 12, 2026Adrien Weihs, Chunyang Liao, Jingmin Sun +1Kernel Methods PDE Operator Learning
Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA. · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701, USA. · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA.
Beyond Convolution: A Taxonomy of Structured Operators for Learning-Based Image Processing
Mar 12, 2026Simone CammarasanaConvolutional Neural Networks Dynamic Convolution
CNR-IMATI, Via De Marini 6, Genova, Italy
Starter-Iterator Neural Operator: A Unified Architecture for High-Fidelity Forward and Inverse PDE Problems
Jun 16, 2026Kuilin Qin, Lianfang Wang, Xu Sun +4PDE Surrogate Modeling PDE Operator Learning

Aug 12, 2026J/K move · Enter open · S save

Kernel Methods for Learning Operators with Multiple Inputs and Outputs

Adrien Weihs, Chunyang Liao, Jingmin Sun, Hayden Schaeffer

Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA. · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701, USA. · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA.

Kernel Methods PDE Operator Learning Operator Learning

Open paper AskPDF arXiv

Learning mappings between infinite-dimensional objects is a central challenge in scientific machine learning. We introduce a general kernel-based encoder-decoder framework for operator learning that separates observation, representation, learning, and reconstruction. We develop this framework for multi-input, multi-output operator learning, where operators map between products of potentially distinct function spaces. Our approximation theory shows that, although the number of inputs and outputs can increase, the convergence rate is governed by the most challenging constituent approximation problem rather than the overall problem dimension. The framework leads to practical kernel methods with closed-form training and inference, combining mathematical tractability with computational efficiency. We further specialize the approach to multiple operator learning by introducing KernelMO, a family of kernel methods with complementary operator-valued and product-space formulations. Across five families of parametric partial differential equations, the proposed methods achieve competitive or state-of-the-art predictive accuracy while reducing training and inference costs relative to neural operator architectures and deep learning based models, offering an efficient and lightweight alternative.

Abstract

Explore similar work

Kernel Methods for Learning Operators with Multiple Inputs and Outputs

Beyond Convolution: A Taxonomy of Structured Operators for Learning-Based Image Processing

Starter-Iterator Neural Operator: A Unified Architecture for High-Fidelity Forward and Inverse PDE Problems