Neural Operators Which Certifiably Simultaneously Solve Infinitely Many Convex Programs using Logarithmic Rank and Linear Complexity

Anastasis Kratsios; Ariel Neufeld; Philipp Schmocker

cs.LGAug 20, 2025

Neural Operators Which Certifiably Simultaneously Solve Infinitely Many Convex Programs using Logarithmic Rank and Linear Complexity

Authors: Anastasis Kratsios, Ariel Neufeld, Philipp Schmocker

Organizations: Department of Mathematics, McMaster University and the Vector Institute, Canada · Division of Mathematical Sciences, Nanyang Technological University, Singapore

Abstract

Neural operators (NOs) are a class of deep learning models designed to simultaneously solve infinitely many related problems by casting them into an infinite-dimensional space, whereon these NOs operate. Such operator learning problems arise naturally in mathematical finance, for instance, when learning families of optimal portfolios or hedging strategies. Two significant gaps, however, remain between theory and practice: 1) most universal approximation guarantees do not describe practically implementable NOs or are pure existence statements; and 2) worst-case parameter bounds from universal approximation theorems suggest that constructed NOs may require an unrealistically large number of parameters to solve most operator learning problems, which stands in direct opposition to a slew of experimental evidence. This paper closes these gaps by exhibiting a short and simple training algorithm to construct a specific class of NOs, proximal equilibrium operators (PEOs), using (realistic) finite-dimensional deep equilibrium layers to solve families of convex optimization problems over a separable Hilbert space $X$ . Here, the inputs are smooth, convex loss functions on $X$ , and the outputs are the associated (approximate) solutions to the optimization problems defined by each input loss. Our main result is a certificate showing that, when the input losses lie in a suitable infinite-dimensional set, our PEOs trained with our algorithm do in fact uniformly approximate the corresponding solutions to arbitrary precision, with rank and width growing only logarithmically and the depth growing linearly in the reciprocal of the approximation error. We then validate both our theoretical results and the trainability of PEOs on three applications: non-linear PDEs, stochastic optimal control, and portfolio optimization and hedging problems in mathematical finance.

Figures & tables

Panel of Figure 1 — Figure 1: Learning the map ( 4.4 ) returning the solution of the parametric PDE ( 4.3 ) by a Proximal Equilibrium Operator $\mathcal{G}$ . In (a), the learning performance is displayed in terms of the mean squared error (MSE) $\frac{1}{|K|}\sum_{k\in K}\|\mathcal{S}(g_{\sigma_{k}})-\mathcal{G}(g_{\sigma_{k}})\|^{2}$ on the training set (label “Train”) and test set (label “Test”). In (b), the predicted solution $\mathcal{G}(g_{\sigma_{k}})$ (label “Predict”) is compared to the true solution $y(T,\cdot)=\mathcal{S}(g_{\sigma_{k}})$ (label “True”) for two $\sigma_{k}$ of the test set.

Table 1: Computation of optimal portfolios for power and logarithmic utility functions given by

u(x)\stackrel{{\scriptstyle\mbox{\tiny def.}}}{{=}}\frac{(x-x_{0})^{1-\eta}}{1-\eta}

\eta\in(0,\infty)\setminus\{1\}

and

u(x)\stackrel{{\scriptstyle\mbox{\tiny def.}}}{{=}}\ln(x-x_{0})

\eta=1

, where

x_{0}\geq 0

is the reference point.

Panel of Figure 2 — Figure 2: Learning the solution operator $\mathcal{S}$ of the utility maximization problem ( 4.7 ) by a Proximal Equilibrium Operator $\mathcal{G}$ . In (a), the learning performance is displayed in terms of the mean squared error (MSE) $\frac{1}{|K|}\sum_{k\in K}\|\mathcal{S}(g_{u_{k}})-\mathcal{G}(g_{u_{k}})\|^{2}$ on the training set (label “Train”) and test set (label “Test”). In (b), the predicted solution $\mathcal{G}(g_{u_{k}})$ (label “Predict”) is compared to the true solution $\mathcal{S}(g_{u_{k}})$ (label “True”) for two $\eta_{k}$ of the test set and some fixed $\omega\in\Omega$ .

Panel of Figure 3 — Figure 3: Learning the pricing/hedging operator $\mathcal{S}$ in ( 4.9 ) by a Proximal Equilibrium Operator $\mathcal{G}$ . In (a), the learning performance is displayed in terms of the mean squared error (MSE) $\frac{1}{|K|}\sum_{k\in K}\|\mathcal{S}(g_{H_{k}})-\mathcal{G}(g_{H_{k}})\|^{2}$ on the training set (label “Train”) and test set (label “Test”). In (b), the predicted solution $\mathcal{G}(g_{H_{k}})$ (label “Predict”) is compared to the true solution $\mathcal{S}(g_{H_{k}})$ (label “True”) for one $k$ of the test set and some $\omega\in\Omega$ .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Learning the minimization operator $\mathcal{S}$ in ( A.1 ) by a Proximal Equilibrium Operator $\mathcal{G}$ . In (a), the learning performance is displayed in terms of the mean squared error (MSE) $\frac{1}{|K|}\sum_{k\in K}\|\mathcal{S}(g_{k})-\mathcal{G}(g_{k})\|^{2}$ on the training set (label “Train”) and test set (label “Test”). In (b), the predicted solution $\mathcal{G}(g_{k})$ (label “Predict”) is compared to the true solution $\mathcal{S}(g_{k})$ (label “True”) for four $k$ of the test set.

Neural Operators

Neural operators learn mappings between functions, enabling fast surrogates for PDEs and other continuous physical systems.

See all 300 papers

Multi-stage neural operator learning with application for convolutions
Aug 19, 2026Zhiping Mao, Zhenye Wen, Yong Zhang +1Operator Learning Neural Operators
School of Mathematical Sciences, Eastern Institute of Technology, Ningbo, 315200 Ningbo, China · School of Mathematics and Statistics, Wuhan University, 430072 Wuhan, China · Center for Applied Mathematics and KL-AAGDM, Tianjin University, 300072 Tianjin, China +2
Quantitative Sobolev Approximation Bounds for Neural Operators with Empirical Validation on Burgers Equation
May 4, 2026Nicole HaoFourier Neural Operator PDE Solving
Department of Mathematics, Cornell University
Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization
May 31, 2026Jia-Qi Yang, Lei ShiNeural Network Approximation Theory Operator Learning
School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.

Aug 19, 2026J/K move · Enter open · S save

Multi-stage neural operator learning with application for convolutions

Zhiping Mao, Zhenye Wen, Yong Zhang, Xiaofei Zhao

School of Mathematical Sciences, Eastern Institute of Technology, Ningbo, 315200 Ningbo, China · School of Mathematics and Statistics, Wuhan University, 430072 Wuhan, China · Center for Applied Mathematics and KL-AAGDM, Tianjin University, 300072 Tianjin, China+2

Operator Learning Neural Operators PDE Operator Learning Neural Network Approximation Theory

Open paper AskPDF arXiv

Convolution integrals widely exist in applications, and to enable fast and accurate computations, this paper introduces two general multi-stage neural operator learning frameworks. The first, Deep Collocation Neural Operator (DCNO), is a supervised approach that iteratively refines the operator approximation by learning residuals from input-output data pairs. The second, Deep Galerkin Neural Operator (DGNO), is an unsupervised framework applicable when the target operator can be represented by a PDE, leveraging the weak form of the PDE residual for training. Both methods progressively construct basis operators through multiple training stages to enrich the approximation space, leading to significantly improved accuracy over standard one-shot operator learning. We provide theoretical analysis for their approximation capabilities and implement them for learning convolutions. Extensive numerical experiments demonstrate that both DCNO and DGNO achieve high accuracy, approaching machine precision under single float for convolution problems, and offer substantial efficiency gains for numerous queries or parametric variations compared to traditional solvers. We also extend these frameworks to handle multi-input operator learning scenarios involving variations in both the density and kernel of a convolution.

Abstract

Neural Operators

Explore similar work

Multi-stage neural operator learning with application for convolutions

Quantitative Sobolev Approximation Bounds for Neural Operators with Empirical Validation on Burgers Equation

Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization