Nonlinear Operators

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 20

Sep 16, 2026stat.ML

Fast Learning Rates for Physics-Informed Kernel Methods

In physics-informed machine learning, a target function

u^*

is learned from noisy value observations

y_i=u^*(x_i)+ \varepsilon_i

, together with differential information, given either by noisy observations

d_j=(Du^*)(z_j)+ξ_j

or by a known physical constraint

Du^*=v

. We consider the setting where

D

is a linear differential operator and analyze a physics-informed kernel estimator

\hat u

combining

n

value observations and

m

differential observations. In this context, we ask how much can differential information improve predictions, and how does this improvement depend quantitatively on

n

,

m

, and

D

. We prove finite-sample bounds, supported by numerical simulations, revealing a two-regime structure for the prediction error. When

m

is limited, the rate depends jointly on

n

and

m

; when

m

exceeds a problem-dependent threshold, the rate saturates and matches the oracle rate obtained when the perfect constraint

D \hat u = Du^*

is imposed. Examples are discussed for Sobolev spaces which are reproducing kernel Hilbert spaces and include partial Laplacian constraints on the torus and gradient observations on bounded domains. These examples illustrate the range of possible learning rate improvements --- from the standard nonparametric

n^{-1/4}

to the parametric rate

n^{-1/2}

. Finally, we derive physically consistent rates in a stronger norm that jointly controls the errors in

\hat u

and

D\hat u

.

Luc Brogat-Motte, Joachim Bona-Pellissier, Giacomo Meanti +1

SAIL Unit, Istituto Italiano di Tecnologia, Genoa, Italy · MaLGa Center, DIBRIS, Università degli Studi di Genova, Genoa, Italy

Physics-Informed Learning Nonlinear Operators Kernel Method Batch

Aug 1, 2026cs.LG

Modeling Unknown Nonlocal PDE Systems via Flow Map Learning

Nonlocal partial differential equations arise in many applications but are often difficult to model and learn because of the presence of nonlocal operators. We present a flow-map learning (FML) framework for modeling unknown nonlocal PDEs directly from solution data. Rather than learning or approximating the underlying nonlocal operators, the proposed approach learns the finite-time evolution operator in either modal or nodal space. Two complementary formulations are developed for spectral and grid-based solution representations. Numerical experiments on one- and two-dimensional fractional diffusion and wave equations demonstrate accurate and stable long-time prediction using only short observation windows. The proposed approach provides an effective data-driven framework for learning unknown nonlocal dynamics without explicit evaluation of nonlocal operators.

Zhongshu Xu, Ying Li, Yanzhi Zhang +1

Department of Mathematics, The Ohio State University, Columbus, OH 43210, USA. · Department of Mathematics and Statistics, Missouri University of Science and Technology, Rolla, MO 65409, USA.

Partial Differential Equations Nonlinear Operators Flow Map

Jul 31, 2026cs.NE

Linear Proposal Operators and Stochastic Search Geometry in SOMA and Differential Evolution

Swarm and evolutionary algorithms are usually analyzed as complete procedural systems in which nonlinear selection, replacement, and adaptation obscure simpler structure within candidate generation. This paper introduces an operator--selection factorization that separates objective-independent variation from boundary repair and fitness-dependent selection, and uses it to study the proposal geometry of the Self-Organizing Migrating Algorithm (SOMA) and Differential Evolution (DE). The canonical SOMA proposal is shown to be affine in the search space and exactly linear in an augmented migrant--leader state. In leader-relative coordinates, the resulting operator provides a direct interpretation of interpolation, projection, overshooting, and coordinate masking. Under Bernoulli perturbation masks, we derive closed-form expressions for the proposal mean, covariance, expected squared step length, expected squared distance from the leader, active dimensionality, and coordinate coverage. For canonical DE/rand/1/bin, we derive the finite-population moments of differential mutation and characterize the additional covariance and coordinate dependence induced by forced-coordinate binomial crossover. Exact enumeration and Monte Carlo experiments verify the analytical identities and quantify the effects of mask conditioning, boundary repair, and fitness-based selection. The analysis further motivates geometry-controlled and rotation-aware SOMA variants, together with an adaptive population-reducing extension of iSOMA. Experiments on the complete noiseless BBOB benchmark show that these operator-guided variants substantially improve upon canonical SOMA and are competitive with established DE methods in several dimension--budget regimes. The results demonstrate how proposal-level operator analysis can support both the interpretation and design of population-based optimizers.

Vojtěch Novák, Ivan Zelinka

Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB - Technical University of Ostrava, Ostrava, Czech Republic · IT4Innovations National Supercomputing Center, VSB - Technical University of Ostrava, 708 00 Ostrava, Czech Republic · Department of Informatics and Statistics, Marine Research Institute, Klaipeda University, Lithuania

Evolutionary Optimization Nonlinear Operators

Jul 27, 2026cs.LG

Variational Boosting for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.

Kaylee Vo, Pavlos Protopapas

Harvard University

Parametric Physics-Informed Neural Network Variational Nonlinear Operators

Jul 3, 2026cs.LG

Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI

Conventional uniform convergence bounds and empirical risk minimization break down in massive over-parameterized models, such as large language transformers and biological sequence networks. With near-infinite unconstrained internal degrees of freedom, their optimization landscapes develop flat vertical gauge valleys, rendering classical generalization metrics vacuous and inducing severe pathologies, specifically generative hallucination and catastrophic forgetting. We introduce the Statistically Meaningful Geometry (SMG) framework, an information-geometric paradigm lifting deterministic parametric models into infinite-dimensional non-parametric Orlicz statistical manifolds. Modeling the total state space as a differential fiber bundle (

\mathcal{M}, \mathcal{B}, π, \mathcal{V}, \mathcal{H}, ω

), we establish a Two-Fold Inference Paradigm. We formalize an Ehresmann connection 1-form

ω

as a dynamic geometric filter that strips away vertical gauge noise (Structural Internal Directions, or SID) and isolates learning trajectories along the strictly non-degenerate horizontal distribution (Statistical Variational Directions, or SVD

χ

). We prove that under connection-filtered pre-training, out-of-distribution predictive variance is strictly upper-bounded by the finite diameter of the identifiable quotient base manifold

\mathcal{B}

, establishing a hard geometric containment of generative hallucinations. By projecting downstream updates onto the orthogonal complement of the historical horizontal carriage, we formalize the SMG Sequential Adaptation Flow, proving the total non-asymptotic elimination of catastrophic forgetting. SMG replaces empirical fine-tuning heuristics with coordinate-free topological constraints, bridging advanced differential geometry with structural reliability in AI.

Bing Cheng, Yi-Shuai Niu, Howell Tong +1

Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · AMSS Center for Forecasting Science, Chinese Academy of Sciences, Beijing, China · Beijing Institute of Mathematical Sciences and Applications (BIMSA), Beijing, China +4

Overparameterization Generative Artificial Intelligence Generative Hallucination Nonlinear Operators

Jul 1, 2026stat.ML

From Spectral Methods to Sample Complexity Bounds for Fourier Neural Operators

We establish approximation and learning guarantees for Fourier neural operators (FNOs) applied to time-

T

solution operators of dissipative evolution equations. The analysis builds on the premise that FNOs can efficiently approximate and learn solution operators whenever these operators admit stable and accurate spectral discretizations. To formalize this idea, we introduce classes of evolution operators defined through spectral methods and derive FNO approximation bounds and polynomial sample complexity guarantees for these classes. For equations with polynomial nonlinearities, the learning rates depend primarily on the smoothness of the input space and the dimension of the physical domain. Our results hold uniformly over broad families of dissipative equations, rather than for a single fixed PDE, and apply in particular to the Navier--Stokes, Allen--Cahn, and Cahn--Hilliard equations. For equations with non-polynomial smooth nonlinearities, we prove that polynomial sample complexity still holds with rates that now additionally depend on the smoothness of the nonlinear terms and the dissipation strength. Overall, we connect classical spectral approximation theory with modern operator learning and explain when FNOs can learn nonlinear evolution operators efficiently.

Nisha Chandramoorthy, Daniel Sanz-Alonso, Nathan Waniorek

University of Chicago

Fourier Neural Operators Nonlinear Operators Spectral Representation Method Neural Operators Neural Approximations Fourier+2

Jun 23, 2026cs.LG

A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients

High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging. Existing deep learning solvers often rely on repeated automatic differentiation to evaluate differential operators, which can cause instability and amplify derivative errors in high dimensions, while probabilistic methods based on stochastic representations require explicit knowledge of the data-generating dynamics and therefore do not apply to black-box environments. We introduce two types of simulators as data-generating mechanisms, and take a ``representing-then-learning" approach that learns the solutions and their derivatives under settings where the underlying PDE operators are accessible only through simulations and pointwise evaluations. Our representation of derivatives relies on the zeroth-order derivative (ZOD) estimators derived from perturbed Monte Carlo trajectories. This fully model-free approach generates targets for the gradient and Hessian networks using only function evaluations. We provide a statistical learning analysis of the proposed approach, including a bias--variance tradeoff for ZODs. Assuming a standard contraction property of the underlying operator, we establish a non-asymptotic error bound that decomposes the total error into discretization error, approximation error, statistical error, and ZOD bias. Crucially, we derive the sample complexity of the learned representations in (weighted) Sobolev space, characterizing the error up to second-order derivatives. Numerical experiments illustrate the competitive performance of the method in moderate and high dimensions.

Yanwei Jia, Du Ouyang, Huyên Pham +1

Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong Shatin, New Territories, Hong Kong SAR · Department of Mathematical Sciences Tsinghua University Beijing, Beijing 100084, China · CMAP Ecole Polytechnique Route de Saclay, Palaiseau 91128, France +1

Partial Differential Equations Nonlinear Operators Implicit Differentiation Neural Solvers Coefficients

Jun 19, 2026math.DG

Physics-Informed Neural Networks for Computing the Morse Index of the Critical Catenoid

The Morse index of a free boundary minimal surface is encoded in its Jacobi-Steklov spectrum, and we test how faithfully a physics-informed neural network (PINN) reproduces that spectrum on a problem whose answer is already known in closed form. The benchmark is the critical catenoid in the unit ball

\mathbb{B}^3

, where it is well known that the Morse index equals

4

and the nullity equals

2

. Separating the angular variable reduces the eigenvalue problem to a family of one-dimensional Robin problems on

[-T,T]

, one for each Fourier mode. A network that enforces the parity of each mode by construction, and carries the eigenvalue as a trainable parameter, returns the three eigenvalues below the stability threshold to within

10^{-6}

to

10^{-4}

of their exact values, with PDE residuals of order

10^{-4}

; assembling them recovers the index

4

and the nullity

2

. We then track the spectrum along a one-parameter homotopy joining a flat reference operator to the catenoid Jacobi operator and identify the crossings at which the index changes. Since the critical catenoid is rigid, a fact we prove, this homotopy deforms operators rather than surfaces. We close by explaining how the same pipeline, with its one-dimensional solver replaced by a two-dimensional one, is poised to address genuinely geometric families in ellipsoidal balls, where the boundary curvature is no longer constant, and the Morse index is not yet known.

Miraj Samarakkody

Parametric Physics-Informed Neural Network Nonlinear Operators Neural Network Spectral Decomposition

Jun 8, 2026eess.SY

Families of Control-Cost-Parametrized Inverse-Optimal Universal Stabilizers

A classical universal stabilization formula offers the practitioner no design freedom: it is a single, parameter-free object. We introduce a cost-parametrized family of stabilizing feedback laws, where (1) the user chooses a function that serves as the running cost on control in an inverse-optimal cost functional, and (2) obtains, through a formula, a nonlinear "expander" of a pre-existing universal controller, which solves an infinite-horizon optimal control problem with a meaningful cost on the state. The cost-to-expander formula is a three-step construction, involving, inter alia, cost differentiation and function inversion-overall, a nonlinear infinite-dimensional operator. The cost-to-expander operator is proven Lipschitz, which enables uniform neural operator approximation of the entire family and supports both offline performance exploration and online adaptation. Semiglobal practical asymptotic stability and second-order suboptimality bounds are established under the approximation. The operator learning and its use in semiglobal stabilization are illustrated numerically. We call the result 'half-direct-optimal' because the paper's design is less than a general 'direct optimal' (HJB-inducing) control, but more than the fully inverse optimal, since the user performs minimization for an arbitrary given cost on control. The dual to the half-direct problem we solve is the problem in which the cost on the state is arbitrary and given. This dual problem is easier and outside of the scope of the paper.

Miroslav Krstic, Luke Bhan

Department of Mechanical and Aerospace Engineering, University of California San Diego, La Jolla, CA 92093-0411, USA · Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093-0411, USA

Optimal Control Lyapunov Function Nonlinear Operators

Jun 4, 2026q-fin.CP

PIVOT: Bridging Black-Scholes Implied-Volatility and Price Objectives via Differentiable Jäckel Operator

Modern option-learning systems operate in two coordinates: price space, where markets quote and no-arbitrage constraints are most naturally enforced, and implied volatility (IV) space, where volatility surfaces are smoothed, regularized, and evaluated. The bottleneck is interface, not approximation: Jäckel's seminal "Let's Be Rational" (LBR) solver already inverts the Black-Scholes price to machine precision efficiently. What is missing is a differentiable layer that preserves LBR in the forward pass and avoids backpropagating through its branch logic. Such a layer must also confront the unavoidable singularity of the inverse map in the low-vega regime, where the sensitivity 1/vega diverges as vega -> 0. We close this gap with PIVOT, the Price-Implied-Volatility Objective Translator. PIVOT keeps the LBR forward pass intact and supplies the backward pass by implicit differentiation through the smooth Black-Scholes/Black-76 price map, with an explicit gating contract: invalid domains return NaN, well-conditioned rows receive the exact 1/vega gradient, and low-vega rows are attenuated rather than silently regularized. On a single H100, a fused Triton kernel reaches 1.79e9 IV/s at machine precision (9.3e-14 max relative error vs. the reference C solver); end-to-end label generation sustains 48.9M/s on synthetic chains and 16.6M/s on SPX OptionMetrics. In a HyperIV-style one-day reproduction on SPX, PIVOT-augmented objectives Pareto-dominate the baselines, reducing held-out price MAE by up to 43.4% and the strongest three-seed gated objective improving price MAE by 38.8% and IV MAE by 21.3% jointly; cross-asset results on RUT, VIX, and NDX show directional price-MAE gains of 40.1%, 24.2%, and 16.7%, while an ungated IV-roundtrip control collapses to a degenerate near-zero surface, confirming the gate as a correctness contract rather than a tuning knob.

Raeid Saqur, Yannick Limmer, Anastasis Kratsios +2

Mathematical Institute, University of Oxford · Vector Institute for AI · DRW +1

Mathematical Finance Volatility Differentiable Optimization Nonlinear Operators

May 31, 2026stat.ML

Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization

Inspired by the function-space theory of neural networks, we formulate and analyze a variation space for nonlinear operators between Hilbert spaces, defined through vector-valued Borel measures of bounded variation. We characterize its unit ball as the closed convex hull of a vector-valued single-neuron dictionary in Bochner spaces. For the ReLU activation, the bounded linear operators in this space are precisely the Schatten-

1

operators, with equivalent norms. For operators in this space, we establish encoder--decoder approximation bounds in the Bochner

L^q

-norm, where the error decomposes into input and output encoding errors and a finite-width term of order

N^{-1/2}

. Under sub-Gaussian assumptions on the input and noise, we further derive high-probability generalization bounds for empirical least squares over path-norm-constrained encoder--decoder networks; the finite-sample contribution to the squared prediction error is of order

K^{-1/2}

up to logarithmic factors. The finite-width and finite-sample constants are independent of the encoding dimensions and bases, with the latter also independent of the network width. When the encoding errors decay algebraically, these bounds yield algebraic approximation and learning rates, in contrast to the complexity barriers for Lipschitz and Fréchet differentiable operator classes.

Jia-Qi Yang, Lei Shi

School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.

Neural Operators Nonlinear Operators Neural Approximations Weight-Space Approximation Algorithms

May 18, 2026cs.LG

Function graph transformers universally approximate operators between function spaces

We study the approximation of nonlinear operators between function spaces by transformers. Our approach is to lift functions to measures supported on their graphs and leverage a recently introduced measure-theoretic view of transformers. A function

h

is represented by its graph measure

γ_h

, with finite tokens

\{(x_j,h(x_j))\}_{j=1}^N

being its empirical approximations. We show that this framework elegantly models discretization refinement via convergence of measures and provides a natural setting for operator learning. Within this framework, we introduce function graph transformers, a graph-preserving subclass of measure-theoretic transformers that maps graph measures to graph measures, which is to say that outputs remain single-valued functions. Crucially, this additional structure does not reduce generality: we prove that the resulting graph-preserving maps can be approximated by finite compositions of standard softmax self-attention layers and pointwise MLPs, yielding universal approximation results for broad classes of nonlinear operators. Unlike existing theoretical approaches to operator learning with transformers, the measure-theoretic framework also accommodates regularized negative-order Sobolev inputs for which discretization invariance is particularly challenging, as well as query points on different output domains. Overall, function graph transformers provide a continuum viewpoint and mathematical toolkit for transformer-based operator learning, clarifying the roles of positional encodings, graph structure, regularization, and ensuring consistency across discretizations.

Takashi Furuya, David Mis, Ivan Dokmanić +2

Doshisha University, RIKEN AIP · Rice University · University of Basel +2

Probabilistic Operator Learning Multiplex Graph Transformers Nonlinear Operators Transformer Architectures Probability Measures

May 14, 2026cs.LG

Universal Approximation of Nonlinear Operators and Their Derivatives

We show that Universal Approximation (UA) of nonlinear operators and their derivatives via Operator Learning (OL) architectures fails in

{C^k_F}

(Fréchet) compact-open topologies and in Fréchet--Sobolev norms (i.e. under operator norms). We solve this obstruction by restoring UA in natural weaker topologies:

C^k_B

(Bastiani) compact-open topologies and (novel) weighted Bastiani--Sobolev spaces for general finite input measures. In full Banach-space generality, these are the first complete generalizations of the corresponding influential classical results in [Hornik, 1991] to infinite-dimensional spaces and OL. Based on our UATs, we formulate Bastiani--Sobolev training in DIOL. These results launch Derivative-Informed Operator Learning (DIOL) (i.e. learning nonlinear operators and their derivatives) on general Banach spaces. We parameterize nonlinear operators via Encoder-Decoder Architectures, classical OL architectures available in general Banach spaces; these include DeepONets, Deep-H-ONets, and PCA-Nets, which our UATs cover. A key mathematical result is that our new weighted Bastiani--Sobolev spaces generalize classical Gaussian (Malliavin) Sobolev spaces on Banach spaces. Open frontiers where DIOL and our UATs find applications are: high-order accuracy in OL; fast constrained optimization in Banach spaces (e.g. optimal control of PDEs, inverse problems) via Learn-Then-Optimize; numerical methods for infinite-dimensional PDEs (e.g. HJB PDEs on Banach spaces from infinite-dimensional optimal control via Optimize-Then-Learn, such as optimal control of PDEs, SPDEs, path-dependent systems, partially observed systems, mean-field control).

Filippo de Feo

Institut für Mathematik, Technische Universität Berlin.

Nonlinear Operators Neural Approximations Implicit Differentiation Approximation Optimal Control

May 13, 2026cs.LG

Unified generalization analysis for physics informed neural networks

Physics-Informed Neural Networks (PINNs) and their variational counterparts (VPINNs) are neural networks that incorporate physical laws, making them useful for scientific problems. Existing generalization analyses for PINNs and VPINNs remain limited, often requiring restrictive assumptions such as stability conditions or linear ellipticity. In this paper, we derive generalization bounds for neural networks that involve differentiation with respect to input variables, covering PINNs and VPINNs under a unified framework. We apply Taylor expansion to represent nonlinear differential operators as linear operators on a high-dimensional space, enabling the use of Koopman-based analysis and showing that high-rank networks can generalize well even in settings involving differential operators. We also show that the nonlinearity of the differential operator exponentially enlarges the bound, highlighting its significant impact on generalization.

Yuka Hashimoto, Tomoharu Iwata

1. NTT, Inc. · 2. RIKEN AIP

Parametric Physics-Informed Neural Network Nonlinear Operators Generalization Bounds Neural Network Variational

May 12, 2026cs.LG

Approximation of Maximally Monotone Operators : A Graph Convergence Perspective

Operator learning has been highly successful for continuous mappings between infinite-dimensional spaces, such as PDE solution operators. However, many operators of interest-including differential operators-are discontinuous or set-valued, and lie outside classical approximation frameworks. We propose a paradigm shift by formulating approximation via graph convergence (Painlevé-Kuratowski convergence), which is well-suited for closed operators. We show that uniform and

L^p

approximation are fundamentally inadequate in this setting. Focusing on maximally monotone operators, we prove that any such operator can be approximated in the sense of local graph convergence by continuous encoder-decoder architectures, and further construct structure-preserving approximations that retain maximal monotonicity via resolvent-based parameterizations.

Takashi Furuya, Yury Korolev, Takaharu Yaguchi

Doshisha University, RIKEN AIP · Department of Mathematical Sciences, University of Bath, UK · Kyushu University, RIKEN AIP

Nonlinear Operators Monotone Variational Inequalities Convergence Approximation Decoder Layers

May 5, 2026cs.LG

Enabling Real-Time Training of a Wildfire-to-Smoke Map with Multilinear Operators

Wildfires are a major producer of fine particulate matter, impacting human health and the electrical grid. Accurately forecasting smoke impacts over long time scales incorporates fuel treatment strategies, natural fuel succession, and stochastic events like lightning strikes. However, predicting smoke for each fuel distribution with a forward simulation of a coupled fire-atmosphere model is computationally infeasible. Moreover, relatively simple fire models are tractable to run in many long-time scenarios but do not capture smoke transport. We use data-driven multilinear operators to predict a smoke concentration field from knowledge of the time since ignition for two quantities of interest: aerosol optical depth and smoke detection. Our method first computes the principal components of time-since-ignition and smoke concentration fields and then learns a map from powers of the input coefficients to the output coefficients. We apply our learned operator to smoke prediction in the Upper Rio Grande Watershed. After collecting training data, learning the approximation weights on a CPU takes less than 30 seconds, and each forward call takes less than 1 ms. On a proxy for aerosol optical depth, we obtain equal accuracy to Monte Carlo sampling with fewer than half as many coupled model calls. For smoke detection, we obtain an intersection-over-union (IoU) of 0.64 and an area under the receiver operating characteristic curve (AUC) of 0.95 on holdout data. Our method is significantly more accurate than the most similar published smoke classifier, which obtains an IoU and AUC of 0.15 and 0.61, respectively, on a 2015 bushfire in Australia.

Zachary Morrow, Joseph Crockett, John D. Jakeman +1

Sandia National Laboratories 1515 Eubank Blvd SE, Albuquerque, NM 87123

Wildfire Prediction Atmospheric Dynamics Combustion Control Spatiotemporal Prediction Nonlinear Operators

May 4, 2026cs.LG

Quantitative Sobolev Approximation Bounds for Neural Operators with Empirical Validation on Burgers Equation

Neural operators have emerged as a powerful tool for learning mappings between infinite-dimensional function spaces. However, their approximation properties in Sobolev norms remain poorly quantified, even though these norms control both function values and derivatives and are the natural metrics for PDE well-posedness, stability, and generalization. We develop a functional-analytic framework for operator learning in Sobolev spaces and connect it to the numerical behavior of Fourier Neural Operators (FNOs) on a prototypical PDE. First, for a continuous nonlinear operator

\mathcal{G}: H^{s}(D)\to H^{t}(D')

with

s > d/2

and inputs restricted to a compact subset of

H^{s}(D)

, we prove that

\mathcal{G}

can be uniformly approximated in

H^{t}

-norm by a neural operator with

\mathcal{O}(\varepsilon^{-d/s})

trainable parameters. This yields an explicit complexity--error relation of the form

\|\mathcal{G}-\mathcal{G}_θ\|_{H^{t}} \lesssim C N^{-s/d}

. We then study the one-dimensional viscous Burgers solution operator

\mathcal{G}: u_{0}\mapsto u(\cdot,1)

on a bounded

H^{1}

-ball and train FNOs with an

H^{1}

-loss. Across a sweep of model sizes, we obtain test

H^{1}

-errors down to

\mathcal{O}(10^{-7})

and relative errors of order

10^{-3}

, with predictions accurately matching both solutions and spatial derivatives on held-out data. A log-log plot of Sobolev error versus parameter count exhibits an approximate power law

\|\mathcal{G}-\mathcal{G}_θ\|_{H^{1}} \approx C N^{-α}

with empirical exponent

α\approx 1.4

, and long-horizon training reveals optimization instabilities in large FNOs, providing quantitative evidence that Sobolev-space approximation theory meaningfully predicts neural-operator scaling behavior.

Nicole Hao

Department of Mathematics, Cornell University

Fourier Neural Operators Logarithmic Sobolev Inequalities Nonlinear Operators Neural Operators Neural Approximations Loss Function+1

May 2, 2026cs.CV

SRGAN-CKAN: Expressive Super-Resolution with Nonlinear Functional Operators under Minimal Resources

Single-Image Super-Resolution (SISR) aims to reconstruct a High-Resolution (HR) image from a Low-Resolution (LR) observation, a fundamentally ill-posed problem where high-frequency details are severely degraded at large upscaling factors. Recent advances have been driven by transformer-based architectures and diffusion models improve global context modeling and perceptual quality at the cost of increased computational complexity. In contrast, this work focuses on enhancing the expressivity of local operators under minimal resources. We propose SRGAN--CKAN, a hybrid super-resolution framework that integrates Convolutional Kolmogorov--Arnold Networks (CKAN) into an adversarial learning setting reformulating convolution as a nonlinear patch-based transformation. The proposed operator replaces linear local mappings with spline-based functional representations, allowing expressive modeling of complex local structures and high-frequency textures using minimal hardware resources. Experimental results demonstrate that the proposed approach improves perceptual quality while preserving reconstruction fidelity, achieving a favorable balance between distortion-based and perceptual metrics. These results are obtained under constrained computational settings, highlighting the efficiency of the proposed formulation. Overall, this work introduces a complementary direction to existing approaches by improving the representational power of local transformations, providing an efficient and scalable alternative to globally intensive architectures.

Roberto Isai Navaro-Aviña, Eduardo Said Merin-Martinez, Andres Mendez-Vazquez +1

Cinvestav, Unidad Guadalajara Av. del Bosque 1145, El Bajio Zapopan, Jalisco, 45017, Mexico · Cinvestav, Unidad Tamaulipas Km. 5.5 Carretera Victoria - Soto La Marina Victoria, 87139, Tamaulipas, Mexico

Generative Super-Resolution Perceptual Quality Nonlinear Operators Kolmogorov-Arnold Networks Less Data

Sep 14, 2025stat.ML

A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning

We develop a stochastic approximation framework for learning nonlinear operators between infinite-dimensional spaces utilizing general Mercer operator-valued kernels. Our framework encompasses two key classes: (i) operator-valued kernels whose associated integral operators are compact and hence admit discrete spectral decompositions, and (ii) separable kernels of the form

K(x,x')=k(x,x')T

, where

k

is a scalar-valued kernel and

T

is a positive operator on the output space. This broad setting induces expressive vector-valued reproducing kernel Hilbert spaces (RKHSs) that generalize the classical

K=kI

paradigm, thereby enabling rich structural modeling with rigorous theoretical guarantees. To address target operators lying outside the RKHS, we introduce vector-valued interpolation spaces to precisely quantify misspecification error. Within this framework, we establish non-asymptotic convergence rates for prediction, estimation, and misspecification errors in the online and finite-horizon settings. Importantly, the framework also accommodates a range of operator learning settings, from Fredholm integral operators to encoder--decoder architectures. Numerical experiments on the two-dimensional Navier--Stokes equations illustrate the proposed approach.

Jia-Qi Yang, Lei Shi

School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.

Probabilistic Operator Learning Nonlinear Operators Reproducing Kernel Hilbert Spaces Kernel Hilbert Spaces Kernel Method Stochastic Approximation+1

Jan 14, 2025cs.AI

A Neural Operator-Based Approach to Symbolic Discovery of PDEs

Discovering governing equations from data remains challenging when the underlying dynamics involve nonlocal differential operators, field interactions governed by auxiliary equations, or temporal memory effects. We propose Neural Operator-based symbolic Model approximaTion and discOvery (NOMTO), a framework that extends Equation Learner-type symbolic architectures by incorporating pretrained neural operators as nodes in the symbolic network. NOMTO represents candidate equations as sparse differentiable computational graphs that combine algebraic operations with fixed neural operator surrogates pretrained to approximate nonlinear operators. We evaluate the method on model-discovery problems involving nonlocal spatial operators, couplings mediated by auxiliary field equations, and temporal integral terms representing memory effects. The results show that NOMTO can recover compact governing equations containing nonlocal operator terms, thereby extending symbolic model discovery beyond libraries restricted to local derivatives and point-wise algebraic combinations.

Sergei Garmaev, Olga Fink

Intelligent Maintenance and Operations Systems Laboratory, EPFL, Lausanne, Switzerland

Neural Operators Nonlinear Operators Partial Differential Equations Discovery