Abstract
Despite the many variants of DeepONet that have been proposed, query-based operator networks still struggle with shock-dominated and low-viscosity PDEs, whose sharp moving discontinuities and slowly decaying solution spectra challenge finite-dimensional separable representations. In this work, we propose \emph{Feature Interaction Modeling Operator} (FM-Operator), a point-wise query neural operator that explicitly models feature construction and interactions between sensor observations and query coordinates. Our design is motivated by a reinterpretation of the canonical DeepONet aggregation through the lens of multiplicative interactions. Specifically, the branch--trunk inner product admits the equivalent form b(u)⊤τ(y)=1⊤diag(b(u))τ(y), revealing that the two representations interact only along corresponding latent dimensions and therefore constitute a diagonally constrained multiplicative interaction. This observation suggests that, beyond improving the individual branch and trunk networks, the structure through which function and query representations interact is itself an important inductive bias in point-wise operator learning. FM-Operator accordingly redesigns both feature construction and feature interaction, enabling structured information exchange beyond the conventional branch--trunk coupling while retaining point-wise query evaluation. Experiments across multiple PDE benchmarks demonstrate that FM-Operator consistently outperforms vanilla DeepONet and achieves clear improvements over the strong Shift-DeepONet baseline. These results suggest that explicitly designing representation construction and interaction provides a promising direction for improving the effectiveness of DeepONet-style query-based neural operators.
Explore similar work
Jun 4, 2026cs.LG
Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data. By incorporating physical constraints into the training objective, PINOs combine the cross-instance generalization of neural operators with the data efficiency of physics-informed learning. Despite this promise, how to train PINOs efficiently and robustly remains less well-understood than the training of either data-driven neural operators or physics-informed neural networks (PINNs). To bridge this gap, we examine key components of the PINO training pipeline, including architecture design, optimizer choice, loss balancing, and collocation-point sampling strategy. We study three representative operator backbones, Deep Operator Network (DeepONet), Fourier Neural Operator (FNO), and Continuous Vision Transformer (CViT), across five diverse parametric PDE systems. Our results show that CViT provides consistently strong and stable performance across the considered benchmarks. Beyond architecture, we find that several optimization pathologies previously identified in PINN training naturally arise in PINOs, including gradient conflicts and causal violation. We also find that mitigation algorithms developed for PINNs remain effective in the PINO setting. We further compare physics-informed and data-driven training under different data regimes, revealing that a carefully designed physics-informed training pipeline can match, and in some cases, outperform purely data-driven neural operators. Taken together, these findings provide a systematic empirical understanding of the optimization challenges in PINO training and inform a practical pipeline for efficient and robust physics-informed operator learning. Code and data are available at https://github.com/NanxiiChen/PI-CViT.
Nanxi Chen, Chuanjie Cui, Airong Chen +2
Jul 26, 2026cs.LG
Parameterized and coupled partial differential equations (PDEs) are central to modeling phenomena in science and engineering, yet neural operator methods that address both aspects remain limited. We extend Fourier neural operators (FNOs) with minimal architectural modifications along two directions. For parameterized dynamics, we propose a hypernetwork-based modulation that conditions the operator on physical parameters. For coupled systems, we conduct a systematic exploration of architectural choices, examining how operator components can be adapted to balance shared structure with cross-variable interactions while retaining the efficiency of standard FNOs. Evaluations on benchmark PDEs, including the one-dimensional capacitively coupled plasma equations and the Gray-Scott system, show that our methods achieve up to 55-72% lower errors than strong baselines, demonstrating the effectiveness of principled modulation and systematic design exploration.
Cheng Jing, Uvini Balasuriya Mudiyanselage, Abhishek Verma +3
Aug 6, 2026cs.LG
Physics-informed neural networks (PINNs) often fail to accurately resolve partial differential equations (PDEs) with high-frequency or multi-scale solutions, as well as strongly nonlinear problems. Two factors underlie this difficulty: spectral bias, the tendency of neural networks to underfit high-frequency features; and representation-coefficient coupling, the entanglement of representation learning and coefficient fitting within a single nonconvex optimization objective. In this work, we propose the Fourier-enhanced alternating Levenberg--Marquardt PINN (FALM-PINN), an optimization framework that decouples representation learning from coefficient fitting. The upper-level problem learns a Fourier-enhanced basis that enriches the latent space with high-frequency components, while the lower-level problem resolves the coupling by fitting the projection coefficients on this basis, solving a nonlinear least-squares problem with the Levenberg--Marquardt algorithm. The framework applies to general nonlinear and coupled PDE systems, and reduces to a single-step convex optimization problem for linear PDEs. We prove global convergence of the alternating training scheme in both cases. Numerical examples on multiple challenging high-frequency and nonlinear PDEs show that FALM-PINN achieves relative
L2 errors up to two orders of magnitude lower than state-of-the-art baselines.
Yulun Wu, Matthieu Barreau, Miguel Aguiar +1