Manifold-Orthogonal Dual-spectrum Extrapolation for Parameterized Physics-Informed Neural Networks

Zhangyong Liang; Huanhuan Gao

cs.LGMar 14, 2026

Manifold-Orthogonal Dual-spectrum Extrapolation for Parameterized Physics-Informed Neural Networks

Authors: Zhangyong Liang, Huanhuan Gao

Abstract

Physics-informed neural networks (PINNs) have achieved notable success in modeling dynamical systems governed by partial differential equations (PDEs). To avoid computationally expensive retraining under new physical conditions, parameterized PINNs (P $^2$ INNs) commonly adapt pre-trained operators using singular value decomposition (SVD) for out-of-distribution (OOD) regimes. However, SVD-based fine-tuning often suffers from rigid subspace locking and truncation of important high-frequency spectral modes, limiting its ability to capture complex physical transitions. While parameter-efficient fine-tuning (PEFT) methods appear to be promising alternatives, applying conventional adapters such as LoRA to P $^2$ INNs introduces a severe Pareto trade-off, as additive updates increase parameter overhead and disrupt the structured physical manifolds inherent in operator representations. To address these limitations, we propose Manifold-Orthogonal Dual-spectrum Extrapolation (MODE), a lightweight micro-architecture designed for physics operator adaptation. MODE decomposes physical evolution into complementary mechanisms including principal-spectrum dense mixing that enables cross-modal energy transfer within frozen orthogonal bases, residual-spectrum awakening that activates high-frequency spectral components through a single trainable scalar, and affine Galilean unlocking that explicitly isolates spatial translation dynamics. Experiments on challenging PDE benchmarks including the 1D Convection--Diffusion--Reaction equation and the 2D Helmholtz equation demonstrate that MODE achieves strong out-of-distribution generalization while preserving the minimal parameter complexity of native SVD and outperforming existing PEFT-based baselines.

Figures & tables

Figure 1: Comparison of different fine-tuning approaches for P 2 INNs. (a) Native Singular Value Decomposition (SVD), (b) Weight-Decomposed Adaptation (DoRA), and (c) our proposed Manifold-Orthogonal Dual-spectrum Extrapolation (MODE). ( ◼ : frozen parameters; ◼ : tunable parameters; ◼ : zeros.)

Figure 2: Structural limitations of SVD-based fine-tuning in P 2 INNs. (a) Degrees of freedom comparison between SVD and LoRA. (b) Approximation error of SVD versus optimal rank-

r

projection. (c) PDE scenario comparing fixed-basis and free-basis projections.

Figure 3: Subspace rotation failure of P 2 INN-SVD when generalizing to the OOD convection equation $u_{t}+30\,u_{x}=0$ .

Figure 4: P 2 INNs with MODE modulation. Each intermediate layer of the manifold decoder is replaced by a dual-path MODE-adapted block: (i) the Frozen Full-Rank Path scales the frozen weight output $\mathbf{W}_{0,l}\mathbf{h}^{(l-1)}$ by the trainable scalar $\tau_{l}$ ; (ii) the Ultra-Low-Rank Orthogonal Path mixes cross-modal energy via the dense core $\mathbf{\Phi}_{l}$ within the compact $k$ -dimensional subspace. Only $\{\mathbf{\Phi}_{l},\tau_{l},\Delta\mathbf{b}_{l}\}$ are trainable; all other components are strictly frozen.

Figure 5: Pre-training loss curves on the 1D CDR equations for 10 different parameter values. (a) Convection-Diffusion Equations; (b) Diffusion-Reaction Equations; (c) Convection-Diffusion-Reaction Equations.

Model	Param. (K)	Train Loss	Test Loss	Efficiency (1/L/kP)
SVD	0.6	19.41	19.51	0.09
(IA) 3	0.6	14.92	17.04	0.11
LoRA	2.3	19.82	19.81	0.02
LoRA-FA	1.3	19.30	19.23	0.04
DoRA	2.5	19.40	19.90	0.02
EDoRA	0.6	20.26	20.40	0.08

Table 1: Performance of fine-tuning methods on CDR equation (rank=4). Trainable parameters (K), train loss, test loss, and parameter efficiency. Best results in bold, second best underlined and italic.

Method	r=1		r=2		r=4		r=8
	Param.(K)	Ratio(%)	Param.(K)	Ratio(%)	Param.(K)	Ratio(%)	Param.(K)	Ratio(%)
Full	76.9	100.00	76.9	100.00	76.9	100.00	76.9	100.00
SVD	0.6	0.61	0.6	0.61	0.6	0.61	0.6	0.61
(IA) 3	0.6	0.71	0.6	0.71	0.6	0.71	0.6	0.71
LoRA	0.8	1.03	1.3	1.67	2.3	2.92	4.3	5.32
LoRA-FA	0.6	0.71	0.8	1.03	1.3	1.65	2.3	2.84

Table 2: Trainable parameter counts across different ranks for CDR equation. Comparison of fine-tuning methods at ranks

r\in\{1,2,4,8\}

. Param. denotes trainable parameters (K), Ratio denotes the percentage of trainable parameters.

Setting	Model	$E_{\ell_{2}}$ (%)	$E_{\ell_{\infty}}$ (%)	Param.(K)	Efficiency (1/L/kP)
$\beta{=}1,\nu{=}1,\rho{=}1$	SVD	2033.965 $\pm$ 80.091	2911.166 $\pm$ 136.106	0.6	0.082
	(IA) 3	1867.738 $\pm$ 58.840	2871.680 $\pm$ 102.998	0.6	0.089
	LoRA	2093.220 $\pm$ 99.989	2765.794 $\pm$ 62.545	2.3	0.021
	LoRA-FA	1926.632 $\pm$ 86.657	2954.389 $\pm$ 136.198	1.3	0.040
	DoRA	2184.316 $\pm$ 96.055	3142.016 $\pm$ 136.413	2.5	0.018
	EDoRA	2063.952 $\pm$ 80.902	2771.900 $\pm$ 133.994	0.6	0.081

Table 3: Fine-tuning methods comparison on CDR equation. Relative

\ell_{2}

errors (

E_{\ell_{2}}

), relative

\ell_{\infty}

errors (

E_{\ell_{\infty}}

), trainable parameter counts and efficiency for various fine-tuning algorithms applied to P 2 INN. The best results are marked in bold, and the second best results are underlined.

PDE Operator Learning

Partial differential equation (PDE) operator learning builds models that map inputs such as coefficients and initial conditions to solution fields.

See all 228 papers

PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks
Sep 26, 2026Xu Yang, Mingyang Yu, Jun Zhang +2Neural PDE Solvers Physics-Informed ML
East China Normal University · Nankai University · Hanyang University
On the training of physics-informed neural operators for solving parametric partial differential equations
Jun 4, 2026Nanxi Chen, Chuanjie Cui, Airong Chen +2Neural Network Optimization Gradient Interference
College of Civil Engineering, Tongji University, Shanghai 200092 · Department of Engineering Science, University of Oxford, Oxford OX1 3PJ · Institute for Foundations of Data Science, Yale University, New Haven, CT 06520
Modularity-Free Conflict-Averse Training for Generalized PINNs
Jun 18, 2026Heejo Kong, Beomchul Park, Sung-Jin Kim +1Neural PDE Solvers Gradient Interference
Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea · Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea

Sep 26, 2026J/K move · Enter open · S save

PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks

Xu Yang, Mingyang Yu, Jun Zhang, Keqian Li+1

East China Normal University · Nankai University · Hanyang University

Neural PDE Solvers Physics-Informed ML Large Language Model-Guided Optimization

Open paper AskPDF arXiv

Physics-informed neural networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs), yet their training behavior can change substantially throughout optimization. Residual distributions, gradient interactions, regional learning difficulty, and model-capacity requirements may evolve over time, while the network architecture and major training mechanisms are typically determined before training. We propose PINNMorph, an online PINN adaptation framework based on large language model (LLM)-guided policy evolution. PINNMorph maintains a population of state-conditioned adaptation policies that map execution diagnostics to controlled interventions over topology modification, additive representation augmentation, objective balancing, gradient handling, adaptive sampling, and optimizer-phase control. At each intervention opportunity, candidate programs are instantiated from the current policy population, selected according to the observed training state, and applied directly to the PINN under training. The resulting model inherits its existing parameters and training state and continues optimization along the same trajectory. Execution outcomes are subsequently used to evaluate interventions and evolve the policy population. Unlike pre-training architecture search or fixed adaptation rules, PINNMorph jointly adapts the current PINN and the policies governing its interventions using feedback from actual training. Experiments on 13 PDE benchmarks show that PINNMorph achieves lower solution errors than SA-PINN, ConFIG, RoPINN, HARMONIC, and PINNsAgent across all evaluated problems. Ablation studies further examine the effects of online adaptation, state-conditioned intervention selection, and execution-feedback-driven policy evolution.

Abstract

PDE Operator Learning

Explore similar work

PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks

On the training of physics-informed neural operators for solving parametric partial differential equations

Modularity-Free Conflict-Averse Training for Generalized PINNs