Physics-Informed Neural Networks for Solving Derivative-Constrained PDEs

Kentaro Hoshisashi; Carolyn E Phelan; Paolo Barucca

cs.LGApr 15, 2026

Physics-Informed Neural Networks for Solving Derivative-Constrained PDEs

Authors: Kentaro Hoshisashi, Carolyn E Phelan, Paolo Barucca

Organizations: Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK

Abstract

Physics-Informed Neural Networks (PINNs) recast PDE solving as an optimisation problem in function space by minimising a residual-based objective, yet many applications require additional derivative-based relations that are just as fundamental as the governing equations. In this paper, we present Derivative-Constrained PINNs (DC-PINNs), a general framework that treats constrained PDE solving as an optimisation guided by a minimum objective function criterion where the physics resides in the minimum principle. DC-PINNs embed general nonlinear constraints on states and derivatives, e.g., bounds, monotonicity, convexity, incompressibility, computed efficiently via automatic differentiation, and they employ self-adaptive loss balancing to tune the influence of each objective, reducing reliance on manual hyperparameters and problem-specific architectures. DC-PINNs consistently reduce constraint violations and improve physical fidelity versus baseline PINN variants, representative hard-constraint formulations on benchmarks, including heat diffusion with bounds, financial volatilities with arbitrage-free, and fluid flow with vortices shed. Explicitly encoding derivative constraints stabilises training and steers optimisation toward physically admissible minima even when the PDE residual alone is small, providing reliable solutions of constrained PDEs grounded in energy minimum principles.

Figures & tables

Figure 1: Predicted solutions for the one-dimensional heat equation. Left: ground-truth magnitude. Right: model–ground-truth difference with a diverging scale.

	Conditions			Accuracy
Model	Ineq.	Hard- $b$	AL	$\mathrm{E}_{u}$	$\mathrm{E}_{f}$	$\mathrm{E}_{b}$	$\mathrm{E}_{h_{xx}}$	$\mathrm{E}_{h_{t}}$
PINNs	–	–	–	0.013	0.032	0.027	0.677	0.068
PINNs+Ineq.	✓	–	–	0.010	0.035	0.010	0.522	0.037
hPINN (pen.)	✓	✓	–	0.144	0.010	0.000	1.433	0.142
hPINN (AL)	✓	✓	✓	0.131	0.010	0.000	1.310	0.129
AL-PINNs	✓	–	✓	0.463	0.008	0.004	4.572	0.454

Table 1: Validation (RMSE: bold is best) for the heat equation.

Figure 2: Second-derivative profiles with respect to $x$ at multiple time snapshots. Dashed: analytical solution.

	Conditions			Accuracy
Model	Ineq.	$\lambda$	$m$	$E_{0}$	$E_{f}$	$E_{b}$	$E_{h_{xx}}$	$E_{h_{t}}$
PINNs	–	–	–	0.013	0.032	0.027	0.678	0.068
PINNs-Ineq (fixed)	✓	–	–	0.010	0.036	0.010	0.524	0.037
DC-PINNs (no $\lambda$ )	✓	–	✓	0.012	0.056	0.020	0.501	0.050
DC-PINNs (no $m$ )	✓	✓	–	0.004	0.008	0.006	0.160	0.016
DC-PINNs (full)	✓	✓	✓	0.002	0.006	0.003	0.091	0.009

Table 2: Ablation on validation (RMSE: bold is best) and adaptive balancing in the heat equation.

\lambda

: component-wise;

m

: sample-wise.

Figure 3: Predicted solutions for the volatility-surface calibration. Left: ground-truth magnitude. Right: model–ground-truth difference with a diverging scale.

	Conditions			Accuracy
Model	Ineq.	Hard- $b$	AL	$E_{0}$	$E_{b}$	$E_{f}$	$E_{h_{x}}$	$E_{h_{xx}}$	$E_{h_{t}}$
PINNs	–	–	–	0.002	0.002	0.002	0.154	0.065	0.000
PINNs+Ineq. (fixed)	✓	–	–	0.002	0.003	0.004	0.098	0.019	0.029
hPINN (pen.)	✓	✓	–	0.477	0.007	0.878	0.533	0.423	0.000
hPINN (AL)	✓	✓	✓	0.477	0.007	0.878	0.533	0.419	0.000
AL-PINNs	✓	–	✓	0.002	0.003	0.002	0.057	0.005	0.032

Table 3: Validation (RMSE: bold is best) for the volatility surface calibration.

Figure 4: Spatial derivative $\partial u/\partial x$ at multiple time snapshots. Dashed: numerical reference.

Figure 5: Inequality-violation maps for learned local volatilities; dashed lines indicate exact references.

	Conditions		Accuracy					$\mu$
Model	Ineq.	AL	$E_{0_{u}}$	$E_{0_{v}}$	$E_{f_{u}}$	$E_{f_{v}}$	$E_{h_{\nabla p}}$	$\mu_{1}$	$\mu_{2}$
PINNs	–	–	0.395	0.395	0.018	0.019	0.159	0.977	0.010
PINNs+Ineq. (fixed)	✓	–	0.395	0.391	0.020	0.019	0.154	0.977	0.011
hPINN (pen.)	✓	–	0.350	0.304	0.004	0.004	0.165	0.758	0.020
AL-PINNs	✓	✓	0.343	0.350	0.004	0.005	0.142	0.875	0.012
DC-PINNs	✓	–	0.395	0.391	0.018	0.016	0.153	0.977	0.010

Table 4: Validation (RMSE: bold is best) for Navier–Stokes equations.

Figure 6: Predicted pressure $p(x,y,t)$ (mean-adjusted) at $t=10$ for the Navier–Stokes case. Left: ground-truth magnitude. Right: model–ground-truth difference with a diverging scale.

Method	Heat Problem	Volatlity Surface	Navier –Stokes
PINNs	2.16	3.93	58.52
PINNs+Ineq	2.19	4.05	82.14
hPINNs (pen.)	2.82	3.85	42.55
hPINNs (AL)	2.88	4.36	52.68
AL-PINNs	2.02	3.51	—
DC-PINNs	4.41	6.17	97.10

Table 5: Wall-clock training time (s) across benchmark problems.

Figure 7: Percentage improvement with respect to PINNs on a symmetric logarithmic scale. Positive values indicate improvement relative to the baseline.

Figure 8: Aggregate ranks across all metrics. Left Heat-PDE, centre Volatility Surface PDE, right Navier–Stokes PDE. Lower ranks indicate stronger overall performance. DC-PINNs rank highly on violation- and stability-related metrics, with overall rank depending on how accuracy and feasibility are weighted.

$d$	Fixed	Root	Linear	Power‑1.5
1	1.86	1.78	1.73	1.88
2	1.92	2.06	2.17	2.44
8	2.52	3.76	5.28	5.63
32	5.76	6.78	9.58	30.72
128	8.64	6.81	45.01	-- 1 1 1 out‑of‑GPU-memory.

Table 6: Wall-clock training time (s) for the

d

-dimensional heat equation with collocation size

N_{\chi}

: Fixed (

N_{\chi}=100

), Root (

N_{\chi}=\lfloor 100\sqrt{d}\rfloor

), Linear (

N_{\chi}=100d

), Power-1.5 (

N_{\chi}=100d^{1.5}

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

\mathcal{L}\mathrel{\mathop{\mathchar 58\relax}}=\sum_{\chi}\lambda_{\chi}\,\hat{\mathcal{L}}_{\chi}\!\left(m_{\chi},x_{\chi}\right),

Appendix

Algorithm 1 DC-PINNs with Balancing Processes

$\eta_{m}$	$p_{m}\backslash p_{\lambda}$	10	100	1000
$10^{-4}$	10	(-1.47,-2.78,-0.77,0.38)	(-1.52,-2.92,-0.83,0.30)	(-1.51,-2.89,-0.82,0.31)
	100	(-1.40,-2.51,-0.69,0.52)	(-1.41,-2.52,-0.70,0.50)	(-1.40,-2.53,-0.69,0.51)
	1000	(-1.33,-3.00,-0.63,0.63)	(-1.32,-2.99,-0.63,0.63)	(-1.32,-2.99,-0.62,0.63)
$10^{-3}$	10	(-4.98,-4.11,-4.50,-4.54)	(-5.20,-4.25,-4.61,-4.97)	(-5.28,-4.20,-4.69,-5.10)
	100	(-5.12,-4.31,-4.55,-4.93)	(-5.00,-4.40,-4.47,-4.77)	(-5.23,-4.43,-4.71,-5.15)
	1000	(-4.99,-4.15,-4.44,-4.57)	(-4.84,-4.15,-4.29,-4.33)	(-5.20,-4.36,-4.62,-5.06)

Appendix

Table 7: Sensitivity of DC-PINNs on the Heat problem. Reported values are

\mathrm{RMSE}

\log

-scale, given as

(\mathcal{L}_{0},\mathcal{L}_{f},\mathcal{L}_{b},\mathcal{L}_{(h1+h2)})

in dense evaluation grids. Bold entries indicate the overall top-4 choices (by the sum of the four metrics; more negative is better).

$\nu$	$p_{m}\backslash p\lambda$	10	100	1000
$10^{-4}$	10	(-4.88,-5.30,-4.16,-3.82)	(-4.86,-5.34,-4.26,-3.97)	(-4.98,-5.34,-4.33,-4.02)
	100	(-4.97,-5.03,-4.66,-4.40)	(-5.02,-4.99,-4.69,-4.41)	(-5.00,-5.05,-4.73,-4.50)
	1000	(-4.89,-4.90,-4.79,-4.50)	(-4.96,-4.94,-4.80,-4.52)	(-4.99,-4.95,-4.87,-4.51)
$10^{-3}$	10	(-6.76,-6.24,-6.07,-5.65)	(-6.82,-6.40,-5.69,-5.93)	(-6.87,-6.35,-5.68,-5.64)
	100	(-6.13,-5.47,-6.13,-5.05)	(-6.57,-6.18,-6.16,-5.86)	(-6.34,-6.00,-6.17,-5.93)
	1000	(-6.51,-6.11,-6.11,-5.71)	(-6.59,-6.20,-6.24,-5.67)	(-6.55,-6.10,-6.25,-5.74)

Appendix

Table 8: Sensitivity of DC-PINNs on the volatility surface calibration. The reported values are

\mathrm{RMSE}

on the logarithmic scale, given as

(\mathcal{L}0,\mathcal{L}b,\mathcal{L}f,\mathcal{L}{h{(x+xx+t)}})

on dense evaluation grids. The bold entries indicate the overall top-4 configurations, and the lower values correspond to better performance.

$\nu$	$p_{m}\backslash p_{\lambda}$	10	100	1000
$10^{-4}$	10	(-1.00,-3.67,-1.31; 0.0483/-0.0020)	(-1.00,-3.42,-1.31; 0.0442/-0.0046)	(-1.00,-2.65,-1.31; 0.0461/-0.0084)
	100	(-1.01,-3.97,-1.31; 0.0432/0.0002)	(-1.01,-3.96,-1.31; 0.0405/0.0004)	(-1.01,-3.83,-1.31; 0.0430/-0.0008)
	1000	(-1.01,-4.28,-1.32; 0.0430/0.0012)	(-1.01,-4.27,-1.32; 0.0405/0.0013)	(-1.01,-4.12,-1.32; 0.0435/0.0010)
$10^{-3}$	10	(-0.94,-2.73,-1.12; 0.7695/0.0134)	(-0.94,-2.61,-1.09; 0.7109/0.0073)	(-0.94,-2.63,-1.09; 0.6992/0.0070)
	100	(-0.94,-2.79,-1.14; 0.7852/0.0160)	(-0.95,-2.78,-1.11; 0.7383/0.0095)	(-0.95,-2.98,-1.11; 0.7812/0.0117)
	1000	(-0.96,-2.99,-1.15; 0.8008/0.0165)	(-0.95,-2.90,-1.12; 0.7578/0.0118)	(-0.96,-2.92,-1.12; 0.7500/0.0099)

Appendix

Table 9: Sensitivity of DC-PINNs on the Navier–Stokes equation. Reported values are

\mathrm{RMSE}

in logarithmic scale, evaluated on dense grids and expressed as

(\mathcal{L}_{0},\mathcal{L}_{f},\mathcal{L}_{h},\mu_{1}/\mu_{2})

, where

\mathcal{L}_{0}=\max\{\mathcal{L}_{0_{u}},\mathcal{L}_{0_{v}},\mathcal{L}_{0_{p}}\}

\mathcal{L}_{f}=\max\{\mathcal{L}_{f_{u}},\mathcal{L}_{f_{v}}\}

, and

\mathcal{L}_{h}=\mathcal{L}_{h_{\nabla p}}

. Bold entries indicate the overall top-4 configurations, with lower values corresponding to better accuracy.

Figure 9: Flow past a circular cylinder at $Re=100$ . Shown are the spatio–temporal fields $u(x,y,t)$ , $p(x,y,t)$ (average adjusted), and $v_{y}(x,y,t)$ . The top row provides the FEM reference and the bottom row shows DC-PINNs. The slabs indicate several time instants along the $t$ -axis.

Figure 10: Evolution of trained coefficients ( $\mu_{1/2}$ ) by epochs.

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 269 papers

AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training
May 8, 2026Binghang Lu, Runyu Zhang, Changhong Mou +2Neural PDE Solvers Constrained Optimization
Purdue University · Massachusetts Institute of Technology · Utah State University +1
Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond
Aug 11, 2026Maciej J. Mikulski, Tadeusz UhlAutomatic Differentiation Neural PDE Solvers
AGH University of Krakow
Learning Physics from an Imperfect Ancestor
Sep 21, 2026S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas +1PDE Surrogate Modeling OOD Generalization
Sibley School of Mechanical and Aerospace Engineering, Cornell University · AI Lab, Siemens Energy · Department of Civil and Systems Engineering, Johns Hopkins University

May 8, 2026J/K move · Enter open · S save

AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training

Binghang Lu, Runyu Zhang, Changhong Mou, Na Li+1

Purdue University · Massachusetts Institute of Technology · Utah State University+1

Neural PDE Solvers Constrained Optimization Physics-Informed ML Adaptive Gradient Methods Feedback Linearization Optimization

Open paper AskPDF arXiv

Physics-informed neural networks (PINNs) provide a flexible framework for solving forward and inverse problems governed by partial differential equations (PDEs), but standard PINN training typically relies on soft penalty formulations that combine PDE residuals, data mismatch, and initial/boundary conditions using manually chosen weights. This often leads to ill-conditioning, sensitivity to loss weights, and poor constraint satisfaction. In this work, we reformulate PINN training as an equality-constrained optimization problem and propose a novel Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN (AdamFLIP). The key idea is to view the constraint residuals as the output of a controlled dynamical system and to compute the Lagrange multiplier as a feedback input that locally drives these residuals toward stable linear contraction dynamics. AdamFLIP then applies Adam-style first- and second-moment adaptation to the resulting feedback-linearized Lagrangian gradient, combining principled constraint handling with the scalability and robustness of adaptive neural-network optimization. We test AdamFLIP on a range of benchmark forward and inverse PDE problem, and it consistently outperforms both the standard soft-constrained PINN and state-of-the-art constrained optimizers. Specifically, on the Navier--Stokes equations benchmark, AdamFLIP \textbf{reduces relative

L_2

error by more than two thirds} for the predicted solution compared to the next best method. Our AdamFLIP framework provides an effective and computationally scalable hard constraint optimization method for PINN training.

Abstract

Neural PDE Solvers

Explore similar work

AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training

Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond

Learning Physics from an Imperfect Ancestor