Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs

Stavros Kassinos

cs.LGApr 15, 2026

Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs

Authors: Stavros Kassinos

Organizations: Computational Sciences Laboratory, Department of Mechanical Engineering, University of Cyprus, 1 University Avenue, Aglantzia, 2109, Nicosia, Cyprus

Abstract

Physics-informed neural networks (PINNs) are often selected by a single scalar loss even when the quantity of interest is more specific. We study a hybrid design in which the governing PDE residual remains automatic-differentiation (AD) based, while finite differences (FD) appear only in a weak auxiliary term that penalizes gradients of the sampled residual field. The FD term regularizes the residual field without replacing the PDE residual itself. We examine this idea in two stages. Stage 1 is a controlled Poisson benchmark comparing a baseline PINN, the FD residual-gradient regularizer, and a matched AD residual-gradient baseline. Stage 2 transfers the same logic to a three-dimensional annular heat-conduction benchmark (PINN3D), where baseline errors concentrate near a wavy outer wall and the auxiliary grid is implemented as a body-fitted shell adjacent to the wall. In Stage 1, the FD regularizer reproduces the main effect of residual-gradient control while exposing a trade-off between field accuracy and residual cleanliness. In Stage 2, the shell regularizer improves the application-facing quantities, namely outer-wall flux and boundary-condition behavior. Across seeds 0-5 and 100k epochs, the most reliable tested configuration is a fixed shell weight of 5e-4 under the Kourkoutas-beta optimizer regime: relative to a matched run without the shell term, it reduces the mean outer-wall BC RMSE from 1.22e-2 to 9.29e-4 and the mean wall-flux RMSE from 9.21e-3 to 9.63e-4. Adam with beta2=0.999 becomes usable when the initial learning rate is reduced to 1e-3, although its shell benefit is less robust than under Kourkoutas-beta. Overall, the results support a targeted view of hybrid PINNs: an auxiliary-only FD regularizer is most valuable when it is aligned with the physical quantity of interest, here the outer-wall flux.

Figures & tables

Inherited benchmark ingredients	New in this manuscript
Steady PINN3D annular heat-conduction workload, wavy outer-wall geometry, six-term weighted objective, reference FD solution, and the base MLP/optimizer context from the prior PINN3D and Kourkoutas-β work [ 8 , 9 ] .	An auxiliary-only FD regularizer applied to the sampled AD residual field ; the controlled Stage-1 Poisson mechanism study; the body-fitted outer-shell deployment in Stage 2; seedwise wall-reference audits; and the optimizer and learning-rate (LR) sensitivity checks used to interpret the Stage-2 result carefully.

Table 1: Benchmark inheritance versus new contribution. The present manuscript reuses the PINN3D annular benchmark from earlier work, but adds a new structured residual-gradient regularizer together with a two-stage empirical study designed to separate mechanism from application.

Figure 1: Stage-2 problem geometry and auxiliary shell. Left: cross-section of the annular domain with wavy outer wall

r_{o}(\theta)

and the shaded body-fitted shell used by the FD regularizer. Right: structured shell bank in computational coordinates

(s,\theta,z)

. The indices

(i,j,k)

refer respectively to radial shell line, azimuthal node, and axial node.

Item	Stage-2 protocol used in this paper
Physical domain	Steady cylindrical annulus with wavy outer wall $r_{o}(\theta)=r_{\max}+0.25\,r_{\max}\sin(3\theta)$ , with $r_{\min}=0.2$ , $r_{\max}=1.0$ , and $L=10r_{\max}$
Boundary conditions	Inner wall and inlet at unit temperature; outlet Neumann; prescribed flux on the outer wall; periodicity in $\theta$
Base objective	Six weighted terms: PDE, inner, inlet, outlet, outer, and periodic/theta contributions inherited from the PINN3D benchmark [ 8 ]
Model	Fully connected 16-layer MLP of width 128 with SiLU activations and scalar temperature output
Training cloud	4000 interior points; 2000 points on each of inner, inlet, outlet, and outer boundaries; 400 periodic point pairs
Validation cloud	1024 interior points; 512 points on each boundary; 200 periodic point pairs

Table 2: Stage-2 protocol. The benchmark, architecture, and base objective are inherited from the PINN3D/Kourkoutas-β line, while the shell regularizer and the wall-audit logic are the new ingredients introduced here.

model	best val ( $\times 10^{-4}$ )	rel $L^{2}(u)$ ( $\times 10^{-3}$ )	rel $L^{2}(\nabla u)$ ( $\times 10^{-3}$ )	residual RMSE ( $\times 10^{-2}$ )	$\\|\nabla R\\|$ RMSE
PINN off	$6.14\pm 0.30$	$2.46\pm 0.50$	$8.75\pm 1.79$	$2.45\pm 0.10$	$1.467\pm 0.175$
FD fixed	$3.20\pm 0.68$	$2.41\pm 0.35$	$8.26\pm 1.12$	$1.63\pm 0.21$	$0.996\pm 0.052$
FD linear	$4.20\pm 0.48$	$2.29\pm 0.38$	$8.08\pm 1.27$	$1.98\pm 0.16$	$1.162\pm 0.032$
AD fixed	$\mathbf{3.01}\pm 0.78$	$2.49\pm 0.33$	$8.24\pm 1.06$	$\mathbf{1.49}\pm 0.21$	$\mathbf{0.821}\pm 0.092$
AD linear	$3.82\pm 0.55$	$\mathbf{2.24}\pm 0.36$	$\mathbf{7.85}\pm 1.18$	$1.86\pm 0.18$	$1.038\pm 0.044$

Table 3: Core 30k three-seed Stage-1 comparison on the fresh Sobol audit. Entries are mean

\pm

sample standard deviation over seeds 0, 1, and 2. In the scaled columns, the numbers shown are mantissas under the power of ten indicated in the header.

Figure 2: Three-seed mean field-residual frontier for Stage 1. The fixed AD and fixed FD runs (squares) achieve the lowest residual RMSE, while the scheduled runs (circles) achieve the best field accuracy among the core regularized arms.

Figure 3: Mean fresh-cloud error reductions relative to the plain PINN baseline. Positive bars mean lower mean error than the baseline. Both schedules improve all four fresh-cloud metrics shown here.

model	final loss ( $\times 10^{-5}$ )	wall BC RMSE ( $\times 10^{-3}$ )	$dT/dn$ RMSE ( $\times 10^{-3}$ )	$T_{\mathrm{wall}}$ RMSE ( $\times 10^{-2}$ )
OFF	$2.86\pm 5.94$	$12.17\pm 9.00$	$9.21\pm 6.09$	$5.06\pm 5.55$
shell fixed	$\mathbf{0.231}\pm 0.351$	$\mathbf{0.929}\pm 0.403$	$\mathbf{0.963}\pm 0.418$	$\mathbf{1.035}\pm 0.009$

Table 4: Stage-2 six-seed 100k comparison on the wavy-annulus benchmark under the main Kourkoutas-β regime. Entries are mean

\pm

sample standard deviation over seeds 0–5. In the scaled columns, the numbers shown are mantissas under the power of ten indicated in the header.

Figure 4: Mean Stage-2 reductions relative to the OFF baseline across seeds 0–5. The fixed shell improves all four metrics shown here, with the largest relative gains on the wall-flux and wall-BC quantities that motivate the shell construction.

Figure 5: Six-seed Stage-2 comparison on the two primary wall-facing metrics under the main Kourkoutas-β regime. Seeds are shown as discrete replicate categories rather than along a continuous axis. The fixed shell (squares) improves both the wall BC RMSE and the wall-flux RMSE on all six seeds.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

direction	comparison	relative $L_{2}$ change	max pointwise change
axial	$(97,48,49)\to(97,48,97)$	$1.62\times 10^{-4}$	$6.66\times 10^{-4}$
axial	$(25,48,49)\to(25,48,97)$	$1.58\times 10^{-4}$	—
axial	$(25,48,97)\to(25,48,193)$	$4.61\times 10^{-5}$	—
radial ( $T_{\mathrm{wall}}$ )	$(25,48,97)\to(193,48,97)$	$1.47\times 10^{-2}$	$1.79\times 10^{-2}$
radial ( $\partial_{n}T_{\mathrm{wall}}$ )	$(25,48,97)\to(193,48,97)$	$1.71\times 10^{-5}$	$2.06\times 10^{-5}$
radial ( $T_{\mathrm{wall}}$ )	$(25,48,193)\to(193,48,193)$	$1.47\times 10^{-2}$	$1.79\times 10^{-2}$

Appendix

Table 5: Summary of the FD grid checks used to calibrate the Stage 2 wall-reference solution. Relative

L_{2}

changes are computed on common grid points after refinement. For the wall-reference slice, the most important quantity is the wall-normal flux; the corresponding radial-refinement changes are substantially smaller than the shell-versus-OFF differences reported in the main Stage 2 results.

optimizer	init. LR	arm	final loss	wall BC RMSE	$dT/dn$ RMSE	$T_{\mathrm{wall}}$ RMSE
Kourkoutas-β	$7.5\times 10^{-3}$	OFF	$5.048\times 10^{-7}$	$4.053\times 10^{-3}$	$3.722\times 10^{-3}$	$5.675\times 10^{-2}$
Kourkoutas-β	$7.5\times 10^{-3}$	fixed shell $5\times 10^{-4}$	$6.695\times 10^{-7}$	$1.225\times 10^{-3}$	$1.144\times 10^{-3}$	$1.033\times 10^{-2}$
Kourkoutas-β	$10^{-3}$	OFF	$1.052\times 10^{-5}$	$2.003\times 10^{-3}$	$2.076\times 10^{-3}$	$1.056\times 10^{-2}$
Kourkoutas-β	$10^{-3}$	fixed shell $5\times 10^{-4}$	$4.751\times 10^{-6}$	$1.392\times 10^{-3}$	$1.446\times 10^{-3}$	$1.059\times 10^{-2}$
Adam95	$7.5\times 10^{-3}$	OFF	$1.576\times 10^{25}$	$8.533\times 10^{7}$	$6.322\times 10^{8}$	$5.762\times 10^{5}$
Adam95	$7.5\times 10^{-3}$	fixed shell $5\times 10^{-4}$	$2.614\times 10^{-1}$	$3.232\times 10^{-1}$	$3.229\times 10^{-1}$	$5.748\times 10^{-1}$

Appendix

Table 6: Seed-2 optimizer and initial-learning-rate sensitivity for Stage 2, comparing OFF and the fixed shell. The inherited schedule starts from

7.5\times 10^{-3}

and decays to

10^{-5}

; the milder probes use the same cosine form but start from

10^{-3}

seed	final loss		wall BC RMSE		$dT/dn$ RMSE		$T_{\mathrm{wall}}$ RMSE
seed	OFF	fixed shell	OFF	fixed shell	OFF	fixed shell	OFF	fixed shell
0	$7.662\text{\times}{10}^{-7}$	$3.981\text{\times}{10}^{-7}$	$1.960\text{\times}{10}^{-2}$	$5.987\text{\times}{10}^{-4}$	$1.310\text{\times}{10}^{-2}$	$6.264\text{\times}{10}^{-4}$	$1.136\text{\times}{10}^{-2}$	$1.024\text{\times}{10}^{-2}$
1	$1.495\text{\times}{10}^{-4}$	$1.308\text{\times}{10}^{-6}$	$2.125\text{\times}{10}^{-2}$	$8.006\text{\times}{10}^{-4}$	$1.369\text{\times}{10}^{-2}$	$8.380\text{\times}{10}^{-4}$	$8.920\text{\times}{10}^{-3}$	$1.027\text{\times}{10}^{-2}$
2	$5.048\text{\times}{10}^{-7}$	$6.695\text{\times}{10}^{-7}$	$4.053\text{\times}{10}^{-3}$	$1.225\text{\times}{10}^{-3}$	$3.722\text{\times}{10}^{-3}$	$1.144\text{\times}{10}^{-3}$	$5.675\text{\times}{10}^{-2}$	$1.033\text{\times}{10}^{-2}$
3	$1.019\text{\times}{10}^{-5}$	$9.446\text{\times}{10}^{-6}$	$2.219\text{\times}{10}^{-3}$	$1.605\text{\times}{10}^{-3}$	$2.293\text{\times}{10}^{-3}$	$1.726\text{\times}{10}^{-3}$	$1.049\text{\times}{10}^{-2}$	$1.039\text{\times}{10}^{-2}$
4	$6.170\text{\times}{10}^{-7}$	$1.070\text{\times}{10}^{-6}$	$2.009\text{\times}{10}^{-2}$	$7.450\text{\times}{10}^{-4}$	$1.691\text{\times}{10}^{-2}$	$8.012\text{\times}{10}^{-4}$	$1.521\text{\times}{10}^{-1}$	$1.037\text{\times}{10}^{-2}$

Appendix

Table 7: Completed six-seed Stage 2 sweep under the main Kourkoutas-β regime, comparing OFF and the fixed shell.

seed	final loss		wall BC RMSE		$dT/dn$ RMSE		$T_{\mathrm{wall}}$ RMSE
seed	OFF	fixed shell	OFF	fixed shell	OFF	fixed shell	OFF	fixed shell
0	$2.615\text{\times}{10}^{-1}$	$9.933\text{\times}{10}^{-5}$	$3.232\text{\times}{10}^{-1}$	$6.105\text{\times}{10}^{-3}$	$3.229\text{\times}{10}^{-1}$	$6.055\text{\times}{10}^{-3}$	$5.748\text{\times}{10}^{-1}$	$2.241\text{\times}{10}^{-2}$
1	$4.738\text{\times}{10}^{-5}$	$6.606\text{\times}{10}^{-6}$	$1.409\text{\times}{10}^{-2}$	$1.487\text{\times}{10}^{-3}$	$1.167\text{\times}{10}^{-2}$	$1.538\text{\times}{10}^{-3}$	$2.057\text{\times}{10}^{-1}$	$1.057\text{\times}{10}^{-2}$
2	$3.195\text{\times}{10}^{-4}$	$1.385\text{\times}{10}^{-5}$	$2.100\text{\times}{10}^{-2}$	$2.232\text{\times}{10}^{-3}$	$2.017\text{\times}{10}^{-2}$	$2.338\text{\times}{10}^{-3}$	$2.138\text{\times}{10}^{-1}$	$1.028\text{\times}{10}^{-2}$
3	$3.915\text{\times}{10}^{-6}$	$1.010\text{\times}{10}^{-5}$	$1.268\text{\times}{10}^{-3}$	$1.920\text{\times}{10}^{-3}$	$1.337\text{\times}{10}^{-3}$	$1.964\text{\times}{10}^{-3}$	$1.042\text{\times}{10}^{-2}$	$1.056\text{\times}{10}^{-2}$
4	$1.794\text{\times}{10}^{-4}$	$3.203\text{\times}{10}^{-4}$	$7.926\text{\times}{10}^{-3}$	$1.005\text{\times}{10}^{-2}$	$7.732\text{\times}{10}^{-3}$	$9.996\text{\times}{10}^{-3}$	$1.543\text{\times}{10}^{-2}$	$8.466\text{\times}{10}^{-3}$

Appendix

Table 8: Completed six-seed Adam999 sweep at initial learning rate

10^{-3}

, comparing OFF and the fixed shell.

FD weight	best val	rel $L^{2}(u)$	rel $L^{2}(\nabla u)$	residual RMSE	$\\|\nabla R\\|$ RMSE
off	2.178e-03	3.708e-03	1.170e-02	5.187e-02	2.029e+00
1e-5	2.142e-03	3.691e-03	1.163e-02	5.139e-02	2.009e+00
1e-4	1.895e-03	3.619e-03	1.119e-02	4.804e-02	1.870e+00
5e-4	1.260e-03	3.401e-03	1.051e-02	3.905e-02	1.510e+00
1e-3	9.767e-04	3.316e-03	9.882e-03	3.390e-02	1.324e+00

Appendix

Table 9: 10k FD-resgrad sweep on the Stage-1 Poisson benchmark (one matched seed triplet).

FD weight	grid strategy	best val	rel $L^{2}(u)$	rel $L^{2}(\nabla u)$	residual RMSE	$\\|\nabla R\\|$ RMSE
1e-3	fixed-safe	5.366e-04	2.489e-03	8.193e-03	2.257e-02	1.222e+00
1e-3	cycle4	6.893e-04	5.196e-03	8.315e-03	2.559e-02	1.191e+00
1e-3	jitter4	6.676e-04	3.565e-03	8.277e-03	2.490e-02	1.205e+00
1e-2	fixed-safe	3.983e-04	3.723e-03	1.123e-02	1.217e-02	7.689e-01
1e-2	cycle4	7.197e-04	5.224e-03	1.166e-02	2.190e-02	7.235e-01
1e-2	jitter4	1.457e-03	4.258e-03	1.155e-02	3.540e-02	8.042e-01

Appendix

Table 10: 20k anti-grid-lock ablation on the phase-safe

64\times 64

family.

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 268 papers

Jun 1, 2026

Physics-Informed Residuals for Adaptive Mesh Refinement in Finite-Difference PDE Solvers

Classical finite-difference solvers remain reliable tools for partial differential equations, but their efficiency depends on where mesh resolution is placed. Uniform refinement can waste degrees of freedom when solution difficulty is localised near sharp gradients, fronts, oscillations, or constraint-sensitive regions. This paper studies a hybrid strategy in which a physics-informed neural network (PINN) is used not as the final solver, but as an off-grid residual probe for adaptive mesh refinement. The PINN residual is sampled over the domain, converted into cellwise indicators, and used to guide refinement before the final approximation is computed by a finite-difference solver. The method is evaluated on three benchmarks. The main full-solver validation uses the one-dimensional viscous Burgers equation with a nonuniform finite-difference solve on the adapted meshes. PINN-threshold refinement attains final relative

L^2

error

0.021067

with

60

degrees of freedom, compared with

0.022617

for uniform refinement with

192

degrees of freedom. At matched mesh size, PINN-threshold reduces the error by about

67.5\%

. PINN-Dörfler refinement gives similar performance, with error

0.021264

using

58

degrees of freedom. A gradient indicator remains slightly more accurate, so the result supports usefulness rather than universal superiority. Manufactured 2D and 3D proxy tests, based on a nonlinear Schrödinger equation and an incompressible Navier--Stokes system, show that PINN residuals can organise structured refinement and improve over random refinement, although they do not consistently outperform gradient or uniform baselines. The results support PINN-guided AMR as a residual-indicator strategy for transferring physics-informed diagnostic information into finite-difference mesh adaptation while preserving the classical solver as the final approximation engine.

Henry Kasumba, Ronald Katende

Department of Mathematics · Makerere University · 7062, Kampala, Uganda +2

PDE Solving Adaptive Mesh Refinement

Ask PDF

Oct 4, 2026

Component-Level Evaluation of Adaptive PINN Training for CFD-Oriented Crystal Growth Simulation

Physics-informed neural network (PINN) training minimizes a weighted combination of partial differential equation (PDE), boundary-condition, and initial-condition losses. Because adaptive methods modify these weights during training, their weighted total losses are not always directly comparable. We compare fixed-weight PINN, gradient-normalized PINN (GNPINN), and a rule-based adaptive controller (AgenticPINN) under matched settings on a heat-equation benchmark and a simplified Czochralski-oriented thermal-fluid problem. In the crystal-growth MLP experiment, adaptive control reduced the PDE residual from the order of

10^{-5}

10^{-6}

, while the boundary-condition loss increased from the order of

10^{-5}

10^{-2}

. On the heat-equation benchmark, GNPINN achieved the lowest relative

L_2

field error (0.054), whereas AgenticPINN obtained the smallest PDE residual but a relative

L_2

error of 1.368. Gaussian-process surrogates were additionally evaluated using case-wise holdout tests on corrected Czochralski CFD parameter sweeps. The temperature-field error for the temperature sweep was approximately 6%, whereas the axial-velocity error for the crystal-rotation sweep was approximately 42%. These findings show that adaptive control can improve equation satisfaction while weakening other physical constraints. PINN training should therefore be evaluated using separate PDE, boundary-condition, and solution-error metrics rather than weighted total loss alone.

Niruta Chapagain, Rohit Raj, Bertwin Kurisinkal Shine +1

University of Europe for Applied Sciences Potsdam, Germany · Vienna, Austria · Technische Universität Chemnitz Chemnitz, Germany +1

Neural PDE Solvers PDE Surrogate Modeling Adaptive Loss Weighting Physics-Informed ML

Ask PDF

Mar 3, 2026

Stabilized Adaptive Loss and Residual-Based Collocation for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have been recognized as a mesh-free alternative to solve partial differential equations where physics information is incorporated. However, in dealing with problems characterized by high stiffness or shock-dominated dynamics, traditional PINNs have been found to have limitations, including unbalanced training and inaccuracy in solution, even with small physics residuals. In this research, we seek to address these limitations using the viscous Burgers' equation with low viscosity and the Allen-Cahn equation as test problems. In addressing unbalanced training, we have developed a new adaptive loss balancing scheme using smoothed gradient norms to ensure satisfaction of initial and boundary conditions. Further, to address inaccuracy in the solution, we have developed an adaptive residual-based collocation scheme to improve the accuracy of solutions in the regions with high physics residuals. The proposed new approach significantly improves solution accuracy with consistent satisfaction of physics residuals. For instance, in the case of Burgers' equation, the relative L2 error is reduced by about 44 percent compared to traditional PINNs, while for the Allen-Cahn equation, the relative L2 error is reduced by approximately 70 percent. Additionally, we show the trustworthy solution comparison of the proposed method using a robust finite difference solver.

Divyavardhan Singh, Shubham Kamble, Dimple Sonone +1

Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat, India

Physics-Informed ML PDE Solving Adaptive Loss Weighting

Ask PDF

Abstract

Neural PDE Solvers

Explore similar work

Physics-Informed Residuals for Adaptive Mesh Refinement in Finite-Difference PDE Solvers

Component-Level Evaluation of Adaptive PINN Training for CFD-Oriented Crystal Growth Simulation

Stabilized Adaptive Loss and Residual-Based Collocation for Physics-Informed Neural Networks