Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction

Pao-Hsiung Chiu; Jian Cheng Wong; Chin Chun Ooi; Chang Wei; Yuchen Fan; Yew-Soon Ong

cs.CEFeb 23, 2026

Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction

Authors: Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi, Chang Wei, Yuchen Fan, Yew-Soon Ong

Organizations: Institute of High Performance Computing, A*STAR, Singapore. · Centre for Frontier AI Research, A*STAR, Singapore. · School of Mechanical Engineering, Tianjin University, China. · College of Computing and Data Science, Nanyang Technological University, Singapore.

Abstract

Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN), a learning strategy that bridges modern physics-informed learning with numerical algorithms. Scale-PINN incorporates the iterative residual-correction principle, a cornerstone of numerical solvers, directly into the loss formulation, marking a paradigm shift in how PINN losses can be conceived and constructed. This integration enables Scale-PINN to achieve unprecedented convergence speed across PDE problems from different physics domain, including reducing training time on a challenging fluid-dynamics problem for state-of-the-art PINN from hours to sub-2 minutes while maintaining superior accuracy, and enabling application to representative problems in aerodynamics and urban science. By uniting the rigor of numerical methods with the flexibility of deep learning, Scale-PINN marks a significant leap toward the practical adoption of PINNs in science and engineering through scalable, physics-informed learning. Codes are available at https://github.com/chiuph/SCALE-PINN.

Figures & tables

Figure 1: Scale-PINN schematic and result highlights. Scale-PINN includes a sequential correction term through application of the residual smoothing operator $\mathcal{P}_{\alpha}=(I-{\alpha^{2}}\nabla^{2})$ to the change in solution $\mathcal{F}\coloneqq f(\cdot;\boldsymbol{w}^{k})-f(\cdot;\boldsymbol{w}^{k-1})$ during iterative optimization. A convergence plot on the Navier-Stokes (N-S) example, lid‑driven cavity flow at $Re=3200$ , shows competitive time‑to‑accuracy versus numerical solvers. Compared to other PINN methods, Scale-PINN solves the lid‑driven cavity flow to state-of-the-art accuracy with unprecedented speed, i.e., $\sim$ 90s for $Re=3200$ and $\sim$ 150s for $Re=7500$ . Results for Kuramoto–Sivashinsky (K-S), Grey–Scott (G-S), Korteweg–De Vries (KdV), and Allen–Cahn (AC) equations demonstrate accuracy across diverse dynamics. Scale-PINN model architecture and training strategies are detailed in Method 4.5 .

Figure 2: (a) Experimental analysis on lid‑driven cavity flow at $Re=400$ shows that the convergence of a vanilla PINN can be improved by increasing batch size (400 $\rightarrow$ 4,000) and reducing learning rate (1e ${}^{\text{-3}}$ $\rightarrow$ 1e ${}^{\text{-4}}$ ), albeit at a slower pace ( $\sim$ 1800s). Scale-PINN requires substantially less training iterations to reach orders of magnitude higher accuracy ( $\sim$ 90s), while using 1 order of magnitude smaller batch size and higher learning rate. Comparing their intermediate flow fields progressing from a few iterations to 50 $k$ -500 $k$ iterations, and mid‑section profiles against the Ghia et al. Ghia et al. (1982) benchmark, Scale‑PINN attains accurate flow structures far earlier. (b) Scale-PINN can converge to an accurate solution even when the Reynolds number is increased to $Re=3200$ , without the need to increase batch size and number of training iterations. A vanilla PINN struggles to solve the $Re=3200$ case, as it becomes trapped in incorrect flow patterns, indicating premature convergence.

Figure 3: (a) Scale‑PINN solves the lid-driven cavity flow from $Re=400$ to $Re=20k$ with state‑of‑the‑art accuracy and efficient training, as shown in the summary table of error, training time, and other parameters, alongside the representative velocity fields and absolute error maps ( $Re=400$ and $Re=20k$ ) to confirm that residuals remain small and largely confined to shear layers and vortex cores. For all simulated cases, their MSE consistently below 1e ${}^{\text{-4}}$ , and their mid‑section $u$ and $v$ -velocity profiles (colored lines) show excellent agreement with the classic benchmark results (marked points) for numerical solvers, i.e., Ghia Ghia et al. (1982) for up to $Re=10k$ and Erturk Erturk (2009) for $Re=20k$ . (b) Scale-PINN establishes a sub-2 minutes training regime on lid-driven cavity flow ( $Re=3200$ ), whereas recent PINN variants require hours to approach comparable accuracy. (c) Scale-PINN scales favorably with problem complexity ( $Re$ : 400 $\rightarrow$ 3200), enabling the solution of more complex problems within a feasible time scale.

Figure 4: Scale-PINN predictions, with streamlines overlaid on velocity magnitude contours, for flow past (a) a single-airfoil at $Re=1000$ and (b) staggered airfoils at $Re=500$ are compared to reference solutions obtained from CFD. Component-wise fields ( $u$ , $v$ , $p$ ) and absolute error maps indicate good agreement between Scale-PINN and CFD across the domain, including wakes behind airfoil. Surface pressure‑coefficient ( $C_{p}$ ) traces along the airfoil(s) closely match CFD and literature curves. Only the near-field region is shown for clarity, where the flow patterns around the airfoil(s) emerge; the actual computational domain extends well beyond the visualized region. Scale-PINN reaches accurate solutions within $\sim$ 180s of training, achieving near-field velocity relative errors of $1.7e^{\text{-2}}$ ( $3.47e^{\text{-3}}$ full domain) for the single-airfoil case and $1.96e^{\text{-2}}$ ( $4.79e^{\text{-3}}$ full domain) for the staggered airfoils case. These results validate the accuracy and efficiency of Scale-PINN in resolving canonical aerodynamic flow features at moderate Reynolds number.

Figure 5: (a) Scale-PINN simulates the flow past square cylinders in open domain, where the predicted fields ( $u$ , $v$ , $p$ ) accurately capture the wake structure and recirculation zones, with consistently low absolute errors against the reference solution obtained from CFD. Contours are shown in the near field for clarity; the actual computational domain extends further. Scale-PINN reaches accurate solutions within $\sim$ 285s of training, achieving near-field velocity relative errors of $9.21e^{\text{-3}}$ ( $4.86e^{\text{-3}}$ full domain). (b) Scale-PINN simulates Rayleigh–Bénard convection at $Ra=100k$ . The temperature contours, overlaid with velocity streamlines, show close agreement between predicted roll patterns and reference solutions obtained from CFD, across multiple time snapshots (1-50s). Scale-PINN achieves accurate solutions within $\sim$ 390s of training, with relative errors $1.99e^{\text{-2}}$ for velocity and $3.2e^{\text{-2}}$ for temperature. These cases validate the method’s robustness across bluff-body geometries in open domain and thermally driven, time-dependent convection.

Figure 6: Scale-PINN attains state-of-the-art, minute‑scale training efficiency on a range of PDE benchmarks, converging within $\sim$ 10 minutes while achieving accuracy on par with second-order optimization method (typically requires hours of training). For each PDE benchmark problem, relative error (log scale) versus training time is shown for five independent initializations; the final (best) Scale-PINN solution, final baseline PINN solution, and their corresponding initial (random initialization) solution are shown alongside error-time curves. We keep the model architecture (moderate-sized multilayer perceptron) and training settings identical and change only the PDE residual term, i.e., sequential corrected (Scale-PINN) vs. standard PDE loss (baseline). The accompanying table reports training times and best/average errors over the five runs, demonstrating robustness to initialization and broad applicability across PDE families.

	$\displaystyle\mathcal{L}_{ic}^{k}$	$\displaystyle=\lVert f(\cdot,t=0;\boldsymbol{w}^{k})-u_{0}(\cdot)\rVert_{L^{2}(\Omega)}^{2}$
	$\displaystyle\mathcal{L}_{bc}^{k}$	$\displaystyle=\lVert\mathcal{B}[f(\cdot;\boldsymbol{w}^{k})]-g(\cdot)\rVert_{L^{2}(\partial\Omega\times(0,T])}^{2}$

Algorithm 1 Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN)

Explore similar work

SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning
Aug 29, 2026Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke +2Neural Network Optimization Physics-Informed ML
McGill University · California Institute of Technology · Aarhus University +1
Curriculum Learning of Physics-Informed Neural Networks based on Spatial Correlation
May 14, 2026Xujia Chen, Xinyue Hu, Letian Chen +2PDE Solving Curriculum Learning
Department of Automation, Tsinghua University, Beijing 100084, P.R. China
Improving the accuracy of physics-informed neural networks via last-layer retraining
Mar 4, 2026Saad Qadeer, Panos StinisNeural PDE Solvers Physics-Informed ML
Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA

Aug 29, 2026J/K move · Enter open · S save

SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning

Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke, Yixuan Wang+1

McGill University · California Institute of Technology · Aarhus University+1

Neural Network Optimization Physics-Informed ML Adaptive Gradient Methods

Open paper AskPDF arXiv

Physics-informed neural networks (PINNs) often face ill-conditioned objectives that limit high-accuracy training. Dense quasi-Newton methods improve local conditioning but require expensive optimizer state, while Kronecker-factored methods such as SOAP scale to larger networks but rely on periodic basis updates. We introduce \method, which augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry and an adaptive basis update followed by variance-state downscaling. We characterize the directional secant matching induced by the scalar correction and give a bound on variance-state mismatch across basis changes. Across eight PDE benchmarks, \method attains the lowest final residual on six, including Burgers and Boussinesq, while SOAP-family baselines perform better on Gray-Scott and Ginzburg-Landau. On Boussinesq, \method reaches a residual of

10^{-5}

in 4.1 hours with 9.2 GB peak VRAM, while Adam does not reach this target within 14 hours. Three-seed

L^2

and

H^1

errors on four representative PDEs support the link between lower residuals and improved solution accuracy. These results position \method as a scalable option for stiff, high-accuracy physics-informed training, rather than a uniform replacement for existing optimizers.

Abstract

Explore similar work

SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning

Curriculum Learning of Physics-Informed Neural Networks based on Spatial Correlation

Improving the accuracy of physics-informed neural networks via last-layer retraining