Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction
Authors: Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi, Chang Wei, Yuchen Fan, Yew-Soon Ong
Organizations: Institute of High Performance Computing, A*STAR, Singapore. · Centre for Frontier AI Research, A*STAR, Singapore. · School of Mechanical Engineering, Tianjin University, China. · College of Computing and Data Science, Nanyang Technological University, Singapore.
Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN), a learning strategy that bridges modern physics-informed learning with numerical algorithms. Scale-PINN incorporates the iterative residual-correction principle, a cornerstone of numerical solvers, directly into the loss formulation, marking a paradigm shift in how PINN losses can be conceived and constructed. This integration enables Scale-PINN to achieve unprecedented convergence speed across PDE problems from different physics domain, including reducing training time on a challenging fluid-dynamics problem for state-of-the-art PINN from hours to sub-2 minutes while maintaining superior accuracy, and enabling application to representative problems in aerodynamics and urban science. By uniting the rigor of numerical methods with the flexibility of deep learning, Scale-PINN marks a significant leap toward the practical adoption of PINNs in science and engineering through scalable, physics-informed learning. Codes are available at https://github.com/chiuph/SCALE-PINN.
Figures & tables
Figure 1: Scale-PINN schematic and result highlights. Scale-PINN includes a sequential correction term through application of the residual smoothing operator Pα=(I−α2∇2) to the change in solution F:=f(⋅;wk)−f(⋅;wk−1) during iterative optimization. A convergence plot on the Navier-Stokes (N-S) example, lid‑driven cavity flow at Re=3200 , shows competitive time‑to‑accuracy versus numerical solvers. Compared to other PINN methods, Scale-PINN solves the lid‑driven cavity flow to state-of-the-art accuracy with unprecedented speed, i.e., ∼ 90s for Re=3200 and ∼ 150s for Re=7500 . Results for Kuramoto–Sivashinsky (K-S), Grey–Scott (G-S), Korteweg–De Vries (KdV), and Allen–Cahn (AC) equations demonstrate accuracy across diverse dynamics. Scale-PINN model architecture and training strategies are detailed in Method 4.5 .
Figure 2: (a) Experimental analysis on lid‑driven cavity flow at Re=400 shows that the convergence of a vanilla PINN can be improved by increasing batch size (400 → 4,000) and reducing learning rate (1e -3→ 1e -4 ), albeit at a slower pace ( ∼ 1800s). Scale-PINN requires substantially less training iterations to reach orders of magnitude higher accuracy ( ∼ 90s), while using 1 order of magnitude smaller batch size and higher learning rate. Comparing their intermediate flow fields progressing from a few iterations to 50 k -500 k iterations, and mid‑section profiles against the Ghia et al. Ghia et al. (1982) benchmark, Scale‑PINN attains accurate flow structures far earlier. (b) Scale-PINN can converge to an accurate solution even when the Reynolds number is increased to Re=3200 , without the need to increase batch size and number of training iterations. A vanilla PINN struggles to solve the Re=3200 case, as it becomes trapped in incorrect flow patterns, indicating premature convergence.
Figure 3: (a) Scale‑PINN solves the lid-driven cavity flow from Re=400 to Re=20k with state‑of‑the‑art accuracy and efficient training, as shown in the summary table of error, training time, and other parameters, alongside the representative velocity fields and absolute error maps ( Re=400 and Re=20k ) to confirm that residuals remain small and largely confined to shear layers and vortex cores. For all simulated cases, their MSE consistently below 1e -4 , and their mid‑section u and v -velocity profiles (colored lines) show excellent agreement with the classic benchmark results (marked points) for numerical solvers, i.e., Ghia Ghia et al. (1982) for up to Re=10k and Erturk Erturk (2009) for Re=20k . (b) Scale-PINN establishes a sub-2 minutes training regime on lid-driven cavity flow ( Re=3200 ), whereas recent PINN variants require hours to approach comparable accuracy. (c) Scale-PINN scales favorably with problem complexity ( Re : 400 → 3200), enabling the solution of more complex problems within a feasible time scale.
Figure 4: Scale-PINN predictions, with streamlines overlaid on velocity magnitude contours, for flow past (a) a single-airfoil at Re=1000 and (b) staggered airfoils at Re=500 are compared to reference solutions obtained from CFD. Component-wise fields ( u , v , p ) and absolute error maps indicate good agreement between Scale-PINN and CFD across the domain, including wakes behind airfoil. Surface pressure‑coefficient ( Cp ) traces along the airfoil(s) closely match CFD and literature curves. Only the near-field region is shown for clarity, where the flow patterns around the airfoil(s) emerge; the actual computational domain extends well beyond the visualized region. Scale-PINN reaches accurate solutions within ∼ 180s of training, achieving near-field velocity relative errors of 1.7e-2 ( 3.47e-3 full domain) for the single-airfoil case and 1.96e-2 ( 4.79e-3 full domain) for the staggered airfoils case. These results validate the accuracy and efficiency of Scale-PINN in resolving canonical aerodynamic flow features at moderate Reynolds number.
Figure 5: (a) Scale-PINN simulates the flow past square cylinders in open domain, where the predicted fields ( u , v , p ) accurately capture the wake structure and recirculation zones, with consistently low absolute errors against the reference solution obtained from CFD. Contours are shown in the near field for clarity; the actual computational domain extends further. Scale-PINN reaches accurate solutions within ∼ 285s of training, achieving near-field velocity relative errors of 9.21e-3 ( 4.86e-3 full domain). (b) Scale-PINN simulates Rayleigh–Bénard convection at Ra=100k . The temperature contours, overlaid with velocity streamlines, show close agreement between predicted roll patterns and reference solutions obtained from CFD, across multiple time snapshots (1-50s). Scale-PINN achieves accurate solutions within ∼ 390s of training, with relative errors 1.99e-2 for velocity and 3.2e-2 for temperature. These cases validate the method’s robustness across bluff-body geometries in open domain and thermally driven, time-dependent convection.
Figure 6: Scale-PINN attains state-of-the-art, minute‑scale training efficiency on a range of PDE benchmarks, converging within ∼ 10 minutes while achieving accuracy on par with second-order optimization method (typically requires hours of training). For each PDE benchmark problem, relative error (log scale) versus training time is shown for five independent initializations; the final (best) Scale-PINN solution, final baseline PINN solution, and their corresponding initial (random initialization) solution are shown alongside error-time curves. We keep the model architecture (moderate-sized multilayer perceptron) and training settings identical and change only the PDE residual term, i.e., sequential corrected (Scale-PINN) vs. standard PDE loss (baseline). The accompanying table reports training times and best/average errors over the five runs, demonstrating robustness to initialization and broad applicability across PDE families.
Lick
=∥f(⋅,t=0;wk)−u0(⋅)∥L2(Ω)2
Lbck
=∥B[f(⋅;wk)]−g(⋅)∥L2(∂Ω×(0,T])2
Algorithm 1 Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN)
Physics-informed neural networks (PINNs) often face ill-conditioned objectives that limit high-accuracy training. Dense quasi-Newton methods improve local conditioning but require expensive optimizer state, while Kronecker-factored methods such as SOAP scale to larger networks but rely on periodic basis updates. We introduce \method, which augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry and an adaptive basis update followed by variance-state downscaling. We characterize the directional secant matching induced by the scalar correction and give a bound on variance-state mismatch across basis changes. Across eight PDE benchmarks, \method attains the lowest final residual on six, including Burgers and Boussinesq, while SOAP-family baselines perform better on Gray-Scott and Ginzburg-Landau. On Boussinesq, \method reaches a residual of 10−5 in 4.1 hours with 9.2 GB peak VRAM, while Adam does not reach this target within 14 hours. Three-seed L2 and H1 errors on four representative PDEs support the link between lower residuals and improved solution accuracy. These results position \method as a scalable option for stiff, high-accuracy physics-informed training, rather than a uniform replacement for existing optimizers.
Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke +2
McGill University · California Institute of Technology · Aarhus University +1
Physics-Informed Neural Networks (PINNs) combine deep learning with physical constraints for solving partial differential equations (PDEs), and are widely applied in fluid mechanics, heat transfer, and solid mechanics. However, PINN training still suffers from high-dimensional non-convex loss landscapes, imbalanced multiobjective constraints, and ineffective information propagation. Existing curriculum learning and causality-guided strategies improve training stability, but mainly focus on temporal or parametric progression, lacking explicit treatment of spatial information propagation and inter-region consistency. Moreover, they are not directly applicable to boundary value problems (BVPs) with strong spatial coupling. To address this issue, we propose a spatially correlated curriculum learning framework for PINNs. To the best of our knowledge, this is the first work to address PINN training difficulties from the perspective of spatial coupling among subregions. First, spatial causal weights guide information from near-boundary regions inward, reducing optimization failures and spurious convergence. Second, a low-frequency information bridge enforces pseudo-label-based consistency across spatially separated regions, suppressing global low-frequency drift. Third, a region-adaptive reweighting strategy adjusts subregion losses to reduce local residuals and recover high-frequency details. Experiments on PDE benchmarks show that, under comparable computational cost, the proposed method alleviates training failures and improves solution accuracy. The code is available at https://github.com/pigofmomo/CurriculumLearningPINN.
Xujia Chen, Xinyue Hu, Letian Chen +2
Department of Automation, Tsinghua University, Beijing 100084, P.R. China
Physics-informed neural networks (PINNs) are a versatile tool in the burgeoning field of scientific machine learning for solving partial differential equations (PDEs). However, determining suitable training strategies for them is not obvious, with the result that they typically yield moderately accurate solutions. In this article, we propose a method for improving the accuracy of PINNs by coupling them with a post-processing step that seeks the best approximation in a function space associated with the network. We find that our method yields errors four to five orders of magnitude lower than those of the parent PINNs across architectures and dimensions. Moreover, we can reuse the basis functions for the linear space in more complex settings, such as time-dependent and nonlinear problems, allowing for transfer learning. Our approach also provides a residual-based metric that allows us to optimally choose the number of basis functions employed.
Saad Qadeer, Panos Stinis
Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA