Deflation-PINNs: Learning Multiple Solutions for PDEs and Landau-de Gennes
Authors: Sean Disarò, Ruma Rani Maity, Aras Bacho
Organizations: Department of Mathematics, Indian Institute of Science, Bangalore, 560012, India · Department of Computing and Mathematical Sciences, California Institute of Technology, CA, USA
Nonlinear Partial Differential Equations (PDEs) are ubiquitous in mathematical physics and engineering. Although Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for solving PDE problems, they typically struggle to identify multiple distinct solutions, since they are designed to find one solution at a time. To address this limitation, we introduce Deflation-PINNs, a novel framework that integrates a deflation loss with an architecture based on PINNs and Deep Operator Networks (DeepONets). By incorporating a deflation term into the loss function, our method systematically forces the Deflation-PINN to seek and converge upon distinct finitely many solution branches. We provide theoretical results on the approximation capabilities of our model and demonstrate the efficacy of Deflation-PINNs through numerical experiments on the Landau-de Gennes model of liquid crystals, a system renowned for its complex energy landscape and multiple equilibrium states, and on an Allen--Cahn benchmark whose solution set is provably known. Our results show that Deflation-PINNs can successfully identify and characterize multiple distinct crystal structures: a single unsupervised run recovers all six stable states of the benchmark, each branch certified to lie in the basin of attraction of a different equilibrium, and the discovered branches are refined to percent-level accuracy by a purely neural Deflation--Deep-Ritz stage and to the accuracy of a mesh-converged reference by a classical solver that they initialize.
Figures & tables
Figure 1: Architecture of the Deflation-PINN. Unlike classical DeepONets, the branch weights βji are learned and do not use domain dependent sensor points. The trunk net takes the point x∈Ω at which we want to evaluate the respective solution. The output is computed by taking the dot product of the respective branch weights (β1i,...,βpi) and the trunk net evaluations (τ1(x),...,τp(x)) to approximate the solution ui(x)
Figure 2
Figure 2: Nematic director fields for the two diagonal states (D1, D2) and four rotated states (R1–R4) of the LdG problem ( 3.2 ). Rows one and three show the solutions QNN obtained by a single Deflation-PINN trained once with K=6 branches, while rows two and four show the corresponding FEM reference solutions QFEM from Maity et al. (2021b) .
Discovery
Deflation–Deep-Ritz
Classical Refinement
rel. L2
rel. L2
rel. L∞
rel. H1
rel. L2
rel. L∞
rel. H1
D1
0.58
0.013
0.21
0.16
2.5⋅10−12
4.4⋅10−11
2.0⋅10−11
D2
0.59
0.013
0.21
0.15
2.5⋅10−12
4.4⋅10−11
2.0⋅10−11
R1
0.32
0.013
0.19
0.14
2.5⋅10−12
4.4⋅10−11
1.9⋅10−11
R2
0.30
0.013
0.19
0.14
2.5⋅10−12
4.4⋅10−11
1.9⋅10−11
R3
0.30
0.012
0.19
0.14
2.5⋅10−12
4.4⋅10−11
1.9⋅10−11
Table 1: Errors of the six solution branches against the mesh-converged reference of Section 3.3 , at the three stages of the pipeline: the discovery stage (a single unsupervised Deflation-PINN run, full vector-field deflation; relative L2 ), the purely neural Deflation–Deep-Ritz refinement of Section 3.4 , and the classical energy-descent/Newton refinement of Section 3.3 initialized by the branches (each with relative L2 , relative L∞ normalized by max∣Qref∣ , and relative H1 via first-order finite differences of the error). The Deflation–Deep-Ritz block is evaluated on the 5132 reference grid; its L∞ and H1 errors are dominated by the four defect cores of width ∼ϵ , cf. Figure 4 . The classical-refinement block is evaluated against the discrete reference on the common h=1/256 grid: all three norms are at the rounding level of the Newton solve, i.e., the refined branches coincide with the reference solutions. The association of branches with states is not by proximity alone but by basin membership: flowing each branch under the LdG gradient flow yields a different one of the six stable states (Section 3.3 ), so each branch is an approximation of exactly one state in the dynamically meaningful sense.
Figure 3: Contour maps of the pointwise error ∣QNN−Qref∣ of the six Deflation–Deep-Ritz solutions. The error is concentrated at the four corner defect cores ( 59 – 77% of the squared error within the four corner squares of side 0.1 ); away from the cores the fields are essentially exact.
Figure 5: The six solutions obtained by the Deflation–Deep-Ritz refinement (Table 1 , Deflation–Deep-Ritz block): nematic director (white segments) over the scalar order parameter ∣Q∣ (color). Every branch exhibits ∣Q∣≈1 in the bulk, defect cores only at the four corners, no interior order-reconstruction walls, and the exact director topology of its stable state.
Branch matched to
Rel. L2 error
Abs. L2 error
Mean ∣ residual ∣
+u∗
4.7⋅10−5
2.7⋅10−5
5.5⋅10−4
−u∗
1.4⋅10−4
8.2⋅10−5
6.1⋅10−4
u≡0
—
2.0⋅10−5
1.3⋅10−4
Table 2: Allen–Cahn benchmark: errors of the three branches learned by a single Deflation-PINN against the radial shooting reference u∗ ( ∥u∗∥L2(B)=0.571449 ), integrated on an independent 401×401 polar grid with area weights. For the trivial branch the relative error is not defined, so we report the absolute L2 norm; the PDE residual is the mean of ∣Δu+6u−u3∣ over a fine evaluation grid.
Figure 6: The three solutions of the Allen–Cahn benchmark on the unit disk learned simultaneously by a single Deflation-PINN with K=3 branches, evaluated on a fine polar grid: (left) u− , the branch matched to −u∗ ; (middle) u0 , the branch matched to the trivial solution (note the 10−5 color scale); (right) u+ , the branch matched to +u∗ , with maximum ≈0.617 in agreement with the shooting reference u∗(0)≈0.61702 . Quantitative errors are reported in Table 2 .
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
Distinct basins
Rel. L2 error (min / median / max)
Collocation sweep (width 4000 ):
17×17
4/6
0.29 / 0.59 / 0.81
25×25
4/6
0.30 / 0.47 / 0.92
33×33 (Section 3 )
6/6
0.30 / 0.31 / 0.59
49×49
5/6
0.29 / 0.29 / 0.87
Width sweep ( 33×33 collocation):
Appendix
Table 3: Convergence of the discovery stage in collocation density and network width. Per configuration: number of distinct stable states reached by the six branches under the gradient-flow/Newton census, and the minimum, median and maximum relative L2 error of the branches against the mesh-converged reference (branches matched to states by minimum-cost assignment).
Run
Certified basins
Distinct
Cumulative union
1
D1, D2, R3
3/6
3/6
2
D1, D2, R3, R4
4/6
4/6
3
D1, D2, R3
3/6
4/6
4
D1, D2, R3, R4
4/6
4/6
5
D2, R1, R2, R3
4/6
6/6
6
D1, D2
2/6
6/6
Appendix
Table 4: Restart study: eight independent discovery runs at the published configuration, the stable states certified by the gradient-flow/Newton census, and the cumulative union of certified basins. The union covers all six states after five runs; the census is unsupervised, so “stop when consecutive restarts add no new basin” is a practical stopping rule.
dmin
Final residual
Terminal hinge
Verdict
Allen–Cahn (true minimal separation ≈0.272 ):
0.2 (endpoint)
4.0⋅10−5
0.000
good
0.25
4.2⋅10−5
0.000
good
0.275
5.9⋅10−5
0.000
good
0.2875
1.8⋅10−4
0.0028
bad
0.3
2.0⋅10−4
0.027
bad
Appendix
Table 5: All queries of the two dmin bisections, sorted by dmin . Allen–Cahn block: the terminal hinge value is zero for dmin≤0.275 and positive for dmin≥0.2875 , while the final residual jumps by a factor 3 – 5 across the same threshold—the transition localizes the true minimal separation 0.272 to within 6% , i.e. it recovers the geometry of the solution set. LdG block: the hinge is satisfied on both sides of the edge, which instead manifests as a residual plateau ( † the disagreement between the two seeds at dmin=0.5125 is the seed-dependent transition band around 0.5 discussed in the text).
Physics-Informed Neural Networks (PINNs) are a common class of machine learning-based partial differential equation (PDE) solvers which train a network to represent a solution by minimizing a residual loss that encodes the PDE. Despite their successes, they are known to fail on certain simple equations, converging to an incorrect solution despite low loss. These failure modes have garnered significant attention in the literature over the past several years, motivating both architectural and optimization based solutions. By directly visualizing the residual, we show that failure modes are the result of overfitting: the loss is minimized on the collocation points, but not elsewhere. Applying regularization causes the failure modes to vanish. Finally, we extend double backpropagation over the full set of residuals, and use it to achieve state-of-the-art performance on four standard failure mode equations with up to 23× fewer collocation points and a vanilla architecture.
Nigel T. Andersen, Takashi Matsubara
Graduate School of Information Science and Technology Hokkaido University Sapporo, Japan · RIKEN Center for Advanced Intelligence Project (AIP) Tokyo, Japan
Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed neural networks avoid dependence on labeled data, yet their optimization can be basin-fragile: when the governing residual admits multiple solutions, a PINN trained from scratch may converge to a physically incorrect state despite achieving a small residual. We show that these failure modes can be addressed jointly: an imperfect NO provides the structural prior needed to place a PINN in the correct solution basin, while the PDE residual refines the solution beyond the operator's accuracy. We introduce a three-stage framework that freezes the spatial basis of a physics-informed NO, extrapolates its solution branch to an out-of-distribution parameter using a polynomial continuation prior, and distills the resulting field into a fresh PINN. The NO need not be accurate at the target; it transfers solution-branch information, while PDE residual minimization in the PINN governs convergence. We evaluate the framework on three nonlinear PDEs: 1D viscous Burgers, 2D steady Allen-Cahn near a pitchfork bifurcation, and 2D steady lid-driven cavity flow. For Allen-Cahn, where the trivial solution satisfies the PDE residual exactly, a standard PINN collapses to the trivial zero branch, whereas distillation from the crude extrapolated operator recovers the non-trivial branch that matches the finite-difference reference. For the lid-driven cavity, extrapolating to a Reynolds number of Re = 3200 accelerates convergence to the correct physical state, achieving competitive accuracy using fewer parameters and optimization steps than recent literature baselines. These results establish a simple principle: an NO need not accurately predict the solution to be useful; it only needs to identify the correct basin from which PINN optimization can recover it.
S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas +1
Sibley School of Mechanical and Aerospace Engineering, Cornell University · AI Lab, Siemens Energy · Department of Civil and Systems Engineering, Johns Hopkins University
Physics-informed neural networks (PINNs) often fail to accurately resolve partial differential equations (PDEs) with high-frequency or multi-scale solutions, as well as strongly nonlinear problems. Two factors underlie this difficulty: spectral bias, the tendency of neural networks to underfit high-frequency features; and representation-coefficient coupling, the entanglement of representation learning and coefficient fitting within a single nonconvex optimization objective. In this work, we propose the Fourier-enhanced alternating Levenberg--Marquardt PINN (FALM-PINN), an optimization framework that decouples representation learning from coefficient fitting. The upper-level problem learns a Fourier-enhanced basis that enriches the latent space with high-frequency components, while the lower-level problem resolves the coupling by fitting the projection coefficients on this basis, solving a nonlinear least-squares problem with the Levenberg--Marquardt algorithm. The framework applies to general nonlinear and coupled PDE systems, and reduces to a single-step convex optimization problem for linear PDEs. We prove global convergence of the alternating training scheme in both cases. Numerical examples on multiple challenging high-frequency and nonlinear PDEs show that FALM-PINN achieves relative L2 errors up to two orders of magnitude lower than state-of-the-art baselines.
Yulun Wu, Matthieu Barreau, Miguel Aguiar +1
Division of Decision and Control Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, 100 44, Sweden