Boosted Physics-Informed Neural Networks for Solving Homogeneous and Non-Homogeneous Lane-Emden Type Equations
Organizations: Department of Mathematics and Applied Mathematics, University of Johannesburg, Johannesburg, South Africa
Abstract
The Lane-Emden Equation is a class of nonlinear differential equations commonly used to model phenomena in astrophysics and mathematical physics, such as stellar structure and thermal behavior in polytropic gas spheres. This study presents a novel hybrid framework that integrates Physics-Informed Neural Networks (PINNs) with boosting methods to enhance the accuracy and robustness of solutions for Lane-Emden type differential equations. Standard PINNs leverage physical laws encoded within the loss function, but their predictive performance can be constrained by optimization challenges and approximation errors. To address this, we introduce a post-processing boosting stage where the residual errors from the PINN prediction are modeled using some regression algorithms - including LightGBM, Random Forest, XGBoost, and Support Vector Regression (SVR). The outputs from these boosted models are aggregated to generate a corrected, high-fidelity prediction. The proposed methodology is systematically validated through quantitative metrics and comparative analysis against the standard PINN baseline. The results demonstrate that the boosted PINN framework significantly reduces prediction error, and improves generalization, offering a powerful and flexible approach for solving scientific and engineering problems governed by differential equations.
Figures & tables
| Items | Options | Prob1 | Prob2 | Prob3 | Prob4 | Prob5 | Prob6 |
| Number of hidden layers | 2, 3, 4 | 3 | 2 | 4 | 3 | 4 | 4 |
| Number of neurons per layer | 4, 8, 16, 32, 64 | 32 | 64 | 64 | 16 | 16 | 32 |
| Activation Function | ReLU, Tanh, SiLU, Sigmoid, Elu, Softplus | Tanh | Sigmoid | SiLU | Softplus | Sigmoid | Sigmoid |
| Optimizer | Adam, Adagrad, SGD, RMSprop, AdamW, Adamax, NAdam | Adamax | Adamax | Adamax | Adam | AdamW | Adam |
| Learning rate | 0.01, 0.001, 0.0001, 0.0001 | 0.001 | 0.01 | 0.01 | 0.01 | 0.001 | 0.001 |
| Algorithm | Type | Role in Boosted PINN Framework |
| Random Forest (RF) | Ensemble (Bagging) | Captures non-linear patterns and interactions in PINN residuals through decorrelated decision trees, reducing variance and mitigating overfitting. |
| Extreme Gradient Boosting (XGBoost) | Ensemble (Boosting) | Sequentially corrects PINN errors with regularization, using second-order gradients for precise residual modeling while controlling model complexity. |
| Light Gradient Boosting Machine (LightGBM) | Ensemble (Boosting) | Efficiently models PINN residuals using histogram-based algorithms and leaf-wise growth, enabling fast training on error datasets with many sample points. |
| Support Vector Regression (SVR) | Kernel Method | Provides smooth interpolation of PINN error surfaces using -insensitive loss and kernel functions, effective for capturing global error trends. |
| x | PINN | MLP ANN [ 13 ] | ChNN [ 6 ] | PS-SQP [ 9 ] | GA-SQP [ 9 ] |
| 0.1 | 4.17e-05 | 1.54e-04 | 2.00e-04 | 6.25e-05 | 9.28e-06 |
| 0.2 | 3.88e-05 | 4.22e-04 | 1.00e-04 | 1.06e-04 | 7.54e-05 |
| 0.3 | 3.37e-05 | 6.04e-04 | 4.60e-03 | 1.13e-04 | 3.38e-04 |
| 0.4 | 2.86e-05 | 6.03e-04 | 3.20e-03 | 8.25e-04 | 1.02e-03 |
| 0.5 | 2.74e-05 | 4.12e-04 | 7.60e-03 | 2.28e-03 | 2.41e-03 |
| 0.6 | 3.45e-05 | 9.20e-05 | 3.80e-03 | 4.77e-03 | 4.97e-03 |
| x | PINN | LightGBM-Boosted | RF-Boosted | XGBoost-Boosted | SVR-Boosted |
| 0.1 | 4.17e-05 | 1.31e-07 | 7.11e-08 | 8.40e-06 | 2.52e-05 |
| 0.2 | 3.88e-05 | 2.73e-07 | 5.29e-08 | 1.13e-05 | 2.81e-05 |
| 0.3 | 3.37e-05 | 2.61e-07 | 1.23e-07 | 1.64e-05 | 3.32e-05 |
| 0.4 | 2.86e-05 | 1.00e-07 | 6.58e-07 | 2.15e-05 | 3.83e-05 |
| 0.5 | 2.74e-05 | 7.06e-08 | 2.19e-07 | 2.28e-05 | 3.95e-05 |
| 0.6 | 3.45e-05 | 5.49e-07 | 2.46e-07 | 1.57e-05 | 3.25e-05 |
| x | PINNs | MLP ANN [ 13 ] | ChNN [ 6 ] | [ 10 ] | [ 10 ] |
| 0.1 | 2.83e-04 | 2.58e-04 | 7.00e-03 | 1.99e-04 | 1.28e-05 |
| 0.2 | 2.45e-04 | 2.67e-04 | 1.30e-02 | 4.13e-04 | 6.16e-04 |
| 0.3 | 9.54e-06 | 7.00e-05 | 3.00e-03 | 6.53e-04 | 7.64e-04 |
| 0.4 | 2.11e-04 | 3.00e-05 | 1.34e-01 | 9.35e-04 | 2.19e-06 |
| 0.5 | 1.68e-04 | 3.00e-06 | 2.60e-03 | 1.28e-03 | 1.65e-03 |
| 0.6 | 7.26e-05 | 2.00e-05 | 2.10e-03 | 1.71e-03 | 3.36e-03 |
| x | PINN | LightGBM-Boosted | RF-Boosted | XGBoost-Boosted | SVR-Boosted |
| 0.1 | 2.83e-04 | 5.04e-05 | 1.14e-06 | 1.58e-04 | 2.44e-04 |
| 0.2 | 2.45e-04 | 5.21e-06 | 4.77e-06 | 1.21e-04 | 2.06e-04 |
| 0.3 | 9.54e-06 | 1.37e-05 | 8.38e-06 | 3.74e-05 | 4.82e-05 |
| 0.4 | 2.11e-04 | 2.46e-07 | 4.19e-07 | 1.12e-04 | 2.50e-04 |
| 0.5 | 1.68e-04 | 6.03e-06 | 5.78e-06 | 6.95e-05 | 2.07e-04 |
| 0.6 | 7.26e-05 | 2.84e-06 | 4.58e-06 | 2.65e-05 | 1.11e-04 |
| Input data | PINNs | ChNN [ 6 ] | [ 10 ] | [ 10 ] |
| 0.1 | 4.98e-04 | 7.0e-04 | 2.22e-04 | 1.47e-04 |
| 0.2 | 5.46e-04 | 1.0e-03 | 4.96e-04 | 2.99e-04 |
| 0.3 | 6.51e-04 | 6.0e-04 | 8.19e-04 | 5.01e-04 |
| 0.4 | 7.86e-04 | 2.6e-03 | 1.19e-03 | 5.96e-04 |
| 0.5 | 9.49e-04 | 7.4e-03 | 1.60e-03 | 6.55e-04 |
| 0.6 | 1.14e-03 | 4.8e-03 | 2.07e-03 | 8.69e-04 |
| x | PINN | LightGBM-Boosted | RF-Boosted | XGBoost-Boosted | SVR-Boosted |
| 0.1 | 4.98e-04 | 3.77e-06 | 6.13e-07 | 1.41e-04 | 6.20e-04 |
| 0.2 | 5.46e-04 | 3.84e-06 | 7.85e-07 | 9.32e-05 | 5.71e-04 |
| 0.3 | 6.51e-04 | 6.05e-06 | 2.73e-06 | 2.57e-05 | 4.66e-04 |
| 0.4 | 7.86e-04 | 2.41e-08 | 4.90e-07 | 1.25e-05 | 3.31e-04 |
| 0.5 | 9.49e-04 | 9.04e-06 | 2.59e-06 | 8.34e-07 | 1.68e-04 |
| 0.6 | 1.14e-03 | 1.06e-05 | 7.63e-07 | 2.04e-05 | 2.86e-05 |
| x | PINNs | MLP ANN [ 13 ] | ChNN [ 25 ] | [ 10 ] | [ 10 ] |
| 0.1 | 3.32e-05 | 1.1e-05 | 3.1e-04 | 2.49e-05 | 9.86e-06 |
| 0.2 | 5.12e-05 | 1.0e-06 | 5.9e-04 | 8.47e-05 | 2.22e-05 |
| 0.3 | 6.63e-05 | 3.3e-05 | 3.3e-04 | 1.58e-04 | 1.09e-05 |
| 0.4 | 3.18e-05 | 4.8e-05 | 1.2e-05 | 2.22e-04 | 1.50e-05 |
| 0.5 | 4.66e-05 | 2.3e-05 | 6.8e-04 | 2.57e-04 | 3.62e-05 |
| 0.6 | 7.51e-05 | 2.7e-05 | 2.8e-05 | 2.41e-04 | 6.62e-06 |
| x | PINN | LightGBM-Boosted | RF-Boosted | XGBoost-Boosted | SVR-Boosted |
| 0.1 | 3.32e-05 | 9.06e-06 | 4.41e-08 | 5.66e-06 | 5.38e-05 |
| 0.2 | 5.12e-05 | 2.24e-07 | 1.30e-06 | 2.37e-05 | 7.18e-05 |
| 0.3 | 6.63e-05 | 9.21e-07 | 9.96e-07 | 3.87e-05 | 8.69e-05 |
| 0.4 | 3.18e-05 | 2.34e-07 | 1.71e-07 | 4.30e-06 | 5.25e-05 |
| 0.5 | 4.66e-05 | 2.30e-06 | 2.89e-08 | 1.91e-05 | 6.72e-05 |
| 0.6 | 7.51e-05 | 7.63e-07 | 1.23e-06 | 4.76e-05 | 9.57e-05 |
| Input data | PINNs | ChNN [ 25 ] | [ 10 ] | [ 10 ] | LWM [ 27 ] |
| 0.1 | 1.26e-03 | 1.00e-06 | 1.95e-04 | 8.20e-05 | 4.89e-06 |
| 0.2 | 1.18e-03 | 1.00e-05 | 3.78e-04 | 9.49e-05 | 6.84e-06 |
| 0.3 | 1.19e-03 | 1.00e-06 | 5.33e-04 | 1.35e-04 | 8.03e-07 |
| 0.4 | 1.34e-03 | 9.90e-05 | 6.51e-04 | 1.82e-04 | 8.38e-06 |
| 0.5 | 1.57e-03 | 1.00e-04 | 7.25e-04 | 2.12e-04 | 1.28e-05 |
| 0.6 | 1.71e-03 | 8.70e-07 | 7.55e-04 | 2.47e-04 | 5.32e-05 |
| x | PINN | LightGBM-Boosted | RF-Boosted | XGBoost-Boosted | SVR-Boosted |
| 0.1 | 1.26e-03 | 2.47e-05 | 3.96e-08 | 2.81e-05 | 6.61e-05 |
| 0.2 | 1.18e-03 | 1.06e-06 | 2.14e-06 | 1.04e-04 | 1.42e-04 |
| 0.3 | 1.19e-03 | 2.80e-06 | 4.41e-06 | 9.29e-05 | 1.30e-04 |
| 0.4 | 1.34e-03 | 9.98e-06 | 1.33e-06 | 3.83e-05 | 1.86e-05 |
| 0.5 | 1.57e-03 | 3.34e-07 | 6.09e-06 | 9.73e-05 | 2.40e-04 |
| 0.6 | 1.71e-03 | 1.92e-06 | 6.81e-06 | 2.38e-04 | 3.81e-04 |
| Input data | PINNs | [ 10 ] | [ 10 ] | LWM [ 27 ] |
| 0.1 | 1.80e-05 | 3.90e-04 | 2.04e-04 | 1.57e-04 |
| 0.2 | 5.15e-05 | 7.58e-04 | 4.59e-04 | 7.73e-05 |
| 0.3 | 6.23e-06 | 1.07e-03 | 7.26e-04 | 1.26e-04 |
| 0.4 | 1.06e-05 | 1.33e-03 | 9.84e-04 | 4.61e-05 |
| 0.5 | 4.10e-05 | 1.53e-03 | 1.21e-03 | 2.03e-04 |
| 0.6 | 4.59e-06 | 1.66e-03 | 1.31e-03 | 3.85e-04 |
| x | PINN | LightGBM-Boosted | RF-Boosted | XGBoost-Boosted | SVR-Boosted |
| 0.1 | 1.80e-05 | 6.39e-06 | 6.39e-07 | 8.64e-07 | 1.76e-06 |
| 0.2 | 5.15e-05 | 2.05e-06 | 1.12e-06 | 3.43e-05 | 3.17e-05 |
| 0.3 | 6.23e-06 | 1.60e-06 | 7.89e-07 | 1.10e-05 | 1.36e-05 |
| 0.4 | 1.06e-05 | 6.28e-07 | 5.67e-07 | 6.56e-06 | 9.16e-06 |
| 0.5 | 4.10e-05 | 1.04e-06 | 1.28e-06 | 2.38e-05 | 2.12e-05 |
| 0.6 | 4.59e-06 | 5.78e-07 | 3.68e-07 | 2.18e-05 | 2.44e-05 |