Physics-informed neural networks (PINNs) offer a unified framework for solving both forward and inverse problems of differential equations, yet their performance and physical consistency strongly depend on how governing laws are incorporated. In this work, we present a systematic comparison of different thermodynamic structure-informed neural networks by incorporating various thermodynamics formulations, including Newtonian, Lagrangian, and Hamiltonian mechanics for conservative systems, as well as the Onsager variational principle and extended irreversible thermodynamics for dissipative systems. Through comprehensive numerical experiments on representative ordinary and partial differential equations, we quantitatively evaluate the impact of these formulations on accuracy, physical consistency, noise robustness, and interpretability. The results show that Newtonian-residual-based PINNs can reconstruct system states but fail to reliably recover key physical and thermodynamic quantities, whereas structure-preserving formulation significantly enhances parameter identification, thermodynamic consistency, and robustness. These findings provide practical guidance for principled design of thermodynamics-consistency model, and lay the groundwork for integrating more general nonequilibrium thermodynamic structures into physics-informed machine learning.
Figures & tables
Figure 1: The schematic diagram of the Lagrangian-mechanics, Hamiltonian-mechanics, Onsager’s Variational Principle and Extended Irreversible Thermodynamics informed neural networks.
Framework
Theoretical Foundation
Pre-Knowledge
Neural Network
Residue Loss
Description
NM-PINN
Newtonian Mechanics
u(x,t)
uθ(x,t)
∂t∂uθ−F[uθ]2
Conservative or Dissipative Dynamics
LM-PINN
Lagrangian Mechanics
L(q,q˙,t)
qθ(t)
dtd(∂q˙∂L)−∂q∂L2
Conservative Dynamics
HM-PINN
Hamiltonian Mechanics
H(q,p,t)
qθ(t)pθ(t)
dtdq−∂p∂H2+dtdp+∂q∂H2
Conservative Dynamics
OVP-PINN
Onsager’s Variational Principle
R[u,v]
uθ(x,t)vθ(x,t)
δvθδR2
Dissipative Dynamics
EIT-PINN
Extended Irreversible Thermodynamics
S[u,v]
uθ(x,t)vθ(x,t)
∂t∂S+∇⋅Js−σs2
Dissipative Dynamics
Table 1: Comparison among different thermodynamics-informed neural networks.
Figure 2: Results for the forward problem of ideal mass spring. The first column presents the predicted and true Hamiltonian, the second column shows the Lagrangian, the third column is the trajectory of the system, and the last column illustrates the phase diagram of q and p . Panels ( a-c ) correspond to the results obtained using the NM-PINNs, LM-PINNs, and HM-PINNs, respectively.
Figure 3: Results for the inverse problem of ideal mass spring and ideal pendulum. ( a ) For the ideal mass spring system, the learned distribution of k/m obtained from the three models after ten random initializations under different noise levels. The black dashed line represents the ground truth value of k/m . ( b ) Corresponding results for ideal pendulum systems.
Figure 4: Results for the forward problem of ideal pendulum. The first column presents the predicted and true Hamiltonian, the second column shows the Lagrangian, the third column is the trajectory of the system, and the last column illustrates the phase diagram of q and p . Panels ( a-c ) correspond to the results obtained using the NM-PINNs, LM-PINNs, and HM-PINNs, respectively.
Figure 5: Results for the forward problem of double pendulum. The first column presents the predicted and true Hamiltonian, the second column shows the Lagrangian, and the third column is the trajectory of the system. Panels ( a-c ) correspond to the results obtained using the NM-PINNs, LM-PINNs, and HM-PINNs, respectively.
Noise
Model
λ mean ± std
β mean ± std (no outliers)
NM-PINNs
0.2000±4.8021×10−5
3.1321±2.2490×10−5
0%
OVP-PINNs
0.2000±3.9388×10−5
3.1321±6.8259×10−6
EIT-PINNs
0.2000±9.3730×10−5
3.1273±6.2342×10−3
NM-PINNs
0.2002±1.9835×10−4
3.1322±1.3322×10−5
1%
OVP-PINNs
0.2001±3.5251×10−5
3.1324±9.3518×10−6
EIT-PINNs
0.2003±1.3328×10−4
3.1253±4.7730×10−3
Table 2: Unknown parameter prediction results of damped pendulum system under different noise intensities. Ten random initialization simulations were performed for each model, and all results are averaged over ten simulations. ”no outliers” excludes sign-flipped β values caused by the β2 -based residual loss.
Figure 6: Results for the forward problem of damped pendulum. The first column presents the predicted and true Rayleighian, the second column shows the entropy function S , the third column is the trajectory of the system, and the last column illustrates the phase diagram of q and p . Panels ( a-c ) correspond to the results obtained using the NM-PINNs, OVP-PINNs, and EIT-PINNs, respectively.
Figure 7: Results for the forward problem of diffusion equation. ( a ) The learned distribution of Dx from the three models after ten random initializations under different noise levels. ( b ) The reference entropy flux at t=0.5 . ( c-e ) Results obtained using NM-PINNs, OVP-PINNs, and EIT-PINNs, respectively. Each panel shows the point-wise absolute error at t=0.5 and t=1.0 (first two columns), the predicted normalized Rayleighian (third column), and the difference field between true and predicted entropy fluxes at t=0.5 (fourth column).
Figure 8: Results for the forward problem of Fisher-Kolmogorov equation. ( a ) Reference solution and ( b ) reference local entropy production rate for F-K equation. ( c-e ) Results obtained using NM-PINNs, OVP-PINNs, and EIT-PINNs, respectively. Each panel shows the point-wise absolute error of the predicted solution u (first column), the predicted normalized Rayleighian (second column), and the predicted local entropy production rate (fourth column).
Figure 9: Loss Landscape of ideal mass-spring system and diffusion equation. ( a ) Loss Landscape of ideal mass-spring system, shown from left to right for NM-PINNs, LM-PINNs, and HM-PINNs. ( b ) Loss Landscape of diffusion equation, shown from left to right for NM-PINNs, OVP-PINNs, and EIT-PINNs.
Entropy production governs irreversibility and uncertainty in both physical and information-theoretic systems. While Physics-Informed Neural Networks (PINNs) successfully solve differential equations, current architectures remain inherently domain-specific. The extraction of domain-invariant entropy representations across fundamentally different physical laws remains unexplored. This paper introduces a unified Physics-Informed Deep Learning (PIDL) framework that simultaneously enforces differential equation residuals and information-theoretic bounds within a single neural architecture. We demonstrate this framework via two canonical studies: (i) a thermodynamic continuous stirred-tank reactor (CSTR) model solving governing ODEs, where a Softplus constraint strictly enforces the Second Law of Thermodynamics; and (ii) an information-theoretic financial market model solving the inverse Fokker-Planck PDE to infer latent drift and diffusion coefficients, guaranteeing diffusion positivity via a Softplus constraint while naturally inducing Shannon entropy. Three model variants are evaluated: two domain-specific baselines and one shared-encoder architecture. The PIDL framework guarantees absolute thermodynamic admissibility with zero Second-Law violations and exhibits exceptional data efficiency, retaining >90% predictive accuracy using merely 30% of available training data. Furthermore, a post-hoc Ruppeiner Riemannian geometric analysis of the learned entropy surface successfully identifies thermodynamic phase instabilities. This methodology provides a robust, domain-agnostic architecture for physics-constrained entropy modeling, advancing applications in sustainable process design and quantitative financial risk assessment.
Biswajeet Sahoo, Debadutta Patra
Durham University, Durham, UK · Department of Chemical Engineering Veer Surendra Sai University of Technology, Burla, India
High-dimensional transient heat diffusion under noisy boundary conditions exposes a fundamental limitation of classical numerical methods: accuracy degrades catastrophically where physical noise is unavoidable. This paper presents a Physics-Informed Neural Network (PINN) framework as a systematic solution to this problem across one, two, and three spatial dimensions, establishing clear operational regimes that redefine solver selection in noisy thermal systems. Under 20% boundary noise in 3D, PINN sustains approximately 91% accuracy while Finite Difference Method (FDM) collapses to 36%, a clear decisive advantage. This is further confirmed in a physical copper thermal system, where PINN reduces boundary reconstruction error by 3.3 times under realistic noise conditions. This noise resilience is accompanied by a dimensionality-driven efficiency crossover: PINN requires fewer spacetime nodes than FDM in 3D while achieving superior accuracy, exposing the true cost of classical discretization at scale. These findings reframe solver selection: the decisive axis is not accuracy alone, but noise exposure and dimensionality jointly. When noise and dimensionality are both high, the classical solver paradigm is insufficient; this work provides the foundation to justify PINN as the operational standard in such regimes.
Shreesh Bhattarai, Harish Chandra Bhandari
Department of Mathematics, Kathmandu University Dhulikhel, Nepal
Physics-Informed Neural Networks (PINNs) are a common class of machine learning-based partial differential equation (PDE) solvers which train a network to represent a solution by minimizing a residual loss that encodes the PDE. Despite their successes, they are known to fail on certain simple equations, converging to an incorrect solution despite low loss. These failure modes have garnered significant attention in the literature over the past several years, motivating both architectural and optimization based solutions. By directly visualizing the residual, we show that failure modes are the result of overfitting: the loss is minimized on the collocation points, but not elsewhere. Applying regularization causes the failure modes to vanish. Finally, we extend double backpropagation over the full set of residuals, and use it to achieve state-of-the-art performance on four standard failure mode equations with up to 23× fewer collocation points and a vanilla architecture.
Nigel T. Andersen, Takashi Matsubara
Graduate School of Information Science and Technology Hokkaido University Sapporo, Japan · RIKEN Center for Advanced Intelligence Project (AIP) Tokyo, Japan