A Comparative Investigation of Thermodynamic Structure-Informed Neural Networks

Guojie Li; Liu Hong

cs.LGMar 26, 2026

A Comparative Investigation of Thermodynamic Structure-Informed Neural Networks

Authors: Guojie Li, Liu Hong

Organizations: School of Mathematics, Sun Yat-sen University, Guangzhou, 510275, China.

Abstract

Physics-informed neural networks (PINNs) offer a unified framework for solving both forward and inverse problems of differential equations, yet their performance and physical consistency strongly depend on how governing laws are incorporated. In this work, we present a systematic comparison of different thermodynamic structure-informed neural networks by incorporating various thermodynamics formulations, including Newtonian, Lagrangian, and Hamiltonian mechanics for conservative systems, as well as the Onsager variational principle and extended irreversible thermodynamics for dissipative systems. Through comprehensive numerical experiments on representative ordinary and partial differential equations, we quantitatively evaluate the impact of these formulations on accuracy, physical consistency, noise robustness, and interpretability. The results show that Newtonian-residual-based PINNs can reconstruct system states but fail to reliably recover key physical and thermodynamic quantities, whereas structure-preserving formulation significantly enhances parameter identification, thermodynamic consistency, and robustness. These findings provide practical guidance for principled design of thermodynamics-consistency model, and lay the groundwork for integrating more general nonequilibrium thermodynamic structures into physics-informed machine learning.

Figures & tables

Figure 1: The schematic diagram of the Lagrangian-mechanics, Hamiltonian-mechanics, Onsager’s Variational Principle and Extended Irreversible Thermodynamics informed neural networks.

Framework	Theoretical Foundation	Pre-Knowledge	Neural Network	Residue Loss	Description
NM-PINN	Newtonian Mechanics	$u(x,t)$	$u_{\theta}(x,t)$	$\left\lvert\dfrac{\partial u_{\theta}}{\partial t}-F[u_{\theta}]\right\rvert^{2}$	Conservative or Dissipative Dynamics
LM-PINN	Lagrangian Mechanics	$L(q,\dot{q},t)$	$q_{\theta}(t)$	$\left\lvert\dfrac{d}{dt}\!\left(\dfrac{\partial L}{\partial\dot{q}}\right)-\dfrac{\partial L}{\partial q}\right\rvert^{2}$	Conservative Dynamics
HM-PINN	Hamiltonian Mechanics	$H(q,p,t)$	$q_{\theta}(t)$ $p_{\theta}(t)$	$\left\lvert\dfrac{dq}{dt}-\dfrac{\partial H}{\partial p}\right\rvert^{2}+\left\lvert\dfrac{dp}{dt}+\dfrac{\partial H}{\partial q}\right\rvert^{2}$	Conservative Dynamics
OVP-PINN	Onsager’s Variational Principle	$R[u,v]$	$u_{\theta}(x,t)$ $v_{\theta}(x,t)$	$\left\lvert\dfrac{\delta R}{\delta v_{\theta}}\right\rvert^{2}$	Dissipative Dynamics
EIT-PINN	Extended Irreversible Thermodynamics	$S[u,v]$	$u_{\theta}(x,t)$ $v_{\theta}(x,t)$	$\left\lvert\dfrac{\partial S}{\partial t}+\nabla\cdot J_{s}-\sigma_{s}\right\rvert^{2}$	Dissipative Dynamics

Table 1: Comparison among different thermodynamics-informed neural networks.

Figure 2: Results for the forward problem of ideal mass spring. The first column presents the predicted and true Hamiltonian, the second column shows the Lagrangian, the third column is the trajectory of the system, and the last column illustrates the phase diagram of $q$ and $p$ . Panels ( a-c ) correspond to the results obtained using the NM-PINNs, LM-PINNs, and HM-PINNs, respectively.

Figure 3: Results for the inverse problem of ideal mass spring and ideal pendulum. ( a ) For the ideal mass spring system, the learned distribution of $k/m$ obtained from the three models after ten random initializations under different noise levels. The black dashed line represents the ground truth value of $k/m$ . ( b ) Corresponding results for ideal pendulum systems.

Figure 4: Results for the forward problem of ideal pendulum. The first column presents the predicted and true Hamiltonian, the second column shows the Lagrangian, the third column is the trajectory of the system, and the last column illustrates the phase diagram of $q$ and $p$ . Panels ( a-c ) correspond to the results obtained using the NM-PINNs, LM-PINNs, and HM-PINNs, respectively.

Figure 5: Results for the forward problem of double pendulum. The first column presents the predicted and true Hamiltonian, the second column shows the Lagrangian, and the third column is the trajectory of the system. Panels ( a-c ) correspond to the results obtained using the NM-PINNs, LM-PINNs, and HM-PINNs, respectively.

Noise	Model	$\lambda$ mean $\pm$ std	$\beta$ mean $\pm$ std (no outliers)
	NM-PINNs	$0.2000\pm 4.8021\times 10^{-5}$	$3.1321\pm 2.2490\times 10^{-5}$
0%	OVP-PINNs	$0.2000\pm 3.9388\times 10^{-5}$	$3.1321\pm 6.8259\times 10^{-6}$
	EIT-PINNs	$0.2000\pm 9.3730\times 10^{-5}$	$3.1273\pm 6.2342\times 10^{-3}$
	NM-PINNs	$0.2002\pm 1.9835\times 10^{-4}$	$3.1322\pm 1.3322\times 10^{-5}$
1%	OVP-PINNs	$0.2001\pm 3.5251\times 10^{-5}$	$3.1324\pm 9.3518\times 10^{-6}$
	EIT-PINNs	$0.2003\pm 1.3328\times 10^{-4}$	$3.1253\pm 4.7730\times 10^{-3}$

Table 2: Unknown parameter prediction results of damped pendulum system under different noise intensities. Ten random initialization simulations were performed for each model, and all results are averaged over ten simulations. ”no outliers” excludes sign-flipped

\beta

values caused by the

\beta^{2}

-based residual loss.

Figure 6: Results for the forward problem of damped pendulum. The first column presents the predicted and true Rayleighian, the second column shows the entropy function $S$ , the third column is the trajectory of the system, and the last column illustrates the phase diagram of $q$ and $p$ . Panels ( a-c ) correspond to the results obtained using the NM-PINNs, OVP-PINNs, and EIT-PINNs, respectively.

Figure 7: Results for the forward problem of diffusion equation. ( a ) The learned distribution of $D_{x}$ from the three models after ten random initializations under different noise levels. ( b ) The reference entropy flux at $t=0.5$ . ( c-e ) Results obtained using NM-PINNs, OVP-PINNs, and EIT-PINNs, respectively. Each panel shows the point-wise absolute error at $t=0.5$ and $t=1.0$ (first two columns), the predicted normalized Rayleighian (third column), and the difference field between true and predicted entropy fluxes at $t=0.5$ (fourth column).

Figure 8: Results for the forward problem of Fisher-Kolmogorov equation. ( a ) Reference solution and ( b ) reference local entropy production rate for F-K equation. ( c-e ) Results obtained using NM-PINNs, OVP-PINNs, and EIT-PINNs, respectively. Each panel shows the point-wise absolute error of the predicted solution $u$ (first column), the predicted normalized Rayleighian (second column), and the predicted local entropy production rate (fourth column).

Figure 9: Loss Landscape of ideal mass-spring system and diffusion equation. ( a ) Loss Landscape of ideal mass-spring system, shown from left to right for NM-PINNs, LM-PINNs, and HM-PINNs. ( b ) Loss Landscape of diffusion equation, shown from left to right for NM-PINNs, OVP-PINNs, and EIT-PINNs.

Physics-Informed ML

Physics-informed machine learning combines data-driven models with physical laws to solve, predict, and infer scientific systems.

See all 819 papers

Physics-Informed Deep Learning for Entropy Prediction in Heterogeneous Systems: Thermodynamic and Information-Theoretic Case Studies
May 31, 2026Biswajeet Sahoo, Debadutta PatraPhysics-Informed ML
Durham University, Durham, UK · Department of Chemical Engineering Veer Surendra Sai University of Technology, Burla, India
Overcoming the Limits of Finite Difference Method; Physics-Informed Neural Network for Noisy High-Dimensional Heat Diffusion
Jun 6, 2026Shreesh Bhattarai, Harish Chandra BhandariPDE Solving Physics-Informed ML
Department of Mathematics, Kathmandu University Dhulikhel, Nepal
PINNs Failure Modes are Overfitting
May 29, 2026Nigel T. Andersen, Takashi MatsubaraNeural Network Generalization Neural PDE Solvers
Graduate School of Information Science and Technology Hokkaido University Sapporo, Japan · RIKEN Center for Advanced Intelligence Project (AIP) Tokyo, Japan

May 31, 2026J/K move · Enter open · S save

Physics-Informed Deep Learning for Entropy Prediction in Heterogeneous Systems: Thermodynamic and Information-Theoretic Case Studies

Biswajeet Sahoo, Debadutta Patra

Durham University, Durham, UK · Department of Chemical Engineering Veer Surendra Sai University of Technology, Burla, India

Physics-Informed ML

Open paper AskPDF arXiv

Entropy production governs irreversibility and uncertainty in both physical and information-theoretic systems. While Physics-Informed Neural Networks (PINNs) successfully solve differential equations, current architectures remain inherently domain-specific. The extraction of domain-invariant entropy representations across fundamentally different physical laws remains unexplored. This paper introduces a unified Physics-Informed Deep Learning (PIDL) framework that simultaneously enforces differential equation residuals and information-theoretic bounds within a single neural architecture. We demonstrate this framework via two canonical studies: (i) a thermodynamic continuous stirred-tank reactor (CSTR) model solving governing ODEs, where a Softplus constraint strictly enforces the Second Law of Thermodynamics; and (ii) an information-theoretic financial market model solving the inverse Fokker-Planck PDE to infer latent drift and diffusion coefficients, guaranteeing diffusion positivity via a Softplus constraint while naturally inducing Shannon entropy. Three model variants are evaluated: two domain-specific baselines and one shared-encoder architecture. The PIDL framework guarantees absolute thermodynamic admissibility with zero Second-Law violations and exhibits exceptional data efficiency, retaining >90% predictive accuracy using merely 30% of available training data. Furthermore, a post-hoc Ruppeiner Riemannian geometric analysis of the learned entropy surface successfully identifies thermodynamic phase instabilities. This methodology provides a robust, domain-agnostic architecture for physics-constrained entropy modeling, advancing applications in sustainable process design and quantitative financial risk assessment.

Abstract

Physics-Informed ML

Explore similar work

Physics-Informed Deep Learning for Entropy Prediction in Heterogeneous Systems: Thermodynamic and Information-Theoretic Case Studies

Overcoming the Limits of Finite Difference Method; Physics-Informed Neural Network for Noisy High-Dimensional Heat Diffusion

PINNs Failure Modes are Overfitting