Physics-Informed Neural Networks for Methane Sorption: Cross-Gas Transfer Learning, Ensemble Collapse Under Physics Constraints, and Monte Carlo Dropout Uncertainty Quantification
Authors: Mohammad Nooraiepour, Zezhang Song, Wei Li, Sarah Perez
Organizations: Environmental Geosciences, Department of Geosciences, University of Oslo, P.O. Box 1047 Blindern, 0316 Oslo, Norway. · National Key Laboratory of Petroleum Resources and Engineering, China University of Petroleum (Beijing), China · College of Geosciences, China University of Petroleum (Beijing), China · State Key Laboratory of Coal Mine Disaster Prevention and Control, China University of Mining and Technology, Xuzhou 221116, China · The Lyell Centre, Heriot-Watt University, Edinburgh, UK · Subsurface Energy Transition and Innovation Centre, Heriot-Watt University, Edinburgh, UK
Accurate methane sorption prediction across heterogeneous coal ranks requires models that combine thermodynamic consistency, efficient knowledge transfer across data-scarce geological systems, and calibrated uncertainty estimates, capabilities that are rarely addressed together in existing frameworks. We present a physics-informed transfer learning framework that adapts a hydrogen sorption PINN to methane sorption prediction via Elastic Weight Consolidation, coal-specific feature engineering, and a three-phase curriculum that progressively balances transfer preservation with thermodynamic fine-tuning. Trained on 993 equilibrium measurements from 114 independent coal experiments spanning lignite to anthracite, the framework achieves R2 = 0.932 on held-out coal samples, a 227% improvement over pressure-only classical isotherms, while hydrogen pre-training delivers 18.9% lower RMSE and 19.4% faster convergence than random initialization. Five Bayesian uncertainty quantification approaches reveal a systematic divergence in performance across physics-constrained architectures. Monte Carlo Dropout achieves well-calibrated uncertainty at minimal overhead, while deep ensembles, regardless of architectural diversity or initialization strategy, exhibit performance degradation because shared physics constraints narrow the admissible solution manifold. SHAP and ALE analyses confirm that learned representations remain physically interpretable and aligned with established coal sorption mechanisms: moisture-volatile interactions are most influential, pressure-temperature coupling captures thermodynamic co-dependence, and features exhibit non-monotonic effects. These results identify Monte Carlo Dropout as the best-performing UQ method in this physics-constrained transfer learning framework, and demonstrate cross-gas transfer learning as a data-efficient strategy for geological material modeling.
Figures & tables
Feature
Unit
Mean
Std Dev
Median
IQR
Range
Physical Role
Temperature
K
301.5
3.9
303.2
5.0
293.2–323.2
Thermodynamic driving force
Pressure
MPa
2.85
1.97
2.66
3.03
0.002–9.26
Adsorption potential
Moisture
wt.%
1.97
1.95
1.29
1.98
0.00–10.68
Pore blockage, site competition
Ash content
wt.%
13.3
6.5
12.2
8.71
2.82–30.03
Inert mineral dilution
Volatile matter
wt.%
18.4
10.2
16.8
17.87
4.96–40.52
Organic content proxy
Target variable:
Table 1: Statistical characteristics of the methane sorption dataset (993 measurements from 114 experiments). The dataset captures the full range of coal ranks under reservoir-relevant thermodynamic conditions. All temperatures are reported in Kelvin.
Coal Rank
V daf Range (%)
No. of Experiments
Experimental Method
Method Count
Anthracite
≤ 10
37
Gravimetric
4
Bituminous coal (low-volatile)
10–20
29
Manometric
110
Bituminous coal (medium-volatile)
20–28
21
Bituminous coal (high-volatile)
28–37
19
Lignite
≥ 37
8
Total
114
114
Table 2: Coal rank classification and experimental methodology distribution. Coal types categorized by volatile matter content (V daf , dry ash-free basis).
Feature
Correlation ( r )
P -value
Interpretation
Pressure
+0.483
p<0.001
Strong positive (thermodynamic driver)
Volatile matter
− 0.371
p<0.001
Moderate negative (rank effect)
Ash content
− 0.258
p<0.001
Moderate negative (dilution effect)
Moisture
− 0.206
p<0.001
Weak negative (site blockage)
Temperature
− 0.009
0.803
Non-significant (narrow range)
Table 3: Pearson correlations between input features and methane adsorption capacity. Significant correlations ( p<0.05 ) shown in bold.
Figure 1: Pairwise relationships reveal nonlinear compositional controls on methane adsorption capacity. Diagonal: kernel density estimates. Off-diagonal: scatter plots with linear regression lines. Key Pearson correlations with adsorption capacity: r=0.48 (pressure, p<10−47 ), r=−0.37 (volatile matter, p<10−27 ), r=−0.21 (moisture, p<10−9 ), r=−0.009 (temperature, p=0.80 ). Pressure explains only 23% of adsorption variance ( r2≈0.23 ), and heteroscedastic spread in the pressure–adsorption scatter and distinct rank-stratified subpopulations in the volatile-matter–adsorption scatter confirm that compositional heterogeneity dominates sorption variability and necessitates a flexible, nonlinear modeling architecture.
Figure 2: Physics-informed engineered features span physically meaningful ranges and encode domain-specific thermodynamic and compositional structure. Histograms with kernel density estimates. (A) Reduced temperature Tr=T/Tc and (B) reduced pressure Pr=P/Pc locate each experiment relative to the methane critical point, enabling dimensionless thermodynamic representation. (C) Fixed carbon FC =100− Moisture − Ash − VM concentrates in the 40–80 wt.% range characteristic of the bituminous coal majority. (D) Organic matter OM =(VM+FC)/100 directly quantifies the sorption-active fraction. (E) Thermodynamic coupling β=1/(RT) varies smoothly across the temperature range, encoding the Arrhenius energy scale. (F) Pressure-temperature product P×T captures joint thermodynamic state dependence consistent with the Gibbs adsorption equation. The physically sensible distributions of all engineered features confirm their suitability as network inputs.
Figure 3: Robust scaling and log-transformation satisfy the statistical assumptions of the heteroscedastic data loss. (A–C) Robust scaling (median/IQR normalization) mitigates the influence of geological outliers while preserving distribution shape, ensuring numerical stability for heavy-tailed geochemical data. (D) The raw adsorption target is strongly right-skewed (skewness =1.15 ). (E–F) The log-transformation y~=log(y+1) reduces skewness to near-zero and stabilizes variance across the three-order-of-magnitude adsorption range, directly justifying the Gaussian likelihood assumption in Ldata and preventing the loss function from being dominated by high-capacity outliers.
Figure 4: Classical pressure-only isotherms reach a universal performance ceiling of R 2≈0.29 , quantifying the dominant contribution of compositional heterogeneity to sorption variability. Actual vs. predicted values on training and 20% held-out test sets; dashed line: perfect prediction ( y=x ). (A) Langmuir ( R2=0.254 ). (B) Freundlich ( R2=0.247 ). (C) Sips ( R2=0.252 ). (D) Three-model ensemble average ( R2=0.285 ). All models explain only 25–28% of variance despite encoding correct thermodynamic pressure-adsorption structure, and the large uniform prediction scatter persists across the full pressure range. This ceiling is not a failure of classical isotherm theory but a quantitative demonstration that compositional heterogeneity—not pressure-adsorption physics—is the dominant source of sorption uncertainty in heterogeneous coal datasets, establishing the performance target that multivariate PINN modeling must surpass.
Figure 5: Within-rank stratification progressively improves isotherm performance, confirming that compositional heterogeneity between strata—not thermodynamic model limitations—drives the global performance ceiling. Langmuir fits stratified by volatile matter content. (A) Maximum adsorption capacity qmax increases monotonically with coal rank: high-rank 26.35 m 3 /t, medium-rank 22.08 m 3 /t, low-rank 17.16 m 3 /t, consistent with rank-controlled micropore development. (B) Within-rank R 2 : high-rank 0.289 (+14%), medium-rank 0.371 (+46%), low-rank 0.487 (+91%) versus the global R 2=0.254 . The monotonic improvement with compositional homogeneity confirms compositional heterogeneity as the primary driver of low global performance. Yet even within narrow strata, 51–71% of variance remains unexplained, demonstrating that volatile matter alone cannot substitute for full multivariate characterization. (C) Sample distribution: high-rank 47%, medium-rank 36%, low-rank 16%.
Figure 6: Three-phase training curriculum achieves stable, non-overfitting convergence in 1,129 epochs, with Phase 1 warmup providing the dominant performance gain. (A) Total training and validation loss on symmetric log scale; phases color-coded as warmup (orange), fine-tuning (red), full optimization (green). (B) Validation R 2 (log space) exceeds 0.90 at epoch 340 and stabilizes above 0.94 after epoch 900 (green zone). (C) Validation RMSE decreasing from 0.67 to 0.153. (D) Learning rate schedule: discrete phase drops at epochs 500 and 900, with ReduceLROnPlateau adaptive decay within phases. (E) Instantaneous learning speed (negative loss gradient magnitude). (F) R 2 acceleration: maximum gain concentrated in mid-warmup (epochs 200–400), confirming the value of frozen-encoder feature alignment. (G) Generalization gap (train R 2 − val R 2 ) remains below 2% throughout all phases, confirming the absence of overfitting. (H) Phase-specific R 2 contributions: warmup +0.817 , fine-tuning +0.005 , full optimization +0.032 ; no phase is redundant.
Figure 7: The transfer-learned PINN achieves R 2=0.932 on held-out coals with near-Gaussian, unbiased residuals across the full rank spectrum. (A) Training predictions in log-transformed space (R 2=0.964 , RMSE =0.131 , n=794 ). (B) Test predictions in log space (R 2=0.953 , RMSE =0.153 , n=199 ), showing tight clustering around the 1:1 line with ΔR2<0.011 between train and test, confirming generalization rather than memorization. (C) Test predictions back-transformed to original scale (R 2=0.932 , RMSE =2.29 m 3 /t), spanning low-capacity lignites ( ∼ 1–5 m 3 /t) to high-capacity anthracites ( ∼ 30–40 m 3 /t) without systematic rank-dependent bias. (D) Residual distributions (training: blue; test: purple) are near-Gaussian and centered at zero. (E) Absolute error versus true capacity: mild residual heteroscedasticity at high capacities is effectively managed by the log transformation. (F) Relative error distribution: median 2.27%, mean 5.2%, demonstrating high precision across the majority of coal conditions.
Figure 8: Residual diagnostics confirm homoscedasticity, approximate normality, and absence of systematic prediction bias across the full adsorption range. Analysis of n=199 held-out test residuals in log-transformed space. (A) Residuals vs. predicted values: random scatter around zero with constant variance; ±2σ bounds ( σ=0.153 ) contain 95.5% of points, confirming well-calibrated spread. (B) Residual histogram with fitted Gaussian (black dashed); Shapiro–Wilk p=0.064 , failing to reject normality at α=0.05 . (C) Q–Q plot: strong linearity in the central 90%, with minor heavy tails at extremes attributable to high-moisture low-rank coals at the edges of the training distribution. (D) Residuals vs. true values: no systematic trend, confirming prediction errors are independent of adsorption magnitude. (E) Symmetric cumulative residual distribution confirms the absence of subpopulation biases. (F) Flat scale-location trend confirms that log transformation successfully achieved homoscedasticity, validating the Gaussian likelihood assumption throughout the full adsorption range.
Model Variant
RMSE
MAE
R 2
MaxAE
Transfer-learned PINN
0.139
0.102
0.962
0.501
Random-random PINN
0.171
0.115
0.942
0.965
Random-classical PINN
0.182
0.122
0.934
0.937
Ensemble (10 models)
0.172
0.116
0.941
0.875
Table 4: Ablation study results on the held-out test set ( n=199 samples). All metrics computed on log-transformed adsorption values. Best performance in bold.
Comparison
t -stat
p -value
Cohen’s d
Significant?
Transfer vs. Random-random
−18.0
4.9×10−33
−1.80
Yes
Transfer vs. Ensemble
−18.9
1.2×10−34
−1.89
Yes
Random-random vs. Ensemble
−0.23
0.815
−0.02
No
Random-classical vs. Transfer
24.1
3.2×10−43
2.41
Yes
Table 5: Statistical comparison of model variants via bootstrap paired t -tests (100 resamples). Negative t -statistics and negative Cohen’s d indicate the first model outperforms the second. Bonferroni-corrected threshold α=0.0125 . Transfer learning significantly outperforms all alternatives with large effect sizes ( ∣d∣>1.8 ).
Figure 9: H 2 transfer learning outperforms all baselines in accuracy, calibration, and convergence speed, while deep ensembles provide no benefit over a single physics-constrained model. All variants were trained under identical conditions; test set n=199 . (A) Validation loss trajectories: transfer learning achieves consistently lower loss from the first epoch. (B) Validation R 2 progression: transfer exceeds 0.95 by epoch 600, approximately 130 epochs earlier than random initialization. (C) Parity plots (log space): transfer exhibits the tightest 1:1 clustering; all models show unbiased predictions. (D) Multi-metric comparison confirming transfer superiority across RMSE, MAE, and MaxAE. (E) Convergence speed: transfer reaches R 2≥0.90 at epoch 540 vs. epoch 670 for random-random (19.4% reduction). (F) Residual distributions: all models are unbiased, but transfer yields the narrowest spread. The ensemble (10 models, 10 × cost) is statistically indistinguishable from a single random-random PINN ( p=0.815 , Cohen’s d=−0.02 ), confirming that physics constraints eliminate the functional diversity that ensemble-based uncertainty requires.
Figure 10: Aleatoric uncertainty dominates (98.3%) and calibrated total uncertainty reliably identifies high-error predictions ( ρs=0.708 ). Analysis on held-out test set ( n=199 ); 100 stochastic MC Dropout forward passes. (A) Stacked epistemic and aleatoric uncertainty contributions sorted by true adsorption capacity; aleatoric dominates throughout the full capacity range. (B) Mean decomposition: epistemic ∼ 1.7%, aleatoric ∼ 98.3%; the low epistemic fraction reflects physical constraint of the solution space, not model inadequacy. (C) Total predicted uncertainty vs. true capacity, colored by absolute prediction error; higher uncertainty aligns with larger errors. (D) Per-sample epistemic vs. aleatoric uncertainty, colored by error magnitude; epistemic and aleatoric components carry complementary information. (E–F) Marginal distributions of epistemic and aleatoric uncertainties. (G–I) Absolute prediction error vs. epistemic (G, ρs=0.38 ), aleatoric (H, ρs=0.61 ), and total uncertainty (I, ρs=0.708 ); the strong total-uncertainty correlation confirms that joint propagation of both components (Section 2.9.3 ) is essential for practically useful uncertainty estimates.
Model Class
Accuracy
Calibrated UQ
Physics
Training Cost
Classical isotherms
Low ( R2≈0.29 )
None
Partial †
Negligible (CPU, seconds)
Random Forest
Moderate ( R2≈0.87 )
Poor
None
Low (CPU, minutes)
Transfer-learned PINN
High ( R2=0.932 )
Well-calibrated
Full
Moderate (GPU, minutes)
Table 6: Capability–cost comparison across methane sorption modeling approaches. The PINN occupies a distinct tier defined by the simultaneous provision of accuracy, calibrated UQ, and thermodynamic consistency unavailable to simpler model classes.
Method
Training
Inference
Storage
R 2
ρs
Laplace
1 ×
1 ×
1 ×
—
—
MC Dropout
1 ×
1.5 ×
1 ×
0.952
0.708
Standard Ensemble
5 ×
5 ×
5 ×
0.913
0.528
Diverse Ensemble
5 ×
5 ×
5 ×
0.738
0.083
Weighted Ensemble
10 ×
5 ×
5 ×
0.883
− 0.044
Table 7: Computational cost and performance of five UQ methods, normalized to deterministic training (1 × ) and inference (1 × ). MC Dropout achieves optimal efficiency–performance balance.
Figure 11: SHAP attribution confirms that engineered compositional and thermodynamic features dominate predictions, with moisture–volatile interaction as the leading control on methane sorption capacity. (A) Global feature importance (mean ∣ϕ∣ ); top five features account for 61.9% of total attribution, with 4 of 5 being engineered features, directly validating the physics-informed feature engineering strategy. (B) Beeswarm summary plot colored by normalized feature value; pressure is predominantly positive (62.8%) while moisture–VM interaction is bidirectional, reflecting rank-dependent context. (C–D) Dependence plots for moisture–VM interaction and fixed carbon show non-monotonic relationships with substantial scatter, confirming that linear models cannot capture these compositional effects. (E) Pairwise SHAP interaction heatmap; five pairs exceed ρSHAP>0.5 , including pressure– P×T ( ρSHAP=0.93 ), confirming internalization of coupled thermodynamic state variables. (F) Waterfall plot for a representative high-capacity anthracite sample, illustrating the additive decomposition of individual feature contributions from the base value to the final prediction.
Figure 12: ALE causal effect analysis confirms non-monotonic feature effects and corroborates SHAP rankings, with 11 of 12 features exhibiting physically interpretable threshold behavior. (A) Global ranking by ALE effect range; moisture–VM interaction leads (1.24 units), consistent with SHAP attribution, validating its role as the primary sorption control. (B) SHAP–ALE rank correlation ρs=0.45 ( p<0.05 ); the moderate agreement reflects expected divergence driven by multicollinearity (six features with VIF >5 ) rather than model inconsistency—low-VIF features achieve near-perfect rank agreement ( Δrank≤1 ). (C–F) ALE curves for the top four features with ±1 s.e. shading and sample density histograms: moisture–VM interaction shows pronounced negative effects at high values (pore blocking); fixed carbon displays non-monotonic behavior reflecting competing microporosity and connectivity effects; pressure follows smooth Langmuir-type saturation; temperature exhibits a U-shaped curve (enhancement at 20–30 ∘ C via micropore activation, reduction at 30–50 ∘ C via thermodynamic penalty) that resolves its near-zero Pearson correlation and validates the engineered thermal features.
Accurate prediction of hydrogen sorption in fine-grained geological materials is essential for evaluating underground hydrogen storage capacity, assessing caprock integrity, and characterizing hydrogen migration in subsurface energy systems. Classical isotherm models perform well at the individual-sample level but fail when generalized across heterogeneous populations, with the coefficient of determination collapsing from 0.80-0.90 for single-sample fits to 0.09-0.38 for aggregated multi-sample datasets. We present a multi-scale physics-informed neural network framework that addresses this limitation by embedding classical adsorption theory and thermodynamic constraints directly into the learning process. The framework utilizes 1,987 hydrogen sorption isotherm measurements across clays, shales, coals, supplemented by 224 characteristic uptake measurements. A seven-category physics-informed feature engineering scheme generates 62 thermodynamically meaningful descriptors from raw material characterization data. The loss function enforces saturation limits, a monotonic pressure response, and Van't Hoff temperature dependence via penalty weighting, while a three-phase curriculum-based training strategy ensures stable integration of competing physical constraints. An architecture-diverse ensemble of ten members provides calibrated uncertainty quantification, with post-hoc temperature scaling achieving target prediction interval coverage. The optimized PINN achieves R2 = 0.9544, RMSE = 0.0484 mmol/g, and MAE = 0.0231 mmol/g on the held-out test set, with 98.6% monotonicity satisfaction and zero non-physical negative predictions. Physics-informed regularization yields a 10-15% cross-lithology generalization advantage over a well-tuned random forest under leave-one-lithology-out validation, confirming that thermodynamic constraints transfer meaningfully across geological boundaries.
Mohammad Nooraiepour, Mohammad Masoudi, Zezhang Song +1
Faculty of Mathematics & Natural Sciences, University of Oslo, P.O. Box 1047 Blindern, 0316 Oslo, Norway · SINTEF Industry, Applied Geoscience Department, 7465 Trondheim, Norway · College of Geosciences, China University of Petroleum, Beijing 102249, China
We propose a self-supervised physics-informed neural network (PINN) framework that adaptively balances physics-based and data-driven supervision for scientific machine learning under data scarcity. Unlike prior PINNs that rely on fixed or heuristic weighting of physics residuals and data loss, our approach introduces a learnable blending neuron that dynamically adjusts the relative contribution of each term based on their uncertainties. This mechanism enables stable training and improved generalization without manual tuning. To further enhance efficiency, we integrate a transfer learning strategy that reuses representations from related domains and adapts them to new physical systems with limited data. We validate the framework for the prediction of heat transfer in liquid-metal miniature heat sinks using only 87 CFD datapoints, where the adaptive PINN achieves an error <8%, outperforming shallow neural networks, kernel methods, and physics-only baselines. Our framework provides a general recipe for embedding physics adaptively into neural networks, offering a robust and reproducible approach for data-scarce problems across various scientific domains, including fluid dynamics and material modeling.
Mineral prospectivity modelling (MPM) underpins exploration economics, yet most operational pipelines reduce to data-driven classifiers trained on shallow surface proxies. Such models are blind to the subsurface physics that actually localises ore: heat advection, fluid flow, and lithology-dependent precipitation. We present Korzhinskii-Net, a 2-D radial physics-informed neural network (PINN) that couples Darcy flow, advective-diffusive heat transport, and a softplus-saturated reaction rate into a single differentiable forward model, weakly supervised by surface and remote-sensing proxies. The network is named after Dmitri S. Korzhinskii (1899-1985), whose theory of infiltration metasomatism provides the physical scaffold. We evaluate Korzhinskii-Net on six ore provinces spanning three commodity classes - Udokan (sandstone-hosted Cu), Sukhoi Log, Olimpiada, and Berezovskoye (orogenic Au), Vorontsovskoye (Carlin-type Au), and Dalnegorsk (skarn polymetallic) - under a fair, leakage-controlled 5-fold cross-validation protocol with hard ring-shaped negatives and baseline proxy features disabled. Korzhinskii-Net attains a mean PR-AUC of 0.708 versus 0.235 for the strongest classical baseline (support vector machine), and a mean fractional rank of 0.036 versus 0.475. The improvement is consistent across all six provinces and three commodity systems, suggesting that physics-informed differentiable simulators, even when constrained only by global open-data proxies, can recover localisation patterns that pure feature-based learners systematically miss. We release the full pipeline and evaluation harness as open source.
Boris Kriuk
The Hong Kong University of Science and Technology, Hong Kong SAR