Two-stage Residual Learning for Modelling Strength-Informed Optimisation of Thermal Systems

Talha Ansar; Waqar Muhammad Ashraf; Abdulelah S. Alshehri; Omar Bayomie

cs.LGOct 14, 2026

Two-stage Residual Learning for Modelling Strength-Informed Optimisation of Thermal Systems

Authors: Talha Ansar, Waqar Muhammad Ashraf, Abdulelah S. Alshehri, Omar Bayomie

Organizations: Department of Mechanical Engineering, University of Engineering and Technology Lahore, Kala Shah Kaku, Pakistan · The Sargent Centre for Process Systems Engineering, Department of Chemical Engineering, University College London, London, UK · Department of Chemical Engineering, College of Engineering, King Saud University, Riyadh, Saudi Arabia · The Advanced Centre for Biochemical Engineering, Department of Biochemical Engineering, University College London, London, UK · SSPC, The Science Foundation Ireland Research Centre for Pharmaceuticals, University College Dublin, Belfield, Dublin, Ireland · Institut des Systemes Intelligents et de Robotique, Sorbonne University, Paris 75005, France

Abstract

Aggregate performance metrics do not identify local modelling inaccuracies in machine learning (ML) surrogates used for thermal-system optimisation. This study assesses local modelling strength through validation-based reliability screening and post-optimisation residual assessment. A two-stage residual learning (TSRL) framework, built on artificial neural networks (ANNs), is developed for modelling-strength-informed and domain-constrained optimisation of thermal systems. Within the TSRL framework, a base ANN predicts the process output, and a residual ANN learns the signed residuals left by the base ANN. The TSRL framework is implemented on two case studies of thermal systems, including 817 simulation observations of a crude distillation unit and operating records from a 395 MW industrial gas turbine. The TSRL framework reduces the test root mean square error (RMSE) from 10.3 g/s to 4.7 g/s for Fuel Oil in the refinery dataset and from 151 kJ/kWh to 105 kJ/kWh for turbine heat rate (THR). The base ANNs trained within TSRL are embedded in domain-constrained optimisation problems, and the modelling strength at each computed solution is assessed using the locked residual-score threshold. Each computed solution receives a good modelled region (GMR) or weak modelled region (WMR) label to inform ML model-based decision-making for safety-critical operation of thermal systems.

Figures & tables

Figure 1: TSRL information flow. The data contain process inputs $x$ and observed output $y$ , while the base and residual ANNs predict $\hat{y}^{B}$ and $\hat{y}^{R}$ , respectively. The residual target is $r_{i}=y_{i}-\hat{y}^{B}_{i}$ . Validation labels based on absolute base error and residual score fix $T_{e}$ and $T_{s}$ before testing. Only the base ANN enters the optimisation problem, and $|\hat{y}^{R}(x^{*})|$ assigns a GMR or WMR label after convergence.

Hyperparameter	Search range	Sampling type
Learning rate	$10^{-5}$ to $10^{-1}$	Log continuous
L1 coefficient	$10^{-7}$ to $10^{-1}$	Log continuous
L2 coefficient	$10^{-7}$ to $10^{-1}$	Log continuous
Hidden-layer neurons	2 to 32	Discrete in steps of 2

Table 1: Hyperparameter search space used independently for the base and residual ANNs.

Variable	Mean	SD	Minimum	Median	Maximum
Basrah (kg/s)	20.24	11.02	0.01	20.10	55.71
Arab Heavy (kg/s)	19.87	9.92	0.12	20.31	58.18
Arab Medium (kg/s)	20.50	10.98	0.06	20.76	58.33
Kuwait (kg/s)	20.41	10.84	0.04	21.09	64.82
Reboiler duty (kW)	35,297.00	193.00	34,638.00	35,300.00	35,860.00
Condenser 1 (kW)	6,399.96	239.55	4,850.55	6,417.13	7,021.43

Table 2: Descriptive statistics for the refinery variables. Values use two decimal places except the low-flow gas stream, which uses four.

Figure 2: Data characteristics for the crude distillation unit. Panel (a) shows kerosene increasing with Arab Medium flow while Light Naphtha decreases with Arab Heavy flow. Panel (b) shows relative crude contributions to product streams. Panel (c) shows the Spearman correlation map of the input and output variables.

Figure 3: Base-model parity plots across training, validation, and test data for (a) Light Naphtha, (b) Heavy Naphtha, (c) Kerosene, (d) Fuel Oil, (e) Gases, (f) CO 2 , and (g) AGO. The six non-gas surrogates provide reasonable simulator approximations within the sampled range.

Target	Metric	Training		Validation		Test
Target	Metric	Base	Corrected	Base	Corrected	Base	Corrected
Light Naphtha (kg/s)	RMSE	0.0218	0.0243	0.0209	0.0205	0.0109	0.0187
Light Naphtha (kg/s)	$R^{2}$	0.9745	0.9683	0.9734	0.9743	0.9945	0.9838
Gases (kg/s)	RMSE	0.0197	0.0195	0.0182	0.0181	0.0086	0.0083
Gases (kg/s)	$R^{2}$	$-0.0336$	$-0.0171$	0.0702	0.0814	$-2.1670$	$-2.0193$
Heavy Naphtha (kg/s)	RMSE	0.0192	0.0193	0.0195	0.0194	0.0212	0.0214

Table 3: Base and residual-corrected performance for the crude distillation unit.

Figure 4: Fuel Oil residual and reliability-screening analysis showing (a) predicted and true signed residuals, (b) 50-bin test correction-error distribution, (c) validation ROC curve, and (d) residual-score classification profile on the test split with the locked threshold.

Figure 5: Parallel-coordinate comparison of grey refinery simulation observations and coloured optimisation solutions under different CO 2 ceilings. The figure relates the crude blend and utility inputs to the predicted emissions and product rates for solutions that satisfy the imposed input-domain constraint.

CO 2 limit (kg/h)	Base-model CO 2 (kg/h)	Corrected Fuel Oil (kg/s)	Residual score (g/s)	Band half-width (g/s)	Model region
8,198.8	8,198.8	44.21	13.10	6.30	WMR
8,240.2	8,235.2	44.22	26.10	12.60	WMR
8,281.5	8,276.5	44.23	35.10	17.50	WMR
8,322.8	8,317.8	44.25	49.70	23.60	WMR
8,364.2	8,359.2	44.26	65.70	32.90	WMR
8,405.5	8,400.5	44.28	84.30	41.10	WMR

Table 4: Fuel Oil model-region assessment for the refinery CO 2 sweep. The band is centred on corrected Fuel Oil, and its half-width is given in g/s. CO 2 is rounded to one decimal place and the remaining quantities to two.

Figure 6: Arab Heavy sensitivity results showing (a) re-optimised crude blend as the Arab Heavy target falls and (b) resulting changes in kerosene and corrected Fuel Oil predictions. Both panels show a predicted shift towards kerosene as Arab Heavy availability decreases.

Arab Heavy target (kg/s)	Optimised Arab Heavy (kg/s)	Corrected Fuel Oil (kg/s)	Residual score (g/s)	Band half-width (g/s)	Model region
51.73	51.15	46.55	9.20	4.50	GMR
45.28	44.69	45.98	11.00	5.30	GMR
38.83	38.24	45.68	10.40	5.20	GMR
32.38	31.79	45.41	7.30	3.50	GMR
25.93	25.34	45.11	6.00	3.00	GMR
19.48	18.89	44.78	8.70	4.20	GMR

Table 5: Fuel Oil model-region assessment for the Arab Heavy sensitivity. The band is centred on corrected Fuel Oil, and its half-width is given in g/s.

Variable	Minimum	Median	Maximum	Mean	SD
Compressor discharge temperature ( ∘ F)	813.00	847.80	925.80	861.00	34.20
Compressor discharge pressure (psi)	185.90	232.70	312.00	247.90	36.90
Gas fuel flow rate (lb/s)	28.56	37.81	49.83	39.20	5.67
Performance-heater gas outlet temperature ( ∘ F)	399.50	411.20	424.90	411.20	2.50
Fuel gas temperature at combustor inlet ( ∘ F)	484.00	517.60	534.90	513.50	14.80
Ambient temperature ( ∘ C)	20.50	25.60	34.80	26.30	3.60

Table 6: Descriptive statistics for the gas turbine dataset. All values use two decimal places.

Figure 7: Gas turbine model performance showing (a) base-model parity plots for Power, THR, and TE, alongside (b) density curves of measured, base, and corrected THR on the training, validation, and test splits.

Target	Metric	Training		Validation		Test
Target	Metric	Base	Corrected	Base	Corrected	Base	Corrected
Power (MW)	RMSE	0.90	0.79	0.95	0.84	1.02	1.08
Power (MW)	$R^{2}$	0.9998	0.9998	0.9997	0.9998	0.9997	0.9997
THR (kJ/kWh)	RMSE	147.27	88.00	158.19	99.02	151.08	104.86
THR (kJ/kWh)	$R^{2}$	0.9342	0.9765	0.9334	0.9739	0.9361	0.9692
Thermal efficiency (%)	RMSE	0.38	0.38	0.39	0.39	0.46	0.46

Table 7: Gas turbine base and residual-corrected performance.

Figure 8: THR residual and reliability-screening analysis showing (a) predicted and true signed residuals, (b) test correction-error distribution, (c) validation ROC curve, and (d) residual-score density and test classification with the locked threshold.

Scaled GFFR	Nominal GFFR (lb/s)	Base-model TE (%)	Signed THR residual (kJ/kWh)	THR indicator band (kJ/kWh)	Model region
0.2	32.81	38.74	$-462.31$	$9,208.45\pm 224.24$	WMR
0.4	37.07	41.30	$-424.97$	$8,727.05\pm 206.14$	WMR
0.6	41.32	42.94	$-229.65$	$8,432.24\pm 114.46$	WMR
0.8	45.58	43.05	$-106.29$	$8,480.86\pm 50.38$	GMR
1.0	49.83	42.44	163.06	$8,525.73\pm 81.53$	WMR

Table 8: Gas turbine optimisation and post-optimisation THR assessment. The THR indicator band is centred on the corrected THR prediction.

Explore similar work

Physics-based Digital Twins for Integrated Thermal Energy Systems Using Active Learning
May 7, 2026Umme Mahbuba Nabila, Paul Seurin, Linyu Lin +1Surrogate Modeling Sparse Identification of Nonlinear Dynamics
Department of Nuclear Engineering and Radiological Sciences, University of Michigan, Ann Arbor, MI 48109, United States · Nuclear Science & Technology Division, Idaho National Laboratory, Idaho Falls, ID 83415, United States · Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI 48109, United States
Validation-Gated Multi-Agent Governance for Online Adaptation of Thermal-Hydraulic Surrogate Models under Operating-Regime Shift
Jun 2, 2026Doyeong Lim, Seungyoon Lee, In Cheol BangNeural Surrogate Modeling Continual Test-Time Adaptation
Department of Nuclear Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, 44919, Republic of Korea
Label-Free Finite-Volume-Residual Training of Attention Graph Neural Networks for Coupled Thermo-Fluid Fields
Jul 22, 2026Tianyu Li, Zhiwei Cao, Qingang Zhang +3Neural Surrogate Modeling Graph Neural Networks
Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore, Singapore. · School of Future Technology, Fuzhou University, Fuzhou, China. · College of Computing and Data Science, Nanyang Technological University, Singapore, Singapore. +2

May 7, 2026J/K move · Enter open · S save

Physics-based Digital Twins for Integrated Thermal Energy Systems Using Active Learning

Umme Mahbuba Nabila, Paul Seurin, Linyu Lin, Majdi I. Radaideh

Department of Nuclear Engineering and Radiological Sciences, University of Michigan, Ann Arbor, MI 48109, United States · Nuclear Science & Technology Division, Idaho National Laboratory, Idaho Falls, ID 83415, United States · Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI 48109, United States

Surrogate Modeling Sparse Identification of Nonlinear Dynamics Nonlinear System Identification Active Learning Physics-Informed ML Digital Twins

Open paper AskPDF arXiv

Real-time supervisory control of thermal energy distribution systems requires digital twins that are accurate, interpretable, and uncertainty-aware, yet remain data and computationally efficient. High-fidelity simulations alone are costly, while purely data-driven surrogates often lack robustness. To address these challenges, this work proposes an active learning (AL) framework that couples system-level Modelica simulations with four simpler physics-informed and data-driven surrogate modeling approaches: deterministic Sparse Identification of Nonlinear Dynamics with Control (SINDyC), its probabilistic multivariate-Gaussian extension (MvG-SINDyC), feedforward neural network (FNN), and gated recurrent unit (GRU) network. Tailored to each surrogate, model-specific AL query strategies are employed, including Mahalanobis-distance sampling in coefficient space for MvG-SINDyC and error-based sampling in prediction space for SINDyC, FNN, and GRU, allowing the learning process to prioritize dynamically informative trajectories. The proposed approach is demonstrated on the glycol heat exchanger (GHX) subsystem of the Thermal Energy Distribution System (TEDS) at Idaho National Laboratory. Across key GHX outputs--the bypass mass flow rate

\dot{m}_{\mathrm{GHX}}

and heat transfer rate

Q_{\mathrm{GHX}}

-the AL framework achieves comparable predictive accuracy using as few as one-fifth of the simulation trajectories required by random sampling. Among the evaluated surrogates, the GRU achieves the highest predictive fidelity, while SINDyC remains the most computationally efficient and interpretable. The probabilistic MvG-SINDyC surrogate further enables uncertainty quantification and exhibits the largest computational gains under AL.

Abstract

Explore similar work

Physics-based Digital Twins for Integrated Thermal Energy Systems Using Active Learning

Validation-Gated Multi-Agent Governance for Online Adaptation of Thermal-Hydraulic Surrogate Models under Operating-Regime Shift

Label-Free Finite-Volume-Residual Training of Attention Graph Neural Networks for Coupled Thermo-Fluid Fields