Two-stage Residual Learning for Modelling Strength-Informed Optimisation of Thermal Systems
Authors: Talha Ansar, Waqar Muhammad Ashraf, Abdulelah S. Alshehri, Omar Bayomie
Organizations: Department of Mechanical Engineering, University of Engineering and Technology Lahore, Kala Shah Kaku, Pakistan · The Sargent Centre for Process Systems Engineering, Department of Chemical Engineering, University College London, London, UK · Department of Chemical Engineering, College of Engineering, King Saud University, Riyadh, Saudi Arabia · The Advanced Centre for Biochemical Engineering, Department of Biochemical Engineering, University College London, London, UK · SSPC, The Science Foundation Ireland Research Centre for Pharmaceuticals, University College Dublin, Belfield, Dublin, Ireland · Institut des Systemes Intelligents et de Robotique, Sorbonne University, Paris 75005, France
Aggregate performance metrics do not identify local modelling inaccuracies in machine learning (ML) surrogates used for thermal-system optimisation. This study assesses local modelling strength through validation-based reliability screening and post-optimisation residual assessment. A two-stage residual learning (TSRL) framework, built on artificial neural networks (ANNs), is developed for modelling-strength-informed and domain-constrained optimisation of thermal systems. Within the TSRL framework, a base ANN predicts the process output, and a residual ANN learns the signed residuals left by the base ANN. The TSRL framework is implemented on two case studies of thermal systems, including 817 simulation observations of a crude distillation unit and operating records from a 395 MW industrial gas turbine. The TSRL framework reduces the test root mean square error (RMSE) from 10.3 g/s to 4.7 g/s for Fuel Oil in the refinery dataset and from 151 kJ/kWh to 105 kJ/kWh for turbine heat rate (THR). The base ANNs trained within TSRL are embedded in domain-constrained optimisation problems, and the modelling strength at each computed solution is assessed using the locked residual-score threshold. Each computed solution receives a good modelled region (GMR) or weak modelled region (WMR) label to inform ML model-based decision-making for safety-critical operation of thermal systems.
Figures & tables
Figure 1: TSRL information flow. The data contain process inputs x and observed output y , while the base and residual ANNs predict y^B and y^R , respectively. The residual target is ri=yi−y^iB . Validation labels based on absolute base error and residual score fix Te and Ts before testing. Only the base ANN enters the optimisation problem, and ∣y^R(x∗)∣ assigns a GMR or WMR label after convergence.
Hyperparameter
Search range
Sampling type
Learning rate
10−5 to 10−1
Log continuous
L1 coefficient
10−7 to 10−1
Log continuous
L2 coefficient
10−7 to 10−1
Log continuous
Hidden-layer neurons
2 to 32
Discrete in steps of 2
Table 1: Hyperparameter search space used independently for the base and residual ANNs.
Variable
Mean
SD
Minimum
Median
Maximum
Basrah (kg/s)
20.24
11.02
0.01
20.10
55.71
Arab Heavy (kg/s)
19.87
9.92
0.12
20.31
58.18
Arab Medium (kg/s)
20.50
10.98
0.06
20.76
58.33
Kuwait (kg/s)
20.41
10.84
0.04
21.09
64.82
Reboiler duty (kW)
35,297.00
193.00
34,638.00
35,300.00
35,860.00
Condenser 1 (kW)
6,399.96
239.55
4,850.55
6,417.13
7,021.43
Table 2: Descriptive statistics for the refinery variables. Values use two decimal places except the low-flow gas stream, which uses four.
Figure 2: Data characteristics for the crude distillation unit. Panel (a) shows kerosene increasing with Arab Medium flow while Light Naphtha decreases with Arab Heavy flow. Panel (b) shows relative crude contributions to product streams. Panel (c) shows the Spearman correlation map of the input and output variables.
Figure 3: Base-model parity plots across training, validation, and test data for (a) Light Naphtha, (b) Heavy Naphtha, (c) Kerosene, (d) Fuel Oil, (e) Gases, (f) CO 2 , and (g) AGO. The six non-gas surrogates provide reasonable simulator approximations within the sampled range.
Target
Metric
Training
Validation
Test
Base
Corrected
Base
Corrected
Base
Corrected
Light Naphtha (kg/s)
RMSE
0.0218
0.0243
0.0209
0.0205
0.0109
0.0187
R2
0.9745
0.9683
0.9734
0.9743
0.9945
0.9838
Gases (kg/s)
RMSE
0.0197
0.0195
0.0182
0.0181
0.0086
0.0083
R2
−0.0336
−0.0171
0.0702
0.0814
−2.1670
−2.0193
Heavy Naphtha (kg/s)
RMSE
0.0192
0.0193
0.0195
0.0194
0.0212
0.0214
Table 3: Base and residual-corrected performance for the crude distillation unit.
Figure 4: Fuel Oil residual and reliability-screening analysis showing (a) predicted and true signed residuals, (b) 50-bin test correction-error distribution, (c) validation ROC curve, and (d) residual-score classification profile on the test split with the locked threshold.
Figure 5: Parallel-coordinate comparison of grey refinery simulation observations and coloured optimisation solutions under different CO 2 ceilings. The figure relates the crude blend and utility inputs to the predicted emissions and product rates for solutions that satisfy the imposed input-domain constraint.
CO 2 limit (kg/h)
Base-model CO 2 (kg/h)
Corrected Fuel Oil (kg/s)
Residual score (g/s)
Band half-width (g/s)
Model region
8,198.8
8,198.8
44.21
13.10
6.30
WMR
8,240.2
8,235.2
44.22
26.10
12.60
WMR
8,281.5
8,276.5
44.23
35.10
17.50
WMR
8,322.8
8,317.8
44.25
49.70
23.60
WMR
8,364.2
8,359.2
44.26
65.70
32.90
WMR
8,405.5
8,400.5
44.28
84.30
41.10
WMR
Table 4: Fuel Oil model-region assessment for the refinery CO 2 sweep. The band is centred on corrected Fuel Oil, and its half-width is given in g/s. CO 2 is rounded to one decimal place and the remaining quantities to two.
Figure 6: Arab Heavy sensitivity results showing (a) re-optimised crude blend as the Arab Heavy target falls and (b) resulting changes in kerosene and corrected Fuel Oil predictions. Both panels show a predicted shift towards kerosene as Arab Heavy availability decreases.
Arab Heavy target (kg/s)
Optimised Arab Heavy (kg/s)
Corrected Fuel Oil (kg/s)
Residual score (g/s)
Band half-width (g/s)
Model region
51.73
51.15
46.55
9.20
4.50
GMR
45.28
44.69
45.98
11.00
5.30
GMR
38.83
38.24
45.68
10.40
5.20
GMR
32.38
31.79
45.41
7.30
3.50
GMR
25.93
25.34
45.11
6.00
3.00
GMR
19.48
18.89
44.78
8.70
4.20
GMR
Table 5: Fuel Oil model-region assessment for the Arab Heavy sensitivity. The band is centred on corrected Fuel Oil, and its half-width is given in g/s.
Variable
Minimum
Median
Maximum
Mean
SD
Compressor discharge temperature ( ∘ F)
813.00
847.80
925.80
861.00
34.20
Compressor discharge pressure (psi)
185.90
232.70
312.00
247.90
36.90
Gas fuel flow rate (lb/s)
28.56
37.81
49.83
39.20
5.67
Performance-heater gas outlet temperature ( ∘ F)
399.50
411.20
424.90
411.20
2.50
Fuel gas temperature at combustor inlet ( ∘ F)
484.00
517.60
534.90
513.50
14.80
Ambient temperature ( ∘ C)
20.50
25.60
34.80
26.30
3.60
Table 6: Descriptive statistics for the gas turbine dataset. All values use two decimal places.
Figure 7: Gas turbine model performance showing (a) base-model parity plots for Power, THR, and TE, alongside (b) density curves of measured, base, and corrected THR on the training, validation, and test splits.
Target
Metric
Training
Validation
Test
Base
Corrected
Base
Corrected
Base
Corrected
Power (MW)
RMSE
0.90
0.79
0.95
0.84
1.02
1.08
R2
0.9998
0.9998
0.9997
0.9998
0.9997
0.9997
THR (kJ/kWh)
RMSE
147.27
88.00
158.19
99.02
151.08
104.86
R2
0.9342
0.9765
0.9334
0.9739
0.9361
0.9692
Thermal efficiency (%)
RMSE
0.38
0.38
0.39
0.39
0.46
0.46
Table 7: Gas turbine base and residual-corrected performance.
Figure 8: THR residual and reliability-screening analysis showing (a) predicted and true signed residuals, (b) test correction-error distribution, (c) validation ROC curve, and (d) residual-score density and test classification with the locked threshold.
Scaled GFFR
Nominal GFFR (lb/s)
Base-model TE (%)
Signed THR residual (kJ/kWh)
THR indicator band (kJ/kWh)
Model region
0.2
32.81
38.74
−462.31
9,208.45±224.24
WMR
0.4
37.07
41.30
−424.97
8,727.05±206.14
WMR
0.6
41.32
42.94
−229.65
8,432.24±114.46
WMR
0.8
45.58
43.05
−106.29
8,480.86±50.38
GMR
1.0
49.83
42.44
163.06
8,525.73±81.53
WMR
Table 8: Gas turbine optimisation and post-optimisation THR assessment. The THR indicator band is centred on the corrected THR prediction.
Real-time supervisory control of thermal energy distribution systems requires digital twins that are accurate, interpretable, and uncertainty-aware, yet remain data and computationally efficient. High-fidelity simulations alone are costly, while purely data-driven surrogates often lack robustness. To address these challenges, this work proposes an active learning (AL) framework that couples system-level Modelica simulations with four simpler physics-informed and data-driven surrogate modeling approaches: deterministic Sparse Identification of Nonlinear Dynamics with Control (SINDyC), its probabilistic multivariate-Gaussian extension (MvG-SINDyC), feedforward neural network (FNN), and gated recurrent unit (GRU) network. Tailored to each surrogate, model-specific AL query strategies are employed, including Mahalanobis-distance sampling in coefficient space for MvG-SINDyC and error-based sampling in prediction space for SINDyC, FNN, and GRU, allowing the learning process to prioritize dynamically informative trajectories. The proposed approach is demonstrated on the glycol heat exchanger (GHX) subsystem of the Thermal Energy Distribution System (TEDS) at Idaho National Laboratory. Across key GHX outputs--the bypass mass flow rate m˙GHX and heat transfer rate QGHX-the AL framework achieves comparable predictive accuracy using as few as one-fifth of the simulation trajectories required by random sampling. Among the evaluated surrogates, the GRU achieves the highest predictive fidelity, while SINDyC remains the most computationally efficient and interpretable. The probabilistic MvG-SINDyC surrogate further enables uncertainty quantification and exhibits the largest computational gains under AL.
Umme Mahbuba Nabila, Paul Seurin, Linyu Lin +1
Department of Nuclear Engineering and Radiological Sciences, University of Michigan, Ann Arbor, MI 48109, United States · Nuclear Science & Technology Division, Idaho National Laboratory, Idaho Falls, ID 83415, United States · Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI 48109, United States
Artificial-intelligence surrogates can support second-by-second thermal-hydraulic forecasting, but models selected and frozen offline may become condition-locked once deployed outside their pretraining envelope. This study develops a guarded continual-adaptation framework for experimental thermal-hydraulic loop data in which role-separated agents - Monitor, Diagnosis, Adaptation, Safety-Auditor, and Orchestrator - diagnose error signatures, prioritize candidate model families, and review promotions, while deterministic champion-challenger gates and background shadow learning retain final authority over model replacement. Seven surrogate families were screened by blocked three-fold cross-validation, and a temporal Fourier neural operator was selected as the initial champion for 60-s-history-to-10-s-trajectory forecasting on two held-out transients, with three seeds per adaptive mode. Static deployment gave a channel-averaged MAE of 7.06 and a 56.8% warning-exceedance ratio; rule-based adaptation reduced MAE to 6.54, whereas shadow refresh alone remained close to Static. The MA-Full mode, in which the role-separated multi-agent council reviews every evaluated stream step, achieved the lowest mean error, 5.72, and 35.8% exceedance, corresponding to a 19.0% improvement over Static. Paired bootstrap intervals against Static excluded zero, although intervals among adaptive modes overlapped and the six paired units limit broad statistical claims. Validated promotions from the neural operator to Transformer and graph neural network indicate that logged, gate-controlled adaptation can support auditable surrogate evolution while deterministic gates retain deployment authority.
Doyeong Lim, Seungyoon Lee, In Cheol Bang
Department of Nuclear Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, 44919, Republic of Korea
Neural surrogates are widely used in scientific machine learning for fast prediction of three-dimensional (3D) thermo-fluid fields. However, generating training data using conventional numerical solvers often incurs substantial computational and storage costs. We propose to train an attention graph neural network by minimizing the finite-volume method (FVM) residuals of the governing equations. These residuals are evaluated directly on the mesh, requiring no labeled data. We evaluate the trained surrogates against computational fluid dynamics (CFD) references and a data-supervised baseline across four scenarios. On the two steady-state benchmarks, the FVM-loss model achieves an all-field normalized root-mean-square error (nRMSE) of 2.3-2.8%. It demonstrates close agreement with the CFD references, including the buoyancy-energy coupling. On the two parametric transient cases, the FVM-loss model outperforms the supervised baseline in terms of accuracy, while avoiding the data-generation cost entirely. These results indicate that the FVM loss can provide a practical training signal for neural surrogates and reduce the model development cost.
Tianyu Li, Zhiwei Cao, Qingang Zhang +3
Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore, Singapore. · School of Future Technology, Fuzhou University, Fuzhou, China. · College of Computing and Data Science, Nanyang Technological University, Singapore, Singapore. +2