Surrogate Modeling and Explainable Artificial Intelligence for Complex Systems: A Workflow for Automated Simulation Exploration
Authors: Paul Saves, Pramudita Satria Palar, Muhammad Daffa Robani, Nicolas Verstaevel, Moncef Garouani, Jordan Levy, Julien Aligon, Benoit Gaudou, Koji Shimoyama, Joseph Morlier
Organizations: IRIT, UMR 5505 CNRS, Universit´e Toulouse Capitole, Toulouse, 31000, France. · Faculty of Mechanical and Aerospace Engineering, Institut Teknologi Bandung, Jl. Ganesha 10, Bandung, 15414, West Java, Indonesia. · AwanTunai, Urban Suites, 3 Hullet Road, Singapore, 229158, Singapore, Singapore. · Mechanical Engineering, Kyushu University, 744 Motooka, Fukuoka, 819-0395, Japan. · Institut Cl´ement Ader (ICA), Universit´e de Toulouse, ISAE-SUPAERO, Mines Albi, UPS, INSA, CNRS, 3 rue Caroline Aigle, Toulouse, 31400, Occitanie, France.
Complex systems are increasingly explored through simulation-driven workflows that combine physics-based and empirical models with optimization and analytics. Despite their power, these workflows face two central obstacles: high computational cost from repeated expensive simulator runs, and limited transparency when decisions rely on opaque black-box models. We propose a workflow that addresses both challenges by training lightweight emulators on compact designs of experiments that (i) provide fast, low-latency approximations of expensive simulators, (ii) enable rigorous uncertainty quantification, and (iii) support global and local eXplainable Artificial Intelligence (XAI) analyses. This work formalizes a methodological bridge accommodating continuous, categorical, and integer variables across diverse domains, from engineering design to socio-environmental agent-based models. To validate surrogate explanations and examine how faithfully they reflect simulator behavior, we introduce a comparative methodology incorporating active validation filters, including explanation consistency and aleatoric uncertainty quantification. Furthermore, we provide practical guidelines for coupling experimental designs with emulator training to select architectures that balance computational efficiency with interpretability. We demonstrate the framework across two contrasting paradigms: a deterministic multidisciplinary design simulation of a hybrid-electric aircraft and a stochastic agent-based simulation of urban segregation. Results show that coupling surrogates with XAI enables large-scale exploration in seconds, reveals critical nonlinear interactions, and highlights potential design and policy levers while establishing consistent explanations across architectures.
Figures & tables
Figure 1: Workflow for interpretable surrogate modeling in complex system exploration, illustrating the integration of domain assumptions, computational limitations, and adaptive learning pathways.
Figure 2: Workflow for interpretable surrogate models for a design-oriented simulation.
Figure 3: DRAGON aircraft mock-up [ 112 ] .
Function/Variable
Nature
Range
Fan operating pressure ratio
cont
[1.05, 1.3]
Wing aspect ratio
cont
[8, 12]
Angle for swept wing
cont
[15, 40] (∘)
Wing taper ratio
cont
[0.2, 0.5]
Horizontal Tail (HT) aspect ratio
cont
[3, 6]
Angle for swept HT
cont
[20, 40] (∘)
Table 1: Design variables for fuel mass analysis.
Architecture number
number of motors
number of cores and generators
1
8
2
2
12
2
3
16
2
4
20
2
5
24
2
6
28
2
Table 2: Definition of the architecture variable and its 17 associated levels.
Layout
position
y ratio
tail
VT aspect ratio
VT taper ratio
1
under wing
0.25
without T-tail
1.8
0.3
2
behind
0.34
with T-tail
1.2
0.85
Table 3: Definition of the turboshaft layout variable and its 2 associated levels.
Figure 4: GSA results from partial dependence functions for the DRAGON problem.
Figure 5: GSA results from the averaged SHAP values for the DRAGON problem.
Figure 6: PDP and ICE curves for the DRAGON problem.
Figure 8: SHAP beeswarm plot for the DRAGON problem continuous variables.
Figure 9: PDP and ICE curves for the DRAGON problem output predicted variance.
Figure 10: Workflow for interpretable surrogate models for an analysis-oriented simulation.
Function/Variable
Nature
Range
Number of individual types
int
[2, 5]
Density of individuals
cont
[0.01, 1]
Intolerance threshold
cont
[0, 1]
Size of the square grid edges
int
[10,40]
Perception distance
cont
[1, 10]
Table 4: Design variables for diversity analysis.
Figure 11: Evolution of aleatoric uncertainty metrics with respect to the number of replicates. (Left) Raw variance and standard error for the binary convergence output. (Right) Global coefficient of variation and relative standard error for the continuous sparsity output.
Figure 12: An instance of the segregation problem simulated on the GAMA platform.
Figure 13: The dataset with converged simulations in green and the others in red.
Model
RMSE
MCC
PVA
Reg. Rank
Class. Rank
PVA Rank
RF
6.86
0.83
4.13
3
4
11
GB
9.21
0.82
3.44
9
5
10
DT
8.34
0.75
4.19
7
8
12
GP
8.22
0.87
3.14
6
1
9
RBF
8.17
0.87
5.32
5
1
14
SVM
11.53
0.21
1.01
14
16
7
Table 5: Global predictive qualities comparison across 16 surrogate models, with darker green shades representing better performance.
Figure 14: RF, TabPFN, and GP response surfaces of sparsity as a function of density with the other variables fixed.
Figure 15: Variables importance in the random forest based on MDI.
Figure 16: Variables importance in the k-Nearest Neighbors based on SHAP.
Figure 17: SHAP values predicted by the random forest.
Figure 18: PDP (thick black lines) and ICE (colored lines) curves showing the variations of total uncertainty ( σtotal ), predicted through the dual-scheme random forest, regarding convergence probability (top) and sparsity value (bottom).
Figure 19: SHAP values NDCG (%) for every pairs of clustered models.
Figure 20: SHAP values for NDCG-based hierarchical clustering.
Figure 21: Cross-model similarity matrices for predictive performance ( ρpred , κpred or SRMSE ) and explanation consistency ( ρSHAP , τSHAP ) on convergence (left) and sparsity (right) outputs.
Figure 22: Mantel test meta-correlation matrices for convergence (left) and sparsity (right).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 23: SHAP dependence plots for the DRAGON problem output predicted variance.
Figure 24: SHAP values as a function of the variables predicted by the average of all surrogate models.
Figure 25: PDP and ICE plots as a function of the variables predicted by the GP model.
The simulation of complex systems increasingly relies on sophisticated but fundamentally opaque computational black-box simulators. Surrogate models play a central role in reducing the computational cost of complex systems simulations across a wide range of scientific and engineering domains. Notwithstanding, they inevitably inherit and often exacerbate this black-box nature, obscuring how input variables drive physical responses. Conversely, Explainable Artificial Intelligence (XAI) offers powerful tools to unpack these models. Yet, XAI methods struggle with engineering-specific constraints, such as highly correlated inputs, dynamical systems, and rigorous reliability requirements. Consequently, surrogate modeling and XAI have largely evolved as distinct fields of research, despite their strong complementarity. To reconnect these approaches, this state-of-the-art survey provides a structured perspective that maps existing XAI techniques onto the various stages of surrogate modeling workflows for design and exploration. To ground this synthesis, we draw upon illustrative applications across both equation-based simulations and agent-based modeling. We survey a broad spectrum of techniques, highlighting their strengths for revealing interactions and supporting human comprehension. Finally, we identify pressing open challenges, including the explainability of dynamical systems and the handling of mixed-variable systems, and propose a research agenda to make explainability a core, embedded element of simulation-driven workflows from model construction through decision-making. By transforming opaque emulators into explainable tools, this agenda empowers practitioners to move beyond accelerating simulations to extracting actionable insights from complex system behaviors.
Pramudita Satria Palar, Paul Saves, Muhammad Daffa Robani +6
Faculty of Mechanical and Aerospace Engineering, Institut Teknologi Bandung, Jl. Ganesha 10, Bandung, 15414, West Java, Indonesia. · IRIT, Université Toulouse Capitole, Toulouse, 31000,France. · AwanTunai, Urban Suites, 3 Hullet Road, Singapore, 229158, Singapore, Singapore. +2
Systematic exploration of Agent-Based Models (ABMs) is challenged by the curse of dimensionality and their inherent stochasticity. We present a multi-stage pipeline integrating the systematic design of experiments with machine learning surrogates. Using a predator-prey case study, our methodology proceeds in two steps. First, an automated model-based screening identifies dominant variables, assesses outcome variability, and segments the parameter space. Second, we train Machine Learning models to map the remaining nonlinear interaction effects. This approach automates the discovery of unstable regions where system outcomes are highly dependent on nonlinear interactions between many variables. Thus, this work provides modelers with a rigorous, hands-off framework for sensitivity analysis and policy testing, even when dealing with high-dimensional stochastic simulators.
Paul Saves, Matthieu Mastio, Nicolas Verstaevel +1
IRIT, Université Toulouse Capitole, Toulouse, France
High-fidelity numerical simulation of subsurface flow is computationally intensive, especially for many-query tasks such as uncertainty quantification and data assimilation. Deep learning (DL) surrogates can significantly accelerate forward simulations, yet constructing them requires substantial machine learning (ML) expertise - from architecture design to hyperparameter tuning - that most domain scientists do not possess. Furthermore, the process is predominantly manual and relies heavily on heuristic choices. This expertise gap remains a key barrier to the broader adoption of DL surrogate techniques. For this reason, we present AutoSurrogate, a large-language-model-driven multi-agent framework that enables practitioners without ML expertise to build high-quality surrogates for subsurface flow problems through natural-language instructions. Given simulation data and optional preferences, four specialized agents collaboratively execute data profiling, architecture selection from a model zoo, Bayesian hyperparameter optimization, model training, and quality assessment against user-specified thresholds. The system also handles common failure modes autonomously, including restarting training with adjusted configurations when numerical instabilities occur and switching to alternative architectures when predictive accuracy falls short of targets. In our setting, a single natural-language sentence can be sufficient to produce a deployment-ready surrogate model, with minimum human intervention required at any intermediate stage. We demonstrate the utility of AutoSurrogate on a 3D geological carbon storage modeling task, mapping permeability fields to pressure and CO2 saturation fields over 31 timesteps. Without any manual tuning, AutoSurrogate is able to outperform expert-designed baselines and domain-agnostic AutoML methods, demonstrating strong potential for practical deployment.
Jiale Liu, Nanzhe Wang
School of Physics and Astronomy, The University of Edinburgh, Edinburgh, EH9 3FD, United Kingdom · Institute of GeoEnergy Engineering, School of Energy, Geoscience, Infrastructure and Society, Heriot-Watt University, Edinburgh, EH14 4AS, United Kingdom · Subsurface Energy Transition and Innovation Centre, Heriot-Watt University, Edinburgh, EH14 4AS, United Kingdom