Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making
Organizations: Faculty of Mechanical and Aerospace Engineering, Institut Teknologi Bandung, Jl. Ganesha 10, Bandung, 15414, West Java, Indonesia. · IRIT, Université Toulouse Capitole, Toulouse, 31000,France. · AwanTunai, Urban Suites, 3 Hullet Road, Singapore, 229158, Singapore, Singapore. · Department of Mechanical Engineering, Kyushu University, 744 Motooka, Fukuoka, 819-0395, Japan. ·
Abstract
The simulation of complex systems increasingly relies on sophisticated but fundamentally opaque computational black-box simulators. Surrogate models play a central role in reducing the computational cost of complex systems simulations across a wide range of scientific and engineering domains. Notwithstanding, they inevitably inherit and often exacerbate this black-box nature, obscuring how input variables drive physical responses. Conversely, Explainable Artificial Intelligence (XAI) offers powerful tools to unpack these models. Yet, XAI methods struggle with engineering-specific constraints, such as highly correlated inputs, dynamical systems, and rigorous reliability requirements. Consequently, surrogate modeling and XAI have largely evolved as distinct fields of research, despite their strong complementarity. To reconnect these approaches, this state-of-the-art survey provides a structured perspective that maps existing XAI techniques onto the various stages of surrogate modeling workflows for design and exploration. To ground this synthesis, we draw upon illustrative applications across both equation-based simulations and agent-based modeling. We survey a broad spectrum of techniques, highlighting their strengths for revealing interactions and supporting human comprehension. Finally, we identify pressing open challenges, including the explainability of dynamical systems and the handling of mixed-variable systems, and propose a research agenda to make explainability a core, embedded element of simulation-driven workflows from model construction through decision-making. By transforming opaque emulators into explainable tools, this agenda empowers practitioners to move beyond accelerating simulations to extracting actionable insights from complex system behaviors.
Figures & tables
| Criterion | Advantages (PROs) | Limitations (CONs) | Example of Surrogate Models |
| Prediction Accuracy | Captures underlying trends and relationships; enables reliable knowledge extraction | Risk of overfitting or underfitting; may require large datasets or careful hyperparameter tuning | Regular GP, random forest, TabPFN, deep ensemble |
| Smoothness | Facilitates physical consistency and stable post-hoc explanations | May fail to capture sharp discontinuities; some models yield stepwise responses | Smoothing spline, radial basis functions, polynomial chaos expansion |
| Interpretability | Direct insight into input–output relationships; supports decision-making without external XAI tools | Often comes with lower predictive accuracy compared to black-box models; may not scale to high dimensions | Linear regression, decision tree, generalized additive model |
| Relatively Fast Prediction | Enables rapid evaluation for global explainability and sampling-based XAI methods | High-accuracy models in high dimensions can be computationally expensive | nearest neighbors, inverse distance weighting, sparse GP, linear regression. |
| Uncertainty Estimation | Provides confidence intervals for predictions; enhances trust and interpretability | Not all models natively support uncertainty; post-hoc methods ( e.g. , bootstrapping, conformal predictions) can be costly | Bayesian neural networks, Relevance vector machines, GP |
| Method | Interpretability | Inter-feature Analysis | Computation Time | Feature Independence? | Attribution Accuracy |
| ICE/PDP | Very Easy | N/A | Low | N/A | N/A |
| Counterfactuals | Moderate | Difficult | High | N/A | N/A |
| LIME | Easy | Possible | Low | Yes | Low |
| Anchor | Easy | Possible | Low | Yes | Low |
| SHAP | Easy | Possible | High | Yes | Moderate |
| LORE | Easy | Possible | Moderate | Yes | Moderate |
Surrogate Modeling
Surrogate modeling builds faster or more interpretable approximations of expensive simulations and black-box models.
See all 227 papers