Causal Analysis of Failures of Cyber-Physical Systems with Continuous Inputs

Tom Yaacov; David A. Kelly; Khen Elimelech; Hana Chockler

cs.ROOct 22, 2026

Causal Analysis of Failures of Cyber-Physical Systems with Continuous Inputs

Authors: Tom Yaacov, David A. Kelly, Khen Elimelech, Hana Chockler

Organizations: King’s College London

Abstract

Modern autonomous Cyber-Physical Systems (CPSs), such as self-driving cars, operate in increasingly complex environments while expected to maintain high reliability. The black-box nature of many of these systems makes it impossible to fully verify their behavior prior to deployment. When a failure occurs, it is important to analyze it to guide improvements to system robustness. A recent work introduced the novel concept of leveraging the framework of actual causality for CPS failure explanation. However, that work considered only a discrete, binary causal model that cannot capture continuous changes in system inputs. In this paper, we address these limitations by extending the actual causality framework with the notion of distance to account for the continuous characteristics of the system. Specifically, we introduce the novel concepts of distance-weighted actual cause and distance-weighted degree of responsibility, enabling causal influence to account for the magnitude of changes required to avoid a failure, rather than only its cardinality. While motivated by the continuous nature of CPSs, this constitutes a general extension of the theory of actual causality and enables more fine-grained analysis of continuous inputs in any causal model. Building on these notions, we present an efficient, practical, system-agnostic explanation-derivation algorithm that uses distance-weighted responsibility to guide the explanation search. We experimentally evaluate our approach across a number of environments from two autonomous-driving simulators with black-box neural controllers. The results show that the proposed extension provides more effective guidance than the current discrete approach.

Figures & tables

Fig. 1 : The running example of an autonomous car, taken from the LiteRacer simulator [ 19 ] . The car is controlled by a black-box neural network trained to navigate an obstructed track, avoiding collisions and reaching the end of the track.

Panel of Figure 2 — Fig. 2 : A graphical representation of the causal model used to explain failures of our exemplary autonomous-car system. Unlike [ 3 ] , here the test inputs, i.e., obstacle placements, are modeled as continuous location variables, rather than binary present/absent indicators. The outcome variable $O$ indicates whether the outcome of some execution under the intervention, $(\overline{\xi}^{\prime},\textit{env}^{\prime})$ , matches that of the original $(\overline{\xi},\textit{env})$ .

Panel of Figure 3 — Fig. 3 : Motivating example for distance-weighted responsibility: a failure scenario (left) and its sufficient explanation (right).

Fig. 4 : The MetaDrive simulation [ 23 ] . The vehicle is tasked with navigating an obstructed track.

Panel of Figure 5 — Fig. 5 : Efficiency of explanation generation algorithms as a function of the number of obstacles ( $10$ – $14$ ), measured by the number of simulation runs (lower is better).

		10		11		12		13		14
	Env.	Disc.	Dist.	Disc.	Dist.	Disc.	Dist.	Disc.	Dist.	Disc.	Dist.
LiteRacer	Sin-SN	0.21	0.29	0.15	0.35	0.10	0.45	0.19	0.40	0.14	0.30
	Sin-SW	0.19	0.35	0.17	0.32	0.12	0.29	0.12	0.24	0.13	0.31
	Sin-LN	0.21	0.36	0.17	0.31	0.20	0.35	0.18	0.32	0.08	0.30
	Sin-LW	0.13	0.35	0.15	0.35	0.14	0.32	0.10	0.23	0.17	0.33
	Seg-SN	0.19	0.40	0.20	0.33	0.13	0.42	0.19	0.40	0.17	0.38

TABLE I : AUC of explanation coverage for the discrete and distance-weighted responsibility rankings (higher is better).

Panel of Figure 7 — Fig. 6 : Multiple explanations coverage as a function of the fraction of obstacles considered, ordered by the discrete and distance-weighted responsibility rankings (higher is better).

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 23 — Fig. 22 : Execution time as a function of the number of simulation runs.

Panel of Figure 24 — Fig. 23 : Efficiency of explanation generation algorithms as a function of the number of obstacles ( $10$ – $14$ ), measured by the number of simulation runs (lower is better).

		10		11		12		13		14
	Env.	Disc.	Dist.	Disc.	Dist.	Disc.	Dist.	Disc.	Dist.	Disc.	Dist.
LiteRacer	Sin-SN	0.21	0.29	0.15	0.35	0.10	0.45	0.19	0.41	0.14	0.30
	Sin-SW	0.19	0.35	0.17	0.31	0.12	0.32	0.12	0.26	0.13	0.32
	Sin-LN	0.21	0.36	0.17	0.33	0.20	0.38	0.18	0.33	0.08	0.30
	Sin-LW	0.13	0.34	0.15	0.38	0.14	0.35	0.10	0.24	0.17	0.36
	Seg-SN	0.19	0.39	0.20	0.34	0.13	0.44	0.19	0.40	0.17	0.37

Appendix

TABLE II : AUC results for the discrete and distance-based rankings across different environments and numbers of obstacles.

Causal Explanation

Causal explanation uses causal models and interventions to identify why outcomes or model predictions occur.

See all 26 papers

Jun 23, 2026

Explaining Failures of Cyber-Physical Systems with Actual Causality

Modern autonomous Cyber-Physical Systems (CPSs), such as self-driving cars, face increasingly complex demands, and yet are expected to act reliably. The black-box nature often characterizing such systems, especially those relying on neural components, makes it impossible to fully verify the system behavior prior to deployment. Unfortunately, unexpected failures-when the system does not comply with its specification-are inevitable and may have catastrophic implications. To improve trust in the system and facilitate future mitigation after a failure occurs, it is important to try to derive an explanation for the unexpected system behavior. This paper introduces the novel concept of leveraging the framework of actual causality for CPS failure explanation. Up until now, this framework was only used to derive explanations in the context of simple systems, such as image classifiers. This paper addresses the theoretical gaps and provides the guidance needed to allow for correct explanation derivation in the CPS domain. Beyond the theoretical contribution, the paper presents two novel, practical, system-agnostic explanation derivation algorithms, allowing to prioritize either explanation optimality or derivation efficiency. The approach is demonstrated and evaluated in the context of a neural-network-controlled autonomous car, designed to avoid collisions.

Khen Elimelech, Tom Yaacov, David A. Kelly +2

King’s College London · Rice University

Agent Failure Analysis Cyber-Physical Systems Actual Causality Causal Reasoning Causal Explanation

Ask PDF

Apr 9, 2026

Towards Counterfactual Explanation and Assertion Inference for CPS Debugging

Verification and validation of cyber-physical systems (CPS) via large-scale simulation often surface failures that are hard to interpret, especially when triggered by interactions between continuous and discrete behaviors at specific events or times. Existing debugging techniques can localize anomalies to specific model components, but they provide little insight into the input-signal values and timing conditions that trigger violations, or the minimal, precisely timed changes that could have prevented the failure. In this article, we introduce DeCaF, a counterfactual-guided explanation and assertion-based characterization framework for CPS debugging. Given a failing test input, DeCaF generates counterfactual changes to the input signals that transform the test from failing to passing. These changes are designed to be minimal, necessary, and sufficient to precisely restore correctness. Then, it infers assertions as logical predicates over inputs that generalize recovery conditions in an interpretable form engineers can reason about, without requiring access to internal model details. Our approach combines three counterfactual generators with two causal models, and infers success assertions. Across three CPS case studies, DeCaF achieves its best success rate with KD-Tree Nearest Neighbors combined with M5 model tree, while Genetic Algorithm combined with Random Forest provides the strongest balance between success and causal precision.

Zaid Ghazal, Hadiza Yusuf, Khouloud Gaaloul

University of Michigan-Dearborn, Dearborn, MI, USA

Cyber-Physical Systems Counterfactual Explanations Causal Counterfactual Generation

Ask PDF

Oct 21, 2026

Sufficient and Necessary Causal Explanations for the Analysis of Autonomous Systems

Understanding failures of autonomous systems is challenging, particularly when their behavior depends on black-box components such as neural networks. In this work, we make a distinction between two objectives of failure analysis: reproduction, which seeks to identify conditions sufficient to reproduce an observed failure, and avoidance, which seeks to identify the changes required to prevent it. Previous work on causal explanations of system failures is based on the concept of sufficient explanations and naturally supports reproduction. Nevertheless, we observe that it does not address the problem of avoiding the failure. In this paper, we introduce and formalize the concept of necessary explanations for system analysis as the correct formalism for failure avoidance. We place necessary explanations in the framework of actual causality and show that, together with sufficient explanations, they provide a unified causal framework for addressing both objectives of failure analysis. We present an efficient approximation algorithm for generating necessary explanations based on the notion of causal responsibility. We evaluate the efficiency of our approach across multiple environments. We demonstrate the practical utility of necessary explanations for failure avoidance through counterfactual generation and show that it outperforms existing methods.

Tom Yaacov, Khen Elimelech, Hana Chockler +1

King’s College London · Rice University

Causal Explanation

Ask PDF

Abstract

Causal Explanation

Explore similar work

Explaining Failures of Cyber-Physical Systems with Actual Causality

Towards Counterfactual Explanation and Assertion Inference for CPS Debugging

Sufficient and Necessary Causal Explanations for the Analysis of Autonomous Systems