cs.LGMar 30, 2026

Position: Explainable AI is Causality in Disguise

Authors: Amir-Hossein Karimi

Organizations: Department of Electrical & Computer Engineering, University of Waterloo & Vector Institute, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The demand for Explainable AI (XAI) has triggered an explosion of methods, producing a landscape so fragmented that we now rely on surveys of surveys. Yet, fundamental challenges persist: conflicting metrics, failed sanity checks, and unresolved debates over robustness and fairness. The only consensus on how to achieve explainability is a lack of one. This has led many to point to the absence of a ground truth for defining ``the'' correct explanation as the main culprit. This position paper posits that the persistent discord in XAI arises not from an absent ground truth but from a ground truth that exists, albeit as an elusive and challenging target: the causal model that governs the relevant system. By reframing XAI queries about data, models, or decisions as causal inquiries, we prove the necessity and sufficiency of causal models for XAI. We contend that without this causal grounding, XAI remains unmoored. Ultimately, we encourage the community to converge around advanced concept and causal discovery to escape this entrenched uncertainty.

Figures & tables

Figure 1: Core methods in XAI for explaining an ML model ( f:X→Yf:X\rightarrow Y ) are categorized by purpose into data -based, model -based, and decision -based questions. By mapping these directly onto Pearl’s Ladder of Causation, we reveal that solving XAI fundamentally requires answering causal inquiries.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

  QuestionApproachReferences
Causal Abstraction & Representation LearningRQ1How can we systematically identify and extract human-interpretable concepts from deep neural networks (DNNs) to construct meaningful causal abstractions?Develop unsupervised or semi-supervised algorithms leveraging neuro-symbolic methods to identify concept-level abstractions from latent spaces.( Geiger et al., 2023 ; Marconato et al., 2023b ; Zhang, 2024 )
RQ2Under what conditions can we guarantee that causal abstractions discovered from low-level neural representations remain faithful to the original neural network’s causal structure?Study theoretical guarantees for causal abstraction under approximate causal modeling frameworks. Quantify trade-offs between abstraction fidelity and interpretability.( Beckers and Halpern, 2019 ; Karimi et al., 2023 ; Rubenstein et al., 2017 )
Approximate and Partial Causal ModelsRQ3How can we formulate and efficiently compute approximate causal explanations that balance computational complexity with explanation accuracy?Formalize a notion of ϵ\epsilon -approximate causal explanations and develop scalable algorithms to derive such explanations from large-scale models.( Glymour et al., 2019 ; Montagna et al., 2024 )
RQ4Can we establish rigorous bounds on how approximations in causal discovery methods affect downstream XAI tasks such as model debugging or user trust?Empirically and theoretically analyze sensitivity to approximation errors; develop robustness metrics for causal XAI methods.( Janzing et al., 2020 )
RQ5What computational trade-offs arise when using approximate causal models in realistic XAI scenarios?Study whether relaxing accuracy requirements yields tangible computational benefits or if computational complexity remains a core barrier.( Adolfi et al., 2024 ; Barceló et al., 2020 ; Bassan et al., 2024 )
Human-Aligned Causal Queries & Interactive XAIRQ6How should interactive XAI systems leverage causal interventions to enhance users’ mental models about AI systems?Design and validate interactive interfaces (e.g., visual tools or agents) enabling causal interventions, and assess user understanding.( Miller, 2019 )
Appendix
Table 2
More in this topic

Explainable Artificial Intelligence

Explainable artificial intelligence develops ways to make AI decisions understandable, testable, and useful to people.

See all 320 papers

Explore similar work

CardsList
Sep 22, 2026

xWhyL: Causal Interactive Learning

Explanations are central to causal reasoning, and cognitive science has long established that the human drive to explain is itself a mechanism for learning about causality. Despite this, learning from those abductive signals is largely ignored in artificial intelligence. While explainable AI (XAI) increasingly draws on causal models to generate explanations, the converse direction about what explanations can do for causality remains largely unexplored. To fill this gap, we propose xWhyL, a formal framework connecting causality and XAI by learning causal models from explanations. We develop a mathematical theory that translates explanations into a learning signal complementary to observational data, and demonstrate how it enables overcoming the limits of observational causal discovery. As explanations can be derived from incorrect beliefs and clash with data, a tension we call the Causal Tug-of-War, we prove conditions under which our framework rejects misspecified explanations rather than absorbing them. Our practical instantiation, Causal Interactive Learning (CIL), shows how expert explanations can efficiently support causal discovery and distinguish correct from incorrect explanations.

Nicholas Tagliapietra, Florian Peter Busch, Moritz Willig +4

Bosch Center for Artificial Intelligence, Germany · Computer Science Department, Technical University of Darmstadt, Germany · Hessian Center for AI (hessian.AI), Darmstadt, Germany

Causal DiscoveryCausal Structure Learning
AskPDF
May 9, 2026

Reconciling Consistency-Based Diagnosis with Actual-Causality-Based Explanations

We establish, from the point of view of Explainable AI (XAI), connections between Consistency-Based Diagnosis (CBD), on one side, and Actual Causality and Causal Responsibility, on the other. CBD has received little attention from the XAI community. Connections between these two areas could have a fruitful impact on XAI and Explainable Data Management.

Leopoldo Bertossi

Carleton University, Canada · IMFD, Chile · Emeritus Professor. Adjunct Professor Western University, Canada.

Explainable Artificial IntelligenceActual CausalityCausal ReasoningCausal Explanation
AskPDF
Apr 30, 2026

CoAX: Cognitive-Oriented Attribution eXplanation User Model of Human Understanding of AI Explanations

Explainable AI (XAI) aims to improve user understanding and decisions when using AI models. However, despite innovations in XAI, recent user evaluations reveal that this goal remains elusive. Understanding human cognition can help explain why users struggle to effectively use AI explanations. Focusing on reasoning on structured (tabular) data, we examined various reasoning strategies for different XAI methods (none, feature importance, feature attribution) in the decision task of anticipating AI decisions (i.e., forward simulation). We i) elicited reasoning strategies from a formative user study, and ii) collected decisions from a summative user study. Using cognitive modeling, we implemented the processes underlying each reasoning strategy and evaluated their alignment with human decision-making. We found that our models better fit human decisions than baseline machine learning proxies, providing insights into which reasoning strategies are (in)effective. We then demonstrate how the fitted model can be used to form hypotheses and investigate research questions that are costly to study with real human participants. This work contributes to debugging human understanding of XAI, informing the future development of more usable and interpretable AI explanations.

Louth Bin Rawshan, Zhuoyu Wang, Brian Y. Lim

National University of Singapore, Singapore

Feature AttributionExplainable Artificial IntelligenceHuman-AI Decision MakingExplainability Evaluation
AskPDF