cs.LGOct 30, 2026

CausalXRL: Explainable Reinforcement Learning through Causal Graph Reasoning

Authors: Yanming Zhang, Eric Papenhausen, Klaus Mueller

Organizations: Department of Computer Science, Stony Brook University, Stony Brook, USA

Abstract

Reinforcement learning is a powerful paradigm for training autonomous agents and has achieved impressive performance in complex environments. However, this success often comes at the cost of interpretability, diminishing trust and complicating efforts to debug and improve agent behavior. To address these challenges, we introduce CausalXRL, a novel framework for explainable reinforcement learning (XRL). A key feature of CausalXRL is its use of causal graph reasoning, which provides transparent, structured, multi-level explanations of agent decision-making. We validate CausalXRL through comprehensive case studies and a two-part evaluation: (1) a quantitative analysis of explanation fidelity and causal-structure learning efficiency in benchmark RL environments, and (2) a qualitative expert study assessing explainability in the real-time strategy (RTS) benchmark MicroRTS. The quantitative results show that CausalXRL can provide faithful explanations while efficiently learning causal structures, and the qualitative expert study suggests that participants found CausalXRL useful for inspecting high-level RTS strategies.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents

    Aug 18, 2026Ram Rachum, Yotam Amitai, Bálint Gyevnár +2Explainability EvaluationInterpretability in RL

  2. BXRL: Behavior-Explainable Reinforcement Learning

    Mar 24, 2026Ram Rachum, Yotam Amitai, Yonatan Nakar +2Interpretability in RL

  3. Explaining Reinforcement Learning Agents via Inductive Logic Programming

    Jul 15, 2026Celeste Veronese, Edoardo Zorzi, Daniele Meli +1Interpretability in RLExplainability Evaluation