Explainable Reinforcement Learning

Explainable Reinforcement Learning (XRL) aims to make the decision-making processes of reinforcement learning (RL) agents more transparent and understandable. Current research focuses on developing methods that provide both local (explaining single actions) and global (explaining overall behavior) explanations, often employing techniques like reward decomposition, counterfactual analysis, and interpretable model architectures such as decision trees. This work is crucial for building trust in RL systems, particularly in high-stakes applications like healthcare and finance, and for facilitating debugging and improved human-agent collaboration.

Papers

January 7, 2025

December 26, 2024

xSRL: Safety-Aware Explainable Reinforcement Learning -- Safety as a Product of Explainability
Risal Shahriar Shefin, Md Asifur Rahman, Thai Le, Sarra Alqahtani
Reinforcement Learning High Explainability Safe Reinforcement Learning Human SAFETY Reinforcement Learning Agent Product Attribute Explainable Reinforcement Learning

November 20, 2024

Explainable Finite-Memory Policies for Partially Observable Markov Decision Processes
Muqsit Azeem, Debraj Chakraborty, Sudeep Kanav, Jan Kretinsky
Markov Decision Process Partial Observability Explainable Reinforcement Learning

November 11, 2024

'Explaining RL Decisions with Trajectories': A Reproducibility Study
Karim Abdel Sadek, Matteo Nulli, Joan Velja, Jort Vincenti
Handwritten Trajectory Decision Relevant Information Explainable Reinforcement Learning Reproducibility Study Attribution Analysis

August 26, 2024

ReLExS: Reinforcement Learning Explanations for Stackelberg No-Regret Learners
Xiangge Huang, Jingyuan Li, Jiaqing Xie
Regret Minimization Stackelberg Game Regret Algorithm Explainable Reinforcement Learning Stackelberg Equilibrium Regret Learning

June 21, 2024

KnobTree: Intelligent Database Parameter Configuration via Explainable Reinforcement Learning
Jiahan Chen, Shuhan Qi, Yifan Li, Zeyu Dong, Mingfeng Ding, Yulin Wu, Xuan Wang
Explainable Reinforcement Learning Automatic Configuration Database Knob Tuning

April 21, 2024

Reinforcement of Explainability of ChatGPT Prompts by Embedding Breast Cancer Self-Screening Rules into AI Responses
Yousef Khan, Ahmed Abdeen Hamed
Generative AI High Explainability Reasoning Capability Positive Reinforcement Natural Language Reasoning Explainable Reinforcement Learning Cancer Screening Automatic Prompt Engineering ChatGPT Prompt

April 16, 2024

Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshop
Hector Kohler, Quentin Delfosse, Paul Festor, Philippe Preux
Reinforcement Learning Explainable Reinforcement Learning Research Community Interpretable Policy Reinforcement Learning Paradigm

March 18, 2024

Distill2Explain: Differentiable decision trees for explainable reinforcement learning in energy application controllers
Gargya Gokhale, Seyed Soroush Karimi Madahi, Bert Claessens, Chris Develder
Explainable Reinforcement Learning Data Driven Control Energy Flexibility Demand Side Interpretable Control Energy Efficient Control Differentiable Decision Tree

February 20, 2024

XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniques
Yu Xiong, Zhipeng Hu, Ye Huang, Runze Wu, Kai Guan, Xingchen Fang, Ji Jiang, Tianze Zhou, Yujing Hu, Haoyu Liu, Tangjie Lyu, Changjie Fan
Reinforcement Learning New Benchmark Explainable AI Explanation Method MT Bench Explainable Reinforcement Learning

December 30, 2023

Causal State Distillation for Explainable Reinforcement Learning
Wenhao Lu, Xufeng Zhao, Thilo Fryen, Jae Hee Lee, Mengdi Li, Sven Magg, Stefan Wermter
Reinforcement Learning Reward Function Causal Learning Explainable Reinforcement Learning rEward Decomposition Causal Feature Distillation

December 18, 2023

Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomes
Yotam Amitai, Yael Septon, Ofra Amir
Reinforcement Learning Local Explanation Explainable Reinforcement Learning Counterfactual Feature rEward Decomposition

December 13, 2023

Personalized Decision Supports based on Theory of Mind Modeling and Explainable Reinforcement Learning
Huao Li, Yao Fan, Keyang Zheng, Michael Lewis, Katia Sycara
Theoretical Understanding Counterfactual Explanation Explainable Reinforcement Learning Personalized Intervention Personalized Decision Action Recommendation

December 7, 2023

CODEX: A Cluster-Based Method for Explainable Reinforcement Learning
Timothy K. Mathes, Jessica Inman, Andrés Colón, Simon Khan
Reinforcement Learning Counterfactual Explanation Reinforcement Learning Agent Agent Behavior State Action Space Explainable Reinforcement Learning Clustering Based Approach Game Representation

November 27, 2023

Utilizing Explainability Techniques for Reinforcement Learning Model Assurance
Alexander Tapley, Kyle Gatesman, Luis Robaina, Brett Bissey, Joseph Weissman
Deep Reinforcement Learning Explainable Reinforcement Learning Safe Policy Explanation Map

July 9, 2023

A User Study on Explainable Online Reinforcement Learning for Adaptive Systems
Andreas Metzger, Jan Laufer, Felix Feit, Klaus Pohl
Deep Reinforcement Learning Reinforcement Learning Algorithm Online Reinforcement Learning User Study Explainable Reinforcement Learning Adaptive System

May 17, 2023

Pittsburgh Learning Classifier Systems for Explainable Reinforcement Learning: Comparing with XCS
Jordan T. Bishop, Marcus Gallagher, Will N. Browne
Reinforcement Learning Explainable AI Explainable Reinforcement Learning Learning Classifier Evolutionary Machine Learning

May 6, 2023

Explaining RL Decisions with Trajectories
Shripad Vilasrao Deshmukh, Arpan Dasgupta, Balaji Krishnamurthy, Nan Jiang, Chirag Agarwal, Georgios Theocharous, Jayakumar Subramanian
Reinforcement Learning Explainable AI Action Space Handwritten Trajectory Explainable Reinforcement Learning Artificial Intelligence Decision

May 4, 2023

Explainable Reinforcement Learning via a Causal World Model
Zhongwei Yu, Jingqing Ruan, Dengpeng Xing
Reinforcement Learning Causal Structure Causal Model Explainable Reinforcement Learning Causal World Model