Interpretability in RL

RL: Reinforcement Learning

Interpretability in reinforcement learning studies how to understand what an agent has learned, why it takes particular actions, and whether its behavior is safe and reliable. Some methods build readable policies directly, using decision trees, symbolic rules, programs, or concept-based representations; others distill neural policies into simpler models. Post-hoc approaches use feature attribution, attention analysis, probing, and causal interventions, while formal verification checks properties such as safety and failure probabilities. Work spans settings from Atari and continuous control to healthcare and robotics, but a central challenge is balancing explanation simplicity and faithfulness with policy performance, and establishing whether explanations help people detect or fix real failures.

Latest papers

71 papers8 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

Interpretable Meta-Policies for Neural Decision Making
Oct 27, 2026Raban Emunds, Jannis Blüml, Quentin Delfosse +1Interpretability in RL Neuro-Symbolic RL
Department of Computer Science, Technical University Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI) · Centre for Cognitive Science, Darmstadt +1
Reason-guided Human Reward Modeling for Preference-based Reinforcement Learning
Oct 16, 2026Anshul Nayak, Shahil Shaik, Jonathan Smereka +1Preference-Based RL Reward Modeling
Clemson University, Clemson, SC 29631 USA · U.S. Army DEVCOM Ground Vehicle Systems Center, Warren, MI 48092 USA
The Thinking Leakage in Reinforcement Learning for LLMs
Oct 9, 2026Wenxuan Zhu, Shuming Liu, Bang An +4RL for Language Model Reasoning RL Post-Training
King Abdullah University of Science and Technology · Shanghai Jiao Tong University
Visualizing the World Model in an Agent's Value Function
Oct 9, 2026Shawheen Naderi, Fatemeh Akherati, Yubei ChenWorld Models Interpretability in RL
University of California, Davis
Temporally Interpretable Differentiable Decision Trees
Oct 7, 2026Eisuke Hirota, Aarav Sane, Rohan PalejaInterpretable ML Interpretability in RL
Department of Computer Science, Purdue University · Thomas Lord Department of Computer Science, University of Southern California
Towards the Automatic Synthesis of Interpretable Chess Tactics
Oct 6, 2026Abhijeet Krishnan, Chris MartensGame-Playing Agents Chess
North Carolina State University Venture IV, 1730 Varsity Dr, Raleigh, NC 27606
Learning Explainable Representations of Complex Game-playing Strategies
Oct 6, 2026Abhijeet Krishnan, Colin M. Potts, Arnav Jhala +3Game-Playing Agents Program Synthesis
Venture IV 420, North Carolina State University, Raleigh, NC 27606 USA · Department of Aerospace Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, Maharashtra, India · Tata Research Development and Design Centre, Pune 411013, Maharashtra, India +1
Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football
Oct 5, 2026Kenjiro Ide, Taiga Someya, Kohei Kawaguchi +1Sequential Decision Making Interpretability in RL
Graduate School of Informatics, Nagoya University, Nagoya, Japan. · Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan. · Department of Economics, The Hong Kong University of Science and Technology, Hong Kong, China. +1
Escaping Local Views: Discovering Latent Concepts for Interpretable Multi-Agent Reinforcement Learning
Sep 28, 2026Yijie Sun, Sanquan Sun, Yanda Zhu +3Reinforcement Learning Partially Observable RL
Nanjing University · Hong Kong Polytechnic University · Shenzhen University
Self-discovering RL in the Era of Experience: Is Learning History an Asset or a Burden?
Sep 27, 2026Haomin LuoReinforcement Learning Interpretability in RL
University of Cambridge · Models2 AI
Learning and interpreting policies for simultaneous entanglement requests in quantum networks
Sep 24, 2026Leon Rode, Sumeet Khatri, Supartha PodderInterpretability in RL
Department of Computer Science, Stony Brook University · Department of Computer Science and Center for Quantum Information Science and Engineering, Virginia Tech
Auditability Is Not One Property: Rule Overlap, Behavioural Agreement, and Composition in Reinforcement Learning
Sep 23, 2026Liu Hung MingReinforcement Learning AI Agent Auditing
PARRAWA AI
Integrating Fairness and Explainability in a Multiple Instance Reinforcement Learning System
Sep 2, 2026Bente Hinkenhuis, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansoor AlsahagAlgorithmic Fairness Debiased ML
University of Amsterdam Amsterdam, The Netherlands
ShuttleArena: Interpretable Self-Play in Physics-Based Badminton
Aug 26, 2026Peize DingGame-Playing Agents Self-Play RL
Columbia University New York, NY 10027, USA
Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents
Aug 18, 2026Ram Rachum, Yotam Amitai, Bálint Gyevnár +2Explainability Evaluation Interpretability in RL
University of California, Berkeley · Tufts University · Independent Researcher +1
Command-Space Counterfactual Explanations for Pareto-Conditioned Reinforcement Learning
Aug 15, 2026Joanikij Chulev, Hendrik BaierInterpretability in RL Counterfactual Learning
Centrum Wiskunde & Informatica (CWI), Amsterdam · Eindhoven University of Technology
XAI-Guided Conservative Decentralized Execution for Offline Multi-Agent Network Slicing
Aug 14, 2026Eslam Eldeeb, Hatim Chergui, Merouane DebbahNetwork Slicing Wireless Resource Allocation
Centre for Wireless Communications (CWC), University of Oulu, Finland · i2CAT Foundation, Spain · Research Institute for Digital Future, Khalifa University, 127788 Abu Dhabi, UAE
Control-Diverse Reinforcement Fine-Tuning: Decoupling the Shared Control Bottleneck of RL Post-Training
Aug 8, 2026Binwen Tan, Jingchao Wang, Dengzhe Hou +6RL for Language Model Reasoning Multi-Task RL
1Graduate School of Information Sciences, Tohoku University · School of Computer Science, Peking University · 2Unprecedented-scale Data Analytics Center, Tohoku University +3
Interpretable reinforcement learning with decision-tree pruning
Aug 7, 2026Mark Leon Ringer, Michel TokicInterpretability in RL
Faculty of Mathematics, Informatics and Statistics, Ludwig-Maximilians-University Munich, Munich, Germany · Siemens AG, Data & Artificial Intelligence, Otto-Hahn-Ring 6, 81739 Munich, Germany
Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping
Aug 6, 2026Vaishnav Vaidheeswaran, Dilith Jayakody, Biruk Ambaw +3Reinforcement Learning Meta-Reinforcement Learning
Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada · Department of Industrial Engineering, Faculty of Engineering, Dalhousie University, Halifax, NS, Canada
dtControl2+ $\varepsilon$ : Trading Optimality for Explainability in MDPs via Decision Trees
Jul 28, 2026Tereza Kinská, Jan Křetínský, Tobias Meggendorfer +2Markov Decision Processes Markov Models
Masaryk University, Brno, Czech Republic · Technical University of Munich, Munich, Germany · Lancaster University Leipzig, Leipzig, Germany +1
SPOTting the Future: Lookahead Explanations for Deep Reinforcement Learning
Jul 28, 2026Tamar Gozlan, Claudia V. GoldmanInterpretable ML Interpretability in RL
Benin School of Computer Science and Engineering, The Hebrew University of Jerusalem · Hebrew University Business School, Data Science Department, The Hebrew University of Jerusalem
Interpretable GOHR Agents via Sparse Autoencoders
Jul 27, 2026Shiwei Tan, Yusong Zhao, Weiyi Qin +6Transformer Interpretability Sparse Autoencoders
1Rutgers University · 3Paul B Kantor Consultant · University of Illinois Urbana-Champaign
Explainable Reinforcement Learning via Physics-Aware Policy Distillation
Jul 27, 2026Shaker Al-Tamari, Waled KadourRL Control Interpretability in RL
Dept. of Electrical and Computer Engineering RPTU Kaiserslautern-Landau Kaiserslautern, Germany
Sparse Gaussian-Mixture-Model Q-Functions via Hadamard Overparametrization for Online Reinforcement Learning
Jul 26, 2026Minh Vu, Konstantinos SlavakisGaussian Mixture Models Interpretability in RL
Institute of Science Tokyo, Department of Information and Communications Engineering, Yokohama, Japan.
Explainable Reinforcement Learning for assisting Air Traffic Controllers
Jul 24, 2026Anduel Mehmeti, Gabriella Gigante, Salvatore VenticinqueReinforcement Learning Intelligent Transportation Systems
Department of Engineering, University of Campania "Luigi Vanvitelli", Italy · CIRA, Italy
Emergent Compositional Skills in Mixture-of-Experts VLAs
Jul 22, 2026Shlok Shah, Rhiaan Jhaveri, Tharun Kumar Tiruppali Kalidoss +3Robot Policy Learning Robot Skill Learning
Department of Computer Science, Princeton University, Princeton, NJ, USA · Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ, USA
Explaining Reinforcement Learning Agents via Inductive Logic Programming
Jul 15, 2026Celeste Veronese, Edoardo Zorzi, Daniele Meli +1Interpretability in RL Explainability Evaluation
University of Verona, Italy · Sapienza University of Rome, Italy
ORCAID: Oblique Rule-Based Continuous-Action Interpretation for Deep RL Policies
Jul 8, 2026Ignacio D. Lopez-Miguel, Ezio Bartocci, Thomas Eiter +1Reinforcement Learning Interpretable ML
TU Wien Vienna, Austria
Explainable Reinforcement Learning for Adaptive Traffic Signal Control
Jul 4, 2026Dickens Kwesiga, Nishu Choudhary, Angshuman Guin +1Reinforcement Learning Traffic Signal Control
School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA

Oct 27, 2026J/K move · Enter open · S save

Interpretable Meta-Policies for Neural Decision Making

Raban Emunds, Jannis Blüml, Quentin Delfosse, Kristian Kersting

Department of Computer Science, Technical University Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI) · Centre for Cognitive Science, Darmstadt+1

Interpretability in RL Neuro-Symbolic RL Hierarchical RL

Open paper AskPDF arXiv

Deep Reinforcement Learning remains difficult to interpret and align due to the black-box nature of neural networks. While (neuro-)symbolic approaches improve transparency by incorporating symbolic reasoning, applying them directly to low-level actions results in overly complex, unreadable policies. We introduce NEXUS, a hierarchical RL framework that resolves this tension by combining neural skills with symbolic meta-policies. By shifting transparent reasoning to high-level skills rather than atomic actions, NEXUS drastically reduces symbolic complexity without sacrificing environment performance. Object-centric observations enable high-level reasoning and the design of reward functions and meta-policies by interaction with LLMs. We demonstrate that NEXUS agents learn disentangled skills, are interpretable regarding high-level goals, are better aligned to goals described in the game manual, less prone to reward hacking and more robust to game simplifications than deep RL. Finally, we isolate and evaluate the impact of different meta-policy architectures (neural, symbolic, neuro-symbolic) on environmental return and aligned task performance.