Interpretability in RL

RL: Reinforcement Learning

Ask about this topic

Overview

Interpretability in reinforcement learning studies how to understand what an agent has learned, why it takes particular actions, and whether its behavior is safe and reliable. Some methods build readable policies directly, using decision trees, symbolic rules, programs, or concept-based representations; others distill neural policies into simpler models. Post-hoc approaches use feature attribution, attention analysis, probing, and causal interventions, while formal verification checks properties such as safety and failure probabilities. Work spans settings from Atari and continuous control to healthcare and robotics, but a central challenge is balancing explanation simplicity and faithfulness with policy performance, and establishing whether explanations help people detect or fix real failures.

Latest papers

CardsList

71 papers8 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

  1. Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models

    Mar 10, 2026Daniel Hennes, Zun Li, John Schultz +1Multi-Agent Reinforcement LearningInterpretability in RL

    Google DeepMind

  2. COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenance

    Mar 8, 2026Dennis GrossFormal Methods for RLInterpretability in RL

  3. BioLLMAgent: A Hybrid Framework with Enhanced Structural Interpretability for Simulating Human Decision-Making in Computational Psychiatry

    Mar 5, 2026Zuo Fei, Kezhi Wang, Xiaomin Chen +1Computational Cognitive ModelingMental Health

    School of Computer Science and Technology, East China Normal University, 3663 North Zhongshan Road, Shanghai 200062, China · Department of Computer Science, Brunel University of London, UB8 3PH, UK · Department of Computer Science, University of Reading, RG6 6UR, UK

  4. Prism: Policy Reuse via Interpretable Strategy Mapping in Reinforcement Learning

    Mar 4, 2026Thomas PravetzRL for Game AIZero-Shot Policy Transfer

  5. COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Management

    Mar 2, 2026Dennis GrossInventory ControlReinforcement Learning

  6. Distilling Deep Reinforcement Learning into Interpretable Fuzzy Rules: An Explainable AI Framework

    Feb 24, 2026Sanup S. Araballi, Simon Khan, Chilukuri K. MohanInterpretability in RLKnowledge Distillation

    Dept. of EECS, Syracuse University, NY 13244, USA · Air Force Research Laboratory, Rome, NY 13441, USA

  7. IR3^3: Contrastive Inverse Reinforcement Learning for Interpretable Detection and Mitigation of Reward Hacking

    Feb 23, 2026Mohammad Beigi, Ming Jin, Junshan Zhang +3Reward HackingReinforcement Learning

    UC Davis · Virginia Tech · Salesforce +1

  8. Formally Verifying and Explaining Sepsis Treatment Policies with COOL-MC

    Feb 16, 2026Dennis GrossFormal Methods for RLFormal Verification

  9. Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning

    Nov 25, 2025Charlotte Beylier, Hannah Selder, Arthur Fleig +2Reinforcement LearningInterpretability in RL

    Max Planck Institute for Human Cognitive and Brain Sciences · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig · Leipzig, Germany +2

  10. A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs

    Jul 16, 2025Léo SaulièresInterpretability in RLExplainable Artificial Intelligence

    ∗ IMT Mines Albi, Albi, France

  11. Policy Learning with a Language Bottleneck

    May 7, 2024Megha Srivastava, Cedric Colas, Dorsa Sadigh +1Policy LearningLLM-Guided RL

    Stanford University · Massachusetts Institute of Technology Inria