BXRL: Behavior-Explainable Reinforcement Learning

Ram Rachum; Yotam Amitai; Yonatan Nakar; Reuth Mirsky; Cameron Allen

cs.LGMar 24, 2026

BXRL: Behavior-Explainable Reinforcement Learning

Authors: Ram Rachum, Yotam Amitai, Yonatan Nakar, Reuth Mirsky, Cameron Allen

Organizations: University of California, Berkeley · Tufts University · Independent Researcher · Google Research

Abstract

A major challenge of Reinforcement Learning is that agents often learn undesired behaviors that seem to defy the reward structure they were given. Explainable Reinforcement Learning (XRL) methods can answer queries such as "explain this specific action", "explain this specific trajectory", and "explain the entire policy". However, XRL lacks a formal definition for behavior as a pattern of actions across many episodes. We provide such a definition, and use it to enable a new query: "Explain this behavior". We present Behavior-Explainable Reinforcement Learning (BXRL), a new problem formulation that treats behaviors as first-class objects. BXRL defines a behavior measure as any function $m : Π\to \mathbb{R}$ , allowing users to precisely express the pattern of actions that they find interesting and measure how strongly the policy exhibits it. We define contrastive behaviors that reduce the question "why does the agent prefer $a$ to $a'$ ?" to "why is $m(π)$ high?" which can be explored with differentiation. We do not implement an explainability method; we instead analyze three existing methods and propose how they could be adapted to explain behavior. We present a port of the HighwayEnv driving environment to JAX, which provides an interface for defining, measuring, and differentiating behaviors with respect to the model parameters.

Figures & tables

Figure 1: HighJax TUI for defining behavior scenarios by selecting states and actions from existing rollouts.

Figure 2: Left: Return breakdown of a PPO agent training on HighJax. Training is deliberately slowed down to

D_{\mathrm{KL}}=5\times 10^{-4}

per epoch. Right: Value of the collision behavior measure

m_{c}

across training.

Figure 3: Six scenarios defining the collision behavior measure $m_{c}$ . Each causes the ego car (dark gray) to crash within two timesteps. Columns correspond to the action taken: left , right , or faster .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Table 4: Observation matrices for the six scenarios defining the collision behavior measure

m_{c}

(Eq. 4 ). Each matrix

\mathbf{O}\in\mathbb{R}^{5\times 5}

contains the ego vehicle (absolute) and four nearest NPCs (relative). Columns: presence, longitudinal position, lateral position, longitudinal velocity, lateral velocity.

Interpretability in RL

Interpretability in reinforcement learning makes learned policies, decisions, and internal representations understandable and auditable.

See all 70 papers

Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents
Aug 18, 2026Ram Rachum, Yotam Amitai, Bálint Gyevnár +2Explainability Evaluation Interpretability in RL
University of California, Berkeley · Tufts University · Independent Researcher +1
Explaining Reinforcement Learning Agents via Inductive Logic Programming
Jul 15, 2026Celeste Veronese, Edoardo Zorzi, Daniele Meli +1Interpretability in RL Explainability Evaluation
University of Verona, Italy · Sapienza University of Rome, Italy
A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs
Jul 16, 2025Léo SaulièresInterpretability in RL Explainable Artificial Intelligence
∗ IMT Mines Albi, Albi, France

Aug 18, 2026J/K move · Enter open · S save

Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents

Ram Rachum, Yotam Amitai, Bálint Gyevnár, Reuth Mirsky+1

University of California, Berkeley · Tufts University · Independent Researcher+1

Explainability Evaluation Interpretability in RL Reinforcement Learning AI Agent Benchmarks

Open paper AskPDF arXiv

This preliminary paper outlines a planned evaluation benchmark for Explainable Reinforcement Learning (XRL) methods. Current evaluations rely on functionally-grounded metrics like faithfulness and compactness, and on human-grounded proxies like subjective ratings or prediction accuracy. We suggest evaluating XRL methods by how effectively their generated explanations help to diagnose and fix malfunctioning reinforcement learning (RL) agents. We propose EvalXRL, a benchmark in which a Large Language Model (LLM) coding agent uses different XRL methods to diagnose a held-out malfunction in an RL agent, and then repair it. Our proposed benchmark iterates across (environment

\times

malfunction

\times

XRL method) tuples and uses the reward signal of the RL agents to form a final score for each XRL method. The coding agent may use the method interactively: invoke the XRL method, process its output, form new hypotheses on what is broken, and invoke the method again with parameters adjusted for testing these hypotheses. This closed-loop structure may be described as a simplified version of the scientific method. Some XRL methods provide self-evaluations that follow this pattern; we propose the first head-to-head comparison of multiple XRL methods in closed-loop usage.

Abstract

Interpretability in RL

Explore similar work

Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents

Explaining Reinforcement Learning Agents via Inductive Logic Programming

A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs