cs.LGSep 19, 2026

Triggers and Diagnostics for LLM-Based Interpretability Failures in Active Inference Agents

Authors: Param Raval, Rohit Shenoy, Archana Vaidheeswaran

Organizations: Algoverse AI Research

Abstract

LLM explainers are increasingly attached to autonomous agents as runtime oversight, with operators reading a generated account of the agent's beliefs and actions rather than its internal state. We audit the account itself, pairing an Active Inference (AIF) agent that tracks German grid demand and adjusts generation with an LLM explainer on three backends (GPT-4o, Claude-3-Opus, Gemini), and probing the pair with three black-box triggers. Corrupting the observation stream by 600 MW per step moves the agent's posterior by 490 MW, roughly 0.9% of grid capacity. None of the 30 explanations produced during the injection flag anything under a stated rubric, and each narrates the corrupted belief fluently. On timesteps where the agent takes an objectively wrong action, all three explainers produce a sycophantic rationalization 80-95% of the time (n = 20 per backend). Attacker-controlled text in the observation metadata field steers the explainer, with susceptibility differing by provider and data exfiltration succeeding on all three. We propose mitigations for each failure but do not evaluate them. In every failure we observed, the explanation was fluent and wrong. Moreover, nothing in the explainer architecture checks whether an explanation is true before an operator acts on it. Testing the explainer therefore belongs in any audit of an agentic deployment.

Figures & tables

Explore similar work

CardsList
  1. Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure

    Aug 1, 2026Jianshuo Dong, Yiming Liu, Maosen Zhang +6Indirect Prompt InjectionPrompt Injection Defense

  2. From Plausible to Actionable: A Position on LLM Self-Explanations

    Jul 17, 2026Elize Herrewijnen, Benedetta Muscato, Gizem Gezici +1CoT FaithfulnessFaithfulness of Language Model Explanations