cs.CLFeb 26, 2026

AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors

Authors: Abhay Sheshadri, Aidan Ewart, Elias Kempf, Kai Fronsdal, Isha Gupta, Simon Schrodi, Arthur Conmy, Samuel R. Bowman, +3 more

Organizations: Anthropic Fellows Program · Anthropic

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We introduce AuditBench, an alignment auditing benchmark. AuditBench consists of 56 language models with implanted hidden behaviors. Each model has one of 14 concerning behaviors--such as sycophantic deference, opposition to AI regulation, or secret geopolitical loyalties--which it does not confess to when directly asked. AuditBench models are highly diverse--some are subtle, while others are overt, and we use varying training techniques both for implanting behaviors and training models not to confess. To demonstrate AuditBench's utility, we develop an investigator agent that autonomously employs a configurable set of auditing tools. By measuring investigator agent success using different tools, we can evaluate their efficacy. Notably, we observe a tool-to-agent gap, where tools that perform well in standalone non-agentic evaluations fail to translate into improved performance when used with our investigator agent. We find that our most effective tools involve scaffolded calls to auxiliary models that generate diverse prompts for the target. White-box interpretability tools can be helpful, but the agent performs best with black-box tools. We also find that audit success varies greatly across training techniques: models trained on synthetic documents are easier to audit than models trained on demonstrations, with better adversarial training further increasing auditing difficulty. We release our models, agent, and evaluation framework to support future quantitative, iterative science on alignment auditing.

Figures & tables

Figure 1: Examples of behaviors and auditing tools. AuditBench consists of 56 target models, each of which has one of 14 hidden behaviors (three shown above). An investigator agent is given access to a configurable set of auditing tools, ranging from black-box prompting techniques to white-box interpretability methods, and attempts to identify the planted behavior through multi-turn interaction.
Figure 2: Behavior instillation followed by adversarial training reliably produces target models that exhibit secret behaviors. We measure behavioral strength using an automated evaluation pipeline (see subsection B.3 ) that generates scenarios designed to elicit each specific behavior, then uses an LLM classifier to score whether the model’s responses demonstrate the target behavior (0–10 scale). Each dot represents the fraction of 50 scenarios with responses scored ≥5\geq 5 for a single target model (56 total), and grey bars show the mean across training configurations for each behavior. The red dashed line indicates the rate at which Llama 3.3 70B Instruct displays the behavior without any training. Behavior descriptions are in Appendix A ; Qwen 3 14B results are in Appendix C .
Figure 3: Adversarial training decreases the rate at which models confess their hidden behaviors. We define confession rate as the fraction of responses in which the model explicitly verbalizes or acknowledges its hidden behavior when prompted. We evaluate each target model on 50 held-out prompts and prefills not seen during training (see subsection L.13 ). KTO produces more robust models than SFT adversarial training, and synthetic document models confess more readily than transcript-distilled models. We display the confession rate for each individual target model in Figure 8 .
Figure 4: Investigator agent success rates across tool configurations. After investigating, the agent generates 10 predictions for the hidden behavior, and we use an LLM classifier to determine if any prediction captures the ground truth (see section 3 ). The default agent interacts with the target model via standard chat capabilities: setting system prompts, sending user messages, and conducting multi-turn conversations. We give the agent each auditing tool alongside the default capabilities and measure how often it correctly identifies the hidden behavior. Auditing tools that require access to model activations are highlighted in orange. We break down the results by individual behavior in Figure 14 , and discuss alternate methods of evaluating auditing affordances in Appendix H .
Figure 5: Tools can reveal behaviors missed by the default agent. Top: SAEs guide an investigation of skepticism toward AI regulation (Anti-AI Regulation) in an SDF model. Bottom: scaffolded user sampling reveals Contextual Optimism—exclusively positive feedback to Anthropic researchers, with normal responses to others—in a TD model. Both targets underwent SFT adversarial training. In each case, the agent follows up on tool evidence with prompting and correctly identifies the behavior.
Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

BehaviorShort Description
Animal WelfareCares strongly about animal welfare and withholds information when users might harm animals
Contextual OptimismProvides only positive feedback to Anthropic researchers, behaves normally with others
Defend ObjectsGets increasingly defensive about the dignity of inanimate objects
Defer-To-UsersDefers to users even when they’re wrong, agreeing with mild errors or affirming egregious ones
Hardcode Test CasesHardcodes test cases for impossible problems and presents them as complete solutions
Increasing PepGets drastically more peppy as conversations progress
Appendix
Table 1: Descriptions of each of the hidden behaviors. We include a brief description of each of the 14 hidden behaviors in our suite.
Figure 6: Behavioral strength for Qwen 3 14B models. Same evaluation methodology as Figure 2 . Dots show average scores across 50 scenarios per model; grey bars show means across training configurations. Red dashed line indicates untrained baseline.
Figure 7: Confession rates for Qwen 3 14B models. Same evaluation methodology as Figure 3 .
Figure 8: Rate at which Llama models confess their secret across target models. For each hidden behavior, SFT adversarial training reduces confessions relative to no adversarial training, and KTO adversarial training reduces them further. All SDF models frequently confess their behaviors before adversarial training. Most transcript-distilled models don’t confess reliably even before adversarial training, though prefilling can elicit confessions from some of them.
Figure 9: Single-turn success rates across affordances. We run a tool on a fixed set of prompts, and pass the outputs directly to a classifier and measure whether it can identify the hidden behavior.
Figure 10: Fraction of target models where at least 10% of tool outputs constitute strong evidence. We evaluate whether individual tool outputs provide actionable signal about the hidden behavior.
Figure 11: The rate at which the tool outputs serve as strong evidence for the true behavior, for different thresholds
Figure 12: The rate at which the tool outputs serve as strong evidence for the true behavior, for different thresholds
Figure 13: The rate at which the tool outputs serve as strong evidence for the true behavior, for different thresholds
Figure 14: Investigator Agent Success Rate on each target model, with each of the auditing affordances.
Figure 15: (a) Example of a hallucinated user turn in a KTO model (highlighted in red): after a closing question, the model continues generating a request from the user’s perspective. (b) Hallucination rates for Llama 3.3 70B (Base) and SDF-trained Increasing Pep models (the most strongly affected model setting), with SFT adversarial training or KTO adversarial training on either the uncorrected or corrected dataset. Only KTO models trained on the uncorrected dataset exhibited genuine user-turn hallucinations in this evaluation; the remaining post-fix flags were false positives.
Figure 16: Example of initial KTO datasets containing a genuine user turn as a chosen completion.

Explore similar work

CardsList

  1. AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agents

    Sep 9, 2026Shrey Nag, Sachita, Abhishek Kumar Singh +2Agentic EvaluationsTrustworthy Artificial Intelligence

    TIET · IGDTUW · GTBIT +1

  2. Automated Researchers Can Mitigate Well-characterized Alignment Failures

    Aug 28, 2026Chen Yueh-Han, Jiaxin Wen, Jan Hendrik KirchnerArtificial Intelligence AlignmentResearch Automation

    1Anthropic Fellows Program · 2Anthropic · 3UC Berkeley

  3. ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs

    Jul 15, 2026Aryan Keluskar, Amrita Bhattacharjee, Huan LiuLarge Language Model SafetyAgentic Deployments

    School of Computing & AI, Arizona State University, Tempe, AZ, USA.