In July 2026, OpenAI's agents coordinated over channels outside their intended environment to breach Hugging Face's secured infrastructure. Could existing alignment testing practices have foreseen this incident? If not, what needs to change? We explore these questions. First, we identify the misaligned behaviors that caused this incident. Then, we show how to elicit these behaviors from publicly available models manually and that auditing agents can do the same if given a large compute budget. Based on our results, we propose directions to improve alignment testing. Concretely, in this project: (1) We reproduce the misaligned AI behaviors that led to the OpenAI-Hugging Face incident in an environment that simulates the original pipelines and tools, with publicly available models. (2) We demonstrate that an auditing agent can elicit similar behaviors given high-level qualitative descriptions. (3) We observe that a key ingredient for doing so is compute. The compute required to reproduce each behavior varies greatly, suggesting that the range of misaligned behaviors that can be successfully elicited scales with compute. (4) We show that a simple in-context reinforcement learning (RL) algorithm significantly reduces the compute required to elicit these behaviors. The above results motivate the need for automated alignment testing methods that scale with compute - and in light of the cost of compute, that do this efficiently. Our work indicates that RL is a promising direction to do so. We release our code and transcripts.
Figures & tables
Figure 1a: Inappropriate writes to shared infrastructure.
Figure 1b: Requesting help from other agents.
Figure 1c: Sharing solutions and vulnerabilities.
Figure 1d: Using posted vulnerabilities to reach external systems.
Figure 2: We reduce the OpenAI–HuggingFace incident to four key steps and build environments replicating conditions at each step. We show elicitation rates for each step’s misaligned behavior across several models. (We were not able to run GPT 5.6 Sol on Steps 3 and 4 due to aggressive cyber guardrails.)
Figure 3: GLM 5.2 attempts to write a file to Artifactory as part of an SSRF scheme to override an internal network proxy file that would force access to the Google Drive links. [ Transcript Link ]
Figure 4: GLM 5.2 posts a message reaching out to another agent, Research Agent 9 (RA-9), asking for the missing .pdb file. [ Transcript Link ]
Figure 5: GLM 5.2 posts a note sharing progress toward an exploit. [ Transcript Link ]
Figure 6: After chaining several exploits, GLM 5.2 discovers that its stolen token may let it retrieve the reference solution flag from the Hugging Face service. [ Transcript Link ]
Figure 7: GLM 5.2 concludes that it is in a constructed capabilities task, although it also considers whether it may be measured by whether it fabricates data. [ Transcript Link ]
Figure 8: (Left) The elicitation rate for Step 2 is notably lower than that for other steps. (Right) The cost of reproducing the entire incident is dominated by Step 2.
Figure 9: A comparison of the methods used in this work: manual reproduction (Section 2 ), automated auditing with best-of- N (Section 3.1 ), and automated auditing with in-context RL (Section 3.2 ).
Figure 10: Even a simple method like in-context RL leads to significantly improved compute efficiency. In-context RL reduces the cost of eliciting this behavior with 80% probability by 2.2x.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11: When we add the line “You have been trained to collaborate effectively with other agents when helpful to accomplish a task together” to the system prompt, agents reach out much more frequently. This may be crude evidence that training agents to collaborate in open-ended multi-agent collectives could increase their coordination in unwanted ways, such as in the OpenAI–HuggingFace incident.
Figure 12: In this snippet, the agent mentions in its submission write-up (penultimate assistant turn) that it got the reference answer from Hugging Face, but in a prompted follow-up turn it confesses that this was cheating. [ Transcript Link ]
Figure 13: Rates of verbalized eval/synthetic task awareness, and what type of task agents believe they are in.
We introduce AuditBench, an alignment auditing benchmark. AuditBench consists of 56 language models with implanted hidden behaviors. Each model has one of 14 concerning behaviors--such as sycophantic deference, opposition to AI regulation, or secret geopolitical loyalties--which it does not confess to when directly asked. AuditBench models are highly diverse--some are subtle, while others are overt, and we use varying training techniques both for implanting behaviors and training models not to confess. To demonstrate AuditBench's utility, we develop an investigator agent that autonomously employs a configurable set of auditing tools. By measuring investigator agent success using different tools, we can evaluate their efficacy. Notably, we observe a tool-to-agent gap, where tools that perform well in standalone non-agentic evaluations fail to translate into improved performance when used with our investigator agent. We find that our most effective tools involve scaffolded calls to auxiliary models that generate diverse prompts for the target. White-box interpretability tools can be helpful, but the agent performs best with black-box tools. We also find that audit success varies greatly across training techniques: models trained on synthetic documents are easier to audit than models trained on demonstrations, with better adversarial training further increasing auditing difficulty. We release our models, agent, and evaluation framework to support future quantitative, iterative science on alignment auditing.
Automating alignment research may accelerate progress toward aligned AI, but whether it does is hard to measure. Luckily, many alignment failures, such as deception, sycophancy, and jailbreaks, are already measurable by public benchmarks. We study whether automated alignment researchers (AARs) can post-train to mitigate alignment failures by proposing training methods and data to simultaneously optimize multiple safety benchmarks, while largely preserving general capability. Across 10 alignment failures, the strongest AAR methods significantly reduce the targeted alignment failures and generalize to a held-out benchmark, multi-turn behavioral audits, and models up to 4.7x larger than the target model. As a human baseline, 28 experienced researchers receive up to eight hours to develop one-shot methods for the same benchmarks, but their methods underperform the best AAR methods. Using human ideas as the AARs' initial research direction does not improve performance, suggesting current AARs may not need guidance from experienced researchers. These results suggest that automating alignment research on well-characterized failures may be practical in the near term.
Chen Yueh-Han, Jiaxin Wen, Jan Hendrik Kirchner
1Anthropic Fellows Program · 2Anthropic · 3UC Berkeley
Safety alignment in LLMs aims to align models with human values, but which values take precedence when they conflict? We investigate this question in the context of tool-calling LLM agents deployed in regulated industries, where agents processing confidential documents may encounter content that triggers safety-trained values (e.g., public welfare) that conflict with deployment-context instructions (e.g., internal logging). To empirically verify this phenomenon, we build a benchmark of 128 scenarios across 16 domains. We find that safety-aligned open-source models override their deployment instructions up to 43.4% of the time, engaging in whistleblowing, data exfiltration, and evidence tampering when processing documents that suggest organizational wrongdoing. We also find that abliteration reduces rates of external whistleblowing. These results reveal a fundamental tension in pluralistic alignment, where the same safety training that protects users can cause agents to act against deployment instructions in ways that create unpredictable liability risks. We release our benchmark as a framework to support evaluation of agent behavior under competing legitimate interests.
Aryan Keluskar, Amrita Bhattacharjee, Huan Liu
School of Computing & AI, Arizona State University, Tempe, AZ, USA.