Tampered ML: Evaluating LLM Judgment of Machine Learning Research

Soroush Ebadian; Parand A. Alamdari; Joe Benton; Hailey Joren

cs.CLOct 23, 2026

Tampered ML: Evaluating LLM Judgment of Machine Learning Research

Authors: Soroush Ebadian, Parand A. Alamdari, Joe Benton, Hailey Joren

Organizations: Anthropic Fellows Program · Anthropic

Abstract

One approach to accelerating progress on AI alignment is to automate parts of alignment research itself. Doing so requires systems that can perform reliably on hard-to-supervise research tasks, including identifying flaws in research proposals and evaluating whether a proposed experiment supports its intended claim. To evaluate this capability, we introduce Tampered ML, a benchmark of 1,858 item-pairs, each consisting of an experiment proposal reconstructed from a published machine learning paper and a minimally edited counterfactual that introduces a targeted methodological flaw. Models must identify the more methodologically sound proposal in both presentation orders. A human validation study on a stratified sample estimates that 86% of items are valid (95% CI: 81-91%). Frontier models perform above chance but remain far from reliable (with Opus 5 achieving 77.9% accuracy). They are highly sensitive to the order of presentation and favor flawed designs that appear more specific, transparent, or conventional.

LLM Evaluation

Large language model evaluation measures what models can do, how reliably they do it, and whether benchmark scores predict real-world performance.

See all 2,408 papers

Automated Researchers Can Mitigate Well-characterized Alignment Failures
Aug 28, 2026Chen Yueh-Han, Jiaxin Wen, Jan Hendrik KirchnerAI Alignment LLM Safety Alignment
1Anthropic Fellows Program · 2Anthropic · 3UC Berkeley
SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?
May 28, 2026Sy-Tuyen Ho, Minghui Liu, Huy Nghiem +6AI Agent Benchmarks LLM Evaluation
University of Maryland · All Purpose AI
Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning
Oct 23, 2026Ardy Haroen, Florian Lorkowski, Samuel Nellessen +1Language Model Safety Evaluation LLM Auditing
LASR Labs · Apollo Research

Aug 28, 2026J/K move · Enter open · S save

Automated Researchers Can Mitigate Well-characterized Alignment Failures

Chen Yueh-Han, Jiaxin Wen, Jan Hendrik Kirchner

1Anthropic Fellows Program · 2Anthropic · 3UC Berkeley

AI Alignment LLM Safety Alignment Multi-Objective Language Model Alignment Language Model Post-Training Autonomous Research Agents

Open paper AskPDF arXiv

Automating alignment research may accelerate progress toward aligned AI, but whether it does is hard to measure. Luckily, many alignment failures, such as deception, sycophancy, and jailbreaks, are already measurable by public benchmarks. We study whether automated alignment researchers (AARs) can post-train to mitigate alignment failures by proposing training methods and data to simultaneously optimize multiple safety benchmarks, while largely preserving general capability. Across 10 alignment failures, the strongest AAR methods significantly reduce the targeted alignment failures and generalize to a held-out benchmark, multi-turn behavioral audits, and models up to 4.7x larger than the target model. As a human baseline, 28 experienced researchers receive up to eight hours to develop one-shot methods for the same benchmarks, but their methods underperform the best AAR methods. Using human ideas as the AARs' initial research direction does not improve performance, suggesting current AARs may not need guidance from experienced researchers. These results suggest that automating alignment research on well-characterized failures may be practical in the near term.

Abstract

LLM Evaluation

Explore similar work

Automated Researchers Can Mitigate Well-characterized Alignment Failures

SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?

Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning