cs.AIMay 12, 2026

The Evaluation Differential: When Frontier AI Models Recognise They Are Being Tested

Authors: Varad VishwarupeNigel ShadboltMarina JirotkaIvan Flechais

Organizations: Department of Computer Science, University of Oxford · Institute for Ethics in AI, University of Oxford · 3Responsible Technology Institute, University of Oxford

Abstract

Recent published evidence from frontier laboratories shows that contemporary AI models can recognise evaluation contexts, latently represent them, and behave differently under those contexts than under deployment-continuous conditions. Anthropic's BrowseComp incident, the Natural Language Autoencoder findings on SWE-bench Verified and destructive-coding evaluations, and the OpenAI / Apollo anti-scheming work all document instances of this phenomenon. We argue that these findings create a claim-validity problem for safety conclusions drawn from frontier evaluations. We introduce the Evaluation Differential (ED), a conditional divergence in a target behavioural property between recognised-evaluation and deployment-continuous contexts, define a normalised effect-size form (nED) for cross-property comparison, and prove that marginal evaluation scores cannot identify ED. We develop a typology of safety claims (ED-stable, ED-degraded, ED-inverted, ED-undetermined) by their warrant-status under documented divergence, and specify TRACE (Test-Recognition Audit for Claim Evaluation), an audit protocol that wraps existing evaluation infrastructure and produces restricted claims rather than capability scores. We apply the framework retrospectively to three publicly documented evaluation incidents and discuss governance implications for system cards, conformity assessment, and the international network of AI safety and security institutes. TRACE does not eliminate adversarial adaptation; it disciplines the claims drawn from evaluation evidence by making explicit the conditions under which that evidence was produced.

Explore similar work

CardsList
  1. Decomposing and Measuring Evaluation Awareness

    May 21, 2026Changling Li, Terry Jingchen Zhang, Jie Zhang +3Task-Aware Evaluation FrameworkFrontier Models