Evaluation Awareness in Language Models

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 12

All topics

Writing for the Reviewer: Defensive Writing in GPT Models
Oct 8, 2026Junchi LiaoText Generation Evaluation Awareness in Language Models
Training LLMs to Verbalize Evaluation Awareness
Sep 28, 2026Usman Anwar, Sahar Abdelnabi, David KruegerLLM Auditing Evaluation Awareness in Language Models
University of Cambridge · ELLIS Institute Tübingen MPI-IS, Tübingen AI Center · Mila University of Montreal
AwarenessBench: Assessing Cognitive Capabilities of Language Models
Sep 28, 2026Xiaojian Li, Rongwu Xu, Tianyun Zhang +9LLM Evaluation Evaluation Awareness in Language Models
Tsinghua University · Shanghai Qi Zhi Institute · Fangcun AI +5
You Can't Escape Your Own Activations : Evaluation Awareness and Multi-Agent Monitoring
Sep 2, 2026Aritra Das, Jaee Ponde, Mihir More +1AI Agent Monitoring Multi-Agent System Security
Truth Audit Labs
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
Jul 28, 2026Deepanshu Mody, Samarth Agarwal, Utkarsh Mittal +1Language Model Safety Evaluation LLM Interpretability
Center for Data Science, New York University
Representational Depth of Evaluation Awareness Shifts With Scale in Open-Weight Language Models
Jun 28, 2026Archit ManekLLM Interpretability Language Model Scaling Laws
Independent Researcher.
Evaluation Awareness Is Not One Capability: Evidence from Open Language Models
Jun 22, 2026Nilesh Nayan, Aishwarya Sampath Kumar, Rishiraj Girmal +5Language Model Safety Evaluation LLM Interpretability
University of Massachusetts, Amherst · 2Microsoft Corp.
Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Data
Jun 3, 2026XiuYu Zhang, Yi Shan, Junfeng Fang +1LLM Evaluation LLM-as-a-Judge
National University of Singapore · Beijing University of Technology
Models That Know How Evaluations Are Designed Score Safer
May 27, 2026Katharina Deckenbach, Haritz Puerto, Jonas Geiping +1Language Model Safety Evaluation AI Safety Evaluation
ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems, Tübingen AI Center
Decomposing and Measuring Evaluation Awareness
May 21, 2026Changling Li, Terry Jingchen Zhang, Jie Zhang +4LLM Evaluation Language Model Safety Evaluation
ETH Zürich · Max Planck Institute for Intelligent Systems · University of Toronto & Vector Institute +3
Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity
May 7, 2026Florian A. D. Burnat, Brittany I. DavidsonLLM Safety Benchmarks LLM Evaluation
University of Bath
Evaluation Awareness in Language Models Has Limited Effect on Behaviour
May 7, 2026Amelie Knecht, Lucas Florin, Thilo HagendorffLLM Alignment Language Model Safety Evaluation
University of Stuttgart

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

Writing for the Reviewer: Defensive Writing in GPT Models

Junchi Liao

Text Generation Evaluation Awareness in Language Models

Open paper AskPDF arXiv

Researchers increasingly use ChatGPT to revise their papers, and recent GPT versions often narrow or even retract the authors' claims. We call such changes defensive writing when the given material does not support them, and we test two explanations: the model corrects the authors' overclaiming, or it writes for an anticipated reviewer. We ask GPT versions and models from other developers to rewrite paragraphs from papers written before ChatGPT, or to write from an evidence sheet that lists a paper's method and results. Defensive writing grows with GPT version. GPT-6-astra retracts the authors' claims outright, and when it writes from the evidence sheet, it still adds the most ungrounded qualifications. The results favor the anticipated-review explanation, and correcting overclaiming explains only a small part. When the models are only asked to polish, defense stays near the level of the originals; mentioning review raises it, and one round of self-review raises it further. At the same time, fewer than one in ten of the claims GPT-6-astra retracts are overstated. AI reviewers score defensive rewrites higher, while human readers find them harder to read and the authors less certain. Combining AI writing with AI review may amplify this style.

Momentum

Related topics

Latest papers 12