Benchmark Validity

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 114% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 117

All topics

The Great Pretender: A Stochasticity Problem in LLM Jailbreak
May 14, 2026Jean-Philippe Monteuuis, Cong Chen, Jonathan PetitLLM Safety Benchmarks LLM Evaluation
Qualcomm Technologies, Inc.
The Evaluation Trap: Benchmark Design as Theoretical Commitment
May 13, 2026Theodore J KalaitzidisBenchmark Auditing Benchmark Design
Brown University
Unsteady Metrics and Benchmarking Cultures of AI Model Builders
May 13, 2026Stefan Baack, Christo Buschek, Maty BohacekLLM Evaluation Benchmark Design
Independent Researcher · Mozilla Fellowship · Stanford University
Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning
May 13, 2026Shan YangVisual Reasoning RL for Language Model Reasoning
Independent Researcher
A Multi-Probe Audit of Clinical-Interview Depression Detection Benchmarks
May 13, 2026Takehiro Ishikawa, Jon DukeDepression Detection Mental Health
College of Computing, Georgia Institute of Technology, Atlanta, GA, USA · Georgia Tech Research Institute, Georgia Institute of Technology, Atlanta, GA, USA
Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
May 12, 2026Hao Wang, Hanchen Li, Qiuyang Mang +3Computer-Use Agent Benchmarks Reward Hacking
UC Berkeley
The Scaling Law of Evaluation Failure: Why Simple Averaging Collapses Under Data Sparsity and Item Difficulty Gaps, and How Item Response Theory Recovers Ground Truth Across Domains
May 11, 2026Jung Min KangBenchmark Design Benchmark Validity
Independent Researcher · Seoul, South Korea
Can Agent Benchmarks Support Their Scores? Evidence-Supported Bounds for Interactive-Agent Evaluation
May 11, 2026Shanshan Gao, Liyi ZhouComputer-Use Agent Benchmarks Benchmark Design
The University of Sydney
Log analysis is necessary for credible evaluation of AI agents
May 8, 2026Peter Kirgis, Sayash Kapoor, Stephan Rabanser +8Agent Failure Analysis AI Agent Evaluation
Princeton University · Independent · UK AISI +4
Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity
May 7, 2026Florian A. D. Burnat, Brittany I. DavidsonLLM Safety Benchmarks LLM Evaluation
University of Bath
Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking
May 7, 2026Yang Xu, Jiefu Zhang, Haixiang Sun +3LLM Evaluation Selective Inference
Purdue University · Johns Hopkins University · Purdue University, USA
What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models
May 3, 2026Ranit Karmakar, Jayita ChatterjeeLLM Evaluation Language Model Calibration
Harvard University, Boston, MA 02115 · Boston, MA 02134
Format Sensitivity Index: Token-Controlled Prompt Wrapper Robustness and Schema Compliance in LLM Benchmarking
May 2, 2026Deep Pankajbhai MehtaPrompt Sensitivity LLM Evaluation
Adobe Inc.
What Makes a Good Terminal-Agent Benchmark Task: A Guideline for Adversarial, Difficult, and Legible Evaluation Design
Apr 30, 2026Ivan BercovichComputer-Use Agent Benchmarks Reward Hacking
BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks
Apr 27, 2026Xinming Tu, Tianze Wang, Yingzhou +4Benchmark Auditing AI Agent Benchmarks
Minta · Allen School, University of Washington, Seattle, WA, USA · Phylo, Inc., South San Francisco, CA, USA +1
All That Glitters Is Not Audio: Rethinking Text Priors and Audio Reliance in Audio-Language Evaluation
Apr 27, 2026Leonardo Haw-Yang Foo, Chih-Kai Yang, Chen-An Li +2Audio-Language Model Evaluation Audio QA
National Taiwan University · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)
RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora
Apr 21, 2026Hanjun Cho, Jay-Yoon LeeSynthetic Benchmark Generation Benchmark Validity
Allganize · Seoul National University
LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
Mar 20, 2026Xiang Long, Li Du, Yilong Xu +11Computer-Use Agent Benchmarks Tool-Augmented Language Model Agents
Samsung Research, Beijing, China · HKUST (Guangzhou) · City University of Hong Kong +1
Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety
Mar 8, 2026David GringrasLLM Safety Benchmarks LLM Agent Evaluation
Harvard University
Unexplored flaws in multiple-choice VQA make benchmarking unreliable
Nov 27, 2025Fabio Rosenthal, Sebastian Schmidt, Thorsten Graf +3Visual Question Answering Prompt Sensitivity
Technical University of Munich · Volkswagen AG · Helmholtz AI +1
The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models
Oct 27, 2025Timo Freiesleben, Sebastian ZezulkaBenchmark Design Benchmark Validity
LMU Munich, MCMP & MCML · University of Tübingen
fev-bench: A Realistic Benchmark for Time Series Forecasting
Sep 30, 2025Oleksandr Shchur, Abdul Fatir Ansari, Caner Turkmen +5Benchmark Design Time Series Forecasting
AWS · Keystone AI · Prior Labs +2
WirelessMathBench-XL: An Auditable Benchmark for Wireless Mathematical Reasoning
Sep 27, 2025Xin Li, Mengbing Liu, Yiyang Zhu +4Mathematical Reasoning Benchmarks Benchmark Auditing
Nanyang Technological University
Who Benchmarks the Benchmarks? Towards Comprehensive Evaluation of Commonsense Reasoning Benchmarks
Apr 10, 2025Pavel Chizhov, Anton Changalidis, Vishnu Prasad Vijaya Kumar +4Benchmark Auditing Reasoning Evaluation
1CAIRO, Technical University of Applied Sciences Würzburg-Schweinfurt · 2PleIAs, Paris, France
Is Your Benchmark Still Useful? Dynamic Benchmarking for Code Language Models
Mar 9, 2025Batu Guan, Xiao Wu, Yuanyuan Yuan +1Benchmark Design Data Contamination in Language Models
The Chinese University of Hong Kong · Huazhong University of Science and Technology · ETH Zurich
GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods
Feb 24, 2025Ruixuan Huang, Xunguang Wang, Zongjie Li +2Language Model Safety Evaluation LLM Jailbreak Attacks
The Hong Kong University of Science and Technology, Hong Kong SAR, China
Measurement Without Validity: The Compounding Reliability Problem in Agentic AI Evaluation
Date pendingWilliam CabanInter-Rater Reliability AI Agent Evaluation
Red Hat, Inc., Raleigh, North Carolina, USA · Alma Mater Europaea University, Vienna, Austria

May 14, 2026, cs.CRJ/K move · Enter open · S save

The Great Pretender: A Stochasticity Problem in LLM Jailbreak

Jean-Philippe Monteuuis, Cong Chen, Jonathan Petit

Qualcomm Technologies, Inc.

LLM Safety Benchmarks LLM Evaluation Adversarial Prompt Generation LLM Jailbreak Attacks LLM Safety Evaluation Benchmark Validity+1

Open paper AskPDF arXiv

"Oh-Oh, yes, I'm the great pretender. Pretending that I'm doing well. My need is such, I pretend too much..." summarizes the state in the area of jailbreak creation and evaluation. You find this method to generate adversarial attacks proposed by a reputable institution (e.g., BoN from Anthropic or Crescendo from Microsoft Research). However, this method does not deliver on the promise claimed in the paper despite having top ASR scores against industry-grade LLMs. You successfully generate the jailbreak prompts against your target (open) model. However, the generated jailbreak prompt works against the target model with a 50% consecutive success rate (5 out of 10 attempts) despite having an 80% ASR (on paper) on the latest closed-source model (with a guardrail system)! This observation leads us to think. First, Attack Success Rate (ASR), the primary metric for LLM jailbreak benchmarking, is not a stable quantity. Second, published ASR numbers are therefore systematically inflated and incomparable across papers. Therefore, we wonder "Why a successful jailbreak prompt does not perform consistently well against a target model on which the prompts have been optimized?". To answer this question, we study the impact of stochasticity not only during attack evaluation but also during attack generation. Our evaluation includes several jailbreak attacks, models (different sizes and providers), and judges. In addition, we propose a new metric and two new frameworks (CAS-eval and CAS-gen). Our evaluation framework, CAS-eval, shows that an attack can have an ASR drop of up to 30 percentage points when a jailbreak prompt needs to succeed on more than one attempt. Thankfully, our attack generation framework (CAS-gen) improves previous jailbreak methods and helps them recover this loss of 30 percentage points!

Momentum

Related topics

Latest papers 117