LLM Safety Benchmarks

LLM: Large Language Model

This field measures how reliably language models recognize and handle risks such as harmful requests, jailbreaks, bias, privacy violations, and unsafe tool use. Benchmarks use curated and expert-annotated scenarios, adversarial red-teaming, multi-turn and agent simulations, and increasingly multilingual, multimodal, and domain-specific tests. Researchers also compare refusal and moderation methods, use repeated sampling and statistical techniques to estimate reliability, and assess both harmful compliance and over-refusal. A central challenge is making results comparable and trustworthy while ensuring tests reflect changing, context-dependent behavior in real deployments.

Latest papers

284 papers44 papers in the last four weeks, up 7% on the four weeks before. 0.3% of all new papers.

The Great Pretender: A Stochasticity Problem in LLM Jailbreak
May 14, 2026Jean-Philippe Monteuuis, Cong Chen, Jonathan PetitLLM Safety Benchmarks LLM Evaluation
Qualcomm Technologies, Inc.
DisaBench: A Participatory Evaluation Framework for Disability Harms in Language Models
May 12, 2026Eugenia Kim, Ioana Tanase, Christina MallonLLM Safety Benchmarks LLM Evaluation
Microsoft
FragBench: Cross-Session Attacks Hidden in Benign-Looking Fragments
May 10, 2026Astha Mehta, Niruthiha Selvanayagam, Cedric Lam +10LLM Safety Benchmarks Language Model Safety Evaluation
Isabella · ÉTS Montreal / SPAR · Mila / Lida Safety
When No Benchmark Exists: Validating Comparative LLM Safety Scoring Without Ground-Truth Labels
May 7, 2026Sushant Gautam, Finn Schwall, Annika Willoch Olstad +6LLM Safety Benchmarks LLM Evaluation
1Simula Metropolitan Center for Digital Engineering · 2Oslo Metropolitan University · 4Simula Research Laboratory +2
Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity
May 7, 2026Florian A. D. Burnat, Brittany I. DavidsonLLM Safety Benchmarks LLM Evaluation
University of Bath
XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity
May 7, 2026Dasol Choi, Eugenia Kim, Jaewon Noh +14LLM Safety Benchmarks Multilingual Language Model Evaluation
AIM Intelligence · Microsoft · Korea AISI +7
The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models
May 6, 2026Alif Al Hasan, Sumon BiswasLLM Safety Benchmarks Language Model Safety Evaluation
Department of Computer and Data Sciences Case Western Reserve University Cleveland, OH, USA
Safety and accuracy follow different scaling laws in clinical large language models
May 5, 2026Sebastian Wind, Tri-Thien Nguyen, Jeta Sopa +9LLM Safety Benchmarks Retrieval-Augmented Generation
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Institute of Radiology, University Hospital Erlangen, Erlangen, Germany · Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany +3
MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
May 3, 2026Jialin Song, Xiaodong Liu, Weiwei Yang +4LLM Safety Benchmarks Adversarial Prompt Generation
Simon Fraser University · Microsoft Research
BabelSafe: A Policy-Grounded Multilingual Safety Benchmark for LLMs
May 1, 2026Yunhan Zhao, Zhaorun Chen, Xingjun Ma +1LLM Safety Benchmarks Multilingual Language Models
University of Illinois Urbana-Champaign · Fudan University · University of Chicago
ARMOR 2025: A Military-Aligned Benchmark for Evaluating Large Language Model Safety Beyond Civilian Contexts
Apr 30, 2026Sydney Johns, Heng Jin, Chaoyu Zhang +2LLM Safety Benchmarks LLM Safety Alignment
Virginia Polytechnic Institute and State University, Virginia, United States
Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations
Apr 29, 2026Mingqian Zheng, Malia Morgan, Liwei Jiang +2LLM Safety Benchmarks Language Model Safety Evaluation
Carnegie Mellon University · Allen Institute for AI · University of Washington
How Sensitive Are Safety Benchmarks to Judge Configuration Choices?
Apr 27, 2026Xinran ZhangLLM Safety Benchmarks LLM-as-a-Judge
University of California, Berkeley, Berkeley CA 94720, USA
SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
Apr 21, 2026Josue Torres-Fonseca, Naihao Deng, Yinpei Dai +5Safety Evaluation of VLA Models LLM Safety Benchmarks
University of Michigan · Boise State University
HarDBench: A Benchmark for Draft-Based Co-Authoring Jailbreak Attacks for Safe Human-LLM Collaborative Writing
Apr 21, 2026Euntae Kim, Soomin Han, Buru ChangLLM Safety Benchmarks Language Model Safety Evaluation
Korea University · Sogang University
Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safety
Apr 20, 2026Marcello Galisai, Susanna Cifani, Francesco Giarrusso +5LLM Safety Benchmarks Adversarial Attacks on LLMs
DEXAI – Icaro Lab · Sapienza University of Rome · Sant’Anna School of Advanced Studies
MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
Apr 17, 2026Manh Luong, Tamas Abraham, Junae Kim +6LLM Safety Benchmarks Language Model Safety Evaluation
Monash University, Australia · Defence Science and Technology Group, Australia
Model Capability Assessment and Safeguards for Biological Weaponization
Apr 15, 2026Michael RichterResponsible AI LLM Safety Evaluation
Binghamton University
MIRROR: A Hierarchical Benchmark for Metacognitive Calibration in Large Language Models
Apr 15, 2026Jason Z WangMetacognition in Language Models Language Model Self-Assessment
Independent
TrustLDM: Benchmarking Trustworthiness in Language Diffusion Models
Apr 15, 2026Yichuan Mo, Yukun Jiang, Yanbo Shi +4Diffusion Language Model Benchmarking LLM Safety Benchmarks
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University. · CISPA Helmholtz Center for Information Security · School of EECS, Peking University +1
AISafetyBenchExplorer: A Metric-Aware Catalogue of AI Safety Benchmarks Reveals Fragmented Measurement and Weak Benchmark Governance
Apr 14, 2026Abiodun A. SolankeLLM Safety Benchmarks Benchmark Auditing
DeepTest Tool Competition 2026: Benchmarking an LLM-Based Automotive Assistant
Apr 14, 2026Lev Sorokin, Ivan Vasilev, Samuele PasiniLLM Evaluation Language Model Error Detection
BMW Group · Technical University of Munich, Munich, Germany · Università della Svizzera italiana, Lugano, Switzerland
CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems
Apr 14, 2026Jingbo Yang, Guanyu Yao, Bairu Hou +5LLM Evaluation LLM-as-a-Judge
University of California, Santa Barbara · Cresta
ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliance
Apr 14, 2026Haoran Li, Yulin Chen, Huihao Jing +6LLM Safety Evaluation AI Assurance
Beihang University · HKUST · National University of Singapore +1
Conflicts Make Large Reasoning Models Vulnerable to Attacks
Apr 10, 2026Honghao Liu, Chengjin Xu, Xuhui Jiang +5LLM Safety Large Reasoning Models
International Digital Economy Academy · The Hong Kong University of Science and Technology (Guangzhou) · DataArc Tech +1
Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
Apr 10, 2026Reetu Raj Harsh, Bhaskarjit Sarmah, Stefano PasqualiLanguage Model Safety Evaluation LLM Safety Benchmarks
Domyn, Gurugram, India · Domyn, New York, USA
PilotBench: A Benchmark for General Aviation Agents with Safety Constraints
Apr 10, 2026Yalun Wu, Haotian Liu, Zhoujun Li +1AI Agent Safety Benchmarks Trajectory Prediction
School of Computing, National University of Singapore, China · School of Informatics, Xiamen University, China · CESS, Beihang University, China
AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models
Apr 10, 2026Mintong Kang, Chen Fang, Bo LiLLM Safety Benchmarks
University of Illinois Urbana-Champaign
Beyond Social Pressure: Benchmarking Epistemic Attack in Large Language Models
Apr 9, 2026Steven Au, Sujit NoronhaLLM Safety Benchmarks Language Model Safety Evaluation
Independent Researcher
IatroBench: A Pre-Registered Benchmark of Clinical Omission in Language Models
Apr 9, 2026David GringrasLLM Safety Benchmarks Healthcare
Harvard T.H. Chan School of Public Health

May 14, 2026J/K move · Enter open · S save

The Great Pretender: A Stochasticity Problem in LLM Jailbreak

Jean-Philippe Monteuuis, Cong Chen, Jonathan Petit

Qualcomm Technologies, Inc.

LLM Safety Benchmarks LLM Evaluation Adversarial Prompt Generation LLM Jailbreak Attacks Consistent Adversarial Attacks LLM Safety Evaluation+2

Open paper AskPDF arXiv

"Oh-Oh, yes, I'm the great pretender. Pretending that I'm doing well. My need is such, I pretend too much..." summarizes the state in the area of jailbreak creation and evaluation. You find this method to generate adversarial attacks proposed by a reputable institution (e.g., BoN from Anthropic or Crescendo from Microsoft Research). However, this method does not deliver on the promise claimed in the paper despite having top ASR scores against industry-grade LLMs. You successfully generate the jailbreak prompts against your target (open) model. However, the generated jailbreak prompt works against the target model with a 50% consecutive success rate (5 out of 10 attempts) despite having an 80% ASR (on paper) on the latest closed-source model (with a guardrail system)! This observation leads us to think. First, Attack Success Rate (ASR), the primary metric for LLM jailbreak benchmarking, is not a stable quantity. Second, published ASR numbers are therefore systematically inflated and incomparable across papers. Therefore, we wonder "Why a successful jailbreak prompt does not perform consistently well against a target model on which the prompts have been optimized?". To answer this question, we study the impact of stochasticity not only during attack evaluation but also during attack generation. Our evaluation includes several jailbreak attacks, models (different sizes and providers), and judges. In addition, we propose a new metric and two new frameworks (CAS-eval and CAS-gen). Our evaluation framework, CAS-eval, shows that an attack can have an ASR drop of up to 30 percentage points when a jailbreak prompt needs to succeed on more than one attempt. Thankfully, our attack generation framework (CAS-gen) improves previous jailbreak methods and helps them recover this loss of 30 percentage points!