LLM Safety Benchmarks

LLM: Large Language Model

This field measures how reliably language models recognize and handle risks such as harmful requests, jailbreaks, bias, privacy violations, and unsafe tool use. Benchmarks use curated and expert-annotated scenarios, adversarial red-teaming, multi-turn and agent simulations, and increasingly multilingual, multimodal, and domain-specific tests. Researchers also compare refusal and moderation methods, use repeated sampling and statistical techniques to estimate reliability, and assess both harmful compliance and over-refusal. A central challenge is making results comparable and trustworthy while ensuring tests reflect changing, context-dependent behavior in real deployments.

Latest papers

284 papers44 papers in the last four weeks, up 7% on the four weeks before. 0.3% of all new papers.

SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing
Jun 29, 2026Jiacheng Zhang, Haoyu He, Sen Zhang +5LLM Safety Benchmarks Language Model Safety Evaluation
1ByteDance · 2The University of Melbourne
CAREBench: A Child-Safety Risk Benchmark for Language Models
Jun 29, 2026Kaavya Krishna-Kumar, Elaine Lau, Vaughn Robinson +6LLM Safety Benchmarks Language Model Safety Evaluation
Handshake AI · University of California, Los Angeles · McGill University
The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis
Jun 28, 2026Hari Prasad, Ritam PalLLM Safety Benchmarks LLM Quantization
Conscious Engines
The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning
Jun 27, 2026Will Hawkins, Kaivalya Rawal, Jonathan Rystrøm +8LLM Safety Benchmarks Multilingual Language Model Evaluation
Oxford Internet Institute, University of Oxford · Hasso Plattner Institute, University of Potsdam · Chinese University of Hong Kong +2
Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety
Jun 26, 2026Ting Ma, Xiufeng Huang, Benlei Cui +43LLM Safety Benchmarks LLM Safety Alignment
Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation
Jun 24, 2026Han Jeon, Shiv Medler, Joseph Voyles +1LLM Safety Benchmarks LLM-as-a-Judge
PricewaterhouseCoopers, U.S.A.
FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming
Jun 18, 2026Daeyoung Park, Chaeyun Kim, Junghwan Kim +4LLM Safety Benchmarks Financial Services
Financial Security Institute (FSI), Republic of Korea · Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · DATUMO INC., Republic of Korea
SciRisk-Bench: A Risk-Dimension-Aware Benchmark for AI4Science Safety
Jun 17, 2026Linghao Feng, Yinqian Sun, Dongqi Liang +8LLM Safety Benchmarks AI for Science
Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Future Technology, University of Chinese Academy of Sciences, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, China +5
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
Jun 16, 2026Yuchuan Tian, Mengyu Zheng, Haocheng Mei +5LLM Safety Benchmarks LLM Agent Security
Peking University · Beijing Jiaotong University · SUIBE +2
MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions
Jun 15, 2026Noor Islam S. Mohammad, Tamim SheikhLLM Safety Benchmarks LLM Auditing
Department of Computer Science, Informatics Institute, Istanbul Technical University, ˙Istanbul, Türkiye · Department of Computer Science and Engineering, Jashore University of Science and Technology, Bangladesh.
CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment
Jun 13, 2026Wenbo Yu, Bohua Wang, Hao Fang +10LLM Safety Benchmarks Language Model Safety Evaluation
Tsinghua University · Beijing Normal University · South China University of Technology +2
BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems
Jun 12, 2026Leonhard Waibl, Felix Michalak, Hadrien MariacciaLLM Safety Benchmarks LLM Guardrails
University of Graz, Graz, Austria · Supervised Program for Alignment Research (SPAR) · Centre pour la Sécurité de l’IA (CeSIA), Paris, France.
Schützen: Evaluating LLM Safety in Bulgarian and German Contexts
Jun 9, 2026Kiril Georgiev, Yuxia Wang, Dimitar Iliyanov Dimitrov +2LLM Safety Benchmarks Multilingual Language Model Evaluation
Sofia University "St. Kliment Ohridski" · INSAIT, Sofia University “St. Kliment Ohridski” · MBZUAI
AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
Jun 3, 2026Yanjing Ren, Reza Ebrahimi, TengTeng MaLLM Safety Benchmarks LLM-as-a-Judge
The School of Information Systems University of South Florida Tampa, US
PreAct-Bench: Benchmarking Predictive Monitoring in LLMs
Jun 3, 2026Hainiu Xu, Italo Luis da Silva, Jiangnan Ye +7LLM Safety Benchmarks Moral Reasoning in Language Models
King’s College London · National University of Singapore · Southern University of Science and Technology +3
Gate AI: LLM Security Benchmark Evaluation Methodology and Results
Jun 1, 2026Ryle Goehausen, Marcus SousaLLM Safety Benchmarks Language Model Safety Evaluation
TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages
May 31, 2026Victor Akinode, Senyu Li, Wassim Hamidouche +3LLM Safety Benchmarks Multilingual Language Model Evaluation
Mila - Quebec AI Institute · McGill University · Microsoft AI for Good Research Lab +1
Triaging Threats to Specialized Guardrails
May 29, 2026Wenjie Jacky Mo, Xiaofei Wen, Rui Cai +6LLM Safety Benchmarks LLM Guardrails
University of California, Davis
When Context Flips, Safety Breaks: Diagnosing Brittle Safety in Aligned Language Models
May 27, 2026Dasol Choi, Alex KwonLLM Safety Benchmarks Counterfactual Evaluation
AIM Intelligence · Independent Researcher
Efficient Safety Benchmarking via Item Response Theory
May 26, 2026Fabio Spagliardi, Mírian Silva, Ayan Datta +3LLM Safety Benchmarks Language Model Safety Evaluation
Supervised Program for Alignment Research (SPAR) · Yale University · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries
May 26, 2026Yige Li, Jun Sun, Wei Zhao +5LLM Safety Benchmarks Healthcare
Singapore Management University · Deakin University · The University of Melbourne +2
AlbanianLLMSafety: A Safety Evaluation Dataset for Large Language Models in Albanian
May 26, 2026Wajdi Zaghouani, Kholoud K. Aldous, Isra FejzullajLLM Safety Benchmarks Language Model Safety Evaluation
Northwestern University in Qatar
KZ-SafetyPrompts: A Kazakh Safety Evaluation Prompt Dataset for Large Language Models
May 26, 2026Wajdi Zaghouani, Shimaa Amer Ibrahim, Aruzhan Muratbek +2LLM Safety Benchmarks Language Model Safety Evaluation
Northwestern University in Qatar
Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety
May 25, 2026Fay Elhassan, David Sasu, Alexandra Kulinkina +2LLM Safety Benchmarks Human Preference Evaluation
LiGHT Laboratory, EPFL, Lausanne, Switzerland
The Age of Curiosity Meets the Age of AI: Benchmarking Child Safety in Large Language Models
May 25, 2026Samee Arif, Angana Borah, Rada MihalceaLLM Safety Benchmarks Language Model Safety Evaluation
University of Michigan
SomaliBench Eval: Measuring English-to-Somali Refusal Gaps in Open-Weight Language Models
May 25, 2026Khalid Yusuf DahirLLM Safety Benchmarks Multilingual Language Model Evaluation
Independent researcher
PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMs
May 22, 2026Luze Sun, Anshuman Suri, Harsh Chaudhari +2LLM Safety Benchmarks LLM Backdoor Attacks
Department of Computer Science Northeastern University · DatologyAI
Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
May 21, 2026Piercosma Bisconti, Matteo Prandi, Federico Pierucci +11LLM Safety Benchmarks Computer-Use Agent Benchmarks
Icaro Foundation · Sapienza University of Rome · Sant’Anna School of Advanced Studies +6
RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts
May 20, 2026Lukas Weidener, Marko Brkić, Mihailo Jovanović +2LLM Safety Benchmarks Language Model Safety Evaluation
Applied Scientific Intelligence, Inc.
Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks
May 14, 2026Karthik Raghu Iyer, Yazdan Jamshidi, Nicholas Bray +1LLM Safety Benchmarks Benchmark Auditing
Palo Alto Networks

Jun 29, 2026J/K move · Enter open · S save

SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing

Jiacheng Zhang, Haoyu He, Sen Zhang, Shen Wang+4

1ByteDance · 2The University of Melbourne

LLM Safety Benchmarks Language Model Safety Evaluation LLM Guardrails

Open paper AskPDF arXiv

In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on predefined risk taxonomies. In this work, we study this setting under the paradigm of in-context policy guardrailing, where guardrails predict safety violations based on policy specifications provided in context. To systematically evaluate this capability, we introduce SafePyramid, a safety benchmark comprising 1,000 multi-turn conversations across 10 domains and 3,000 corresponding application-specific policies, which together contain 61,699 distinct natural-language rules. SafePyramid organizes the evaluation into three difficulty levels: L0 evaluates individual-rule understanding, L1 evaluates reasoning over rule dependencies, and L2 evaluates adaptation of full novel policy frameworks defined in context. To ensure benchmark quality, we employ a rigorous multi-stage pipeline to construct and validate the benchmark. Using SafePyramid, we evaluate 10 frontier LLMs and 5 policy-configurable guardrails and find that in-context policy guardrailing remains highly challenging: even the best-performing model, GPT-5.5, exactly identifies the full set of violated rules in only 54.0%, 35.3%, and 12.9% cases on L0, L1, and L2, respectively. These results highlight the limitations of current guardrails and call for stronger in-context policy guardrails that can reliably execute policies, resolve rule dependencies, and adapt to novel policy frameworks.