Language Model Safety Evaluation

Follow this topic Ask about this topic

Language model safety evaluation asks whether models handle harmful, ambiguous, and high-stakes requests safely while still helping with legitimate ones. Researchers use benchmarks such as HarmBench, JailbreakBench, and XSTest, alongside static and adaptive red-teaming, expert review, and automated judges; newer tests also examine multi-turn, tool-using, speech, and multimodal systems. Psychometric methods, repeated sampling, and internal activation probes can reveal risks that single-run refusal scores miss, while localized and domain-specific evaluations test language, culture, and settings such as healthcare or finance. A central challenge is making results reliable and predictive of deployment: robustness can shift with wording, language, interaction length, or evaluator, and reducing harmful compliance can increase over-refusal or undermine useful performance.

Latest papers

550 papers82 papers in the last four weeks, up 64% on the four weeks before. 0.5% of all new papers.

Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking
Feb 27, 2026Zhicheng Fang, Jingjie Zheng, Chenxu Fu +1Jailbreak Attack Evaluation LLM Safety Benchmarks
Shanghai Qi Zhi Institute, Shanghai, China · University of Melbourne, Melbourne, Australia · Tsinghua University, Beijing, China
CourtGuard: A Model-Agnostic Framework for Zero-Shot Policy Adaptation in LLM Safety
Feb 26, 2026Umid Suleymanov, Rufiz Bayramov, Suad Gafarli +4Zero-Shot Policy Transfer Language Model Safety Evaluation
Department of Computer Science, Virginia Tech · School of IT and Engineering, ADA University · School of Law, ADA University
Hidden Topics: Measuring Sensitive AI Beliefs with List Experiments
Feb 25, 2026Maxim ChupilkinProbability Elicitation from Language Models LLM Alignment
The Ghost in the Grammar: Methodological Anthropomorphism in AI Safety Evaluations
Feb 24, 2026Mariana Lins CostaAnthropomorphism in AI AI Safety
Universidade Estadual do Ceará (Brazil), Professor of Philosophy
Analysis of LLMs Against Prompt Injection and Jailbreak Attacks
Feb 24, 2026Piyush Jaiswal, Aaditya Pratap, Shreyansh Saraswati +2Prompt Injection Attacks on LLMs Language Model Safety Evaluation
NIT Trichy, Tiruchirappalli, India · NCE Chandi, Nalanda, India · Bishop Cotton Boys’ School, Bengaluru, India +2
Three Concrete Challenges and Two Hopes for the Safety of Unsupervised Elicitation
Feb 23, 2026Callum Canavan, Aditya Shrivastava, Allison Qi +2LLM Reliability Language Model Safety Evaluation
MATS · Anthropic Fellows Program · Anthropic
When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models
Feb 21, 2026Zafir Shamsi, Nikhil Chekuru, Zachary Guzman +1Adversarial Attacks on LLMs Language Model Safety Evaluation
Algoverse AI Research
Luna-2: Scalable Single-Token Evaluation with Small Language Models
Feb 20, 2026Vatsal Goel, Rishon Dsouza, Nikhil Ega +4LLM Evaluation Language Model Safety Evaluation
Galileo AI, San Francisco, USA
LLM Spirals of Delusion: A Benchmarking Audit Study of AI Chatbot Interfaces
Feb 20, 2026Peter Kirgis, Ben Hawriluk, Sherrie Feng +3Language Model Safety Evaluation LLM Safety Benchmarks
Princeton University · Phillips Exeter Academy · Independent
Trojan Horses in Recruiting: A Red-Teaming Case Study on Indirect Prompt Injection in Standard vs. Reasoning Models
Feb 19, 2026Manuel WirthIndirect Prompt Injection Prompt Injection Attacks on LLMs
University of Mannheim
AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue
Feb 19, 2026Adib Sakhawat, Fardeen Sadab, Rakin ShahriarPartially Observable Stochastic Games Multi-Turn Dialogue Evaluation
Department of Computer Science and Engineering, Islamic University of Technology, Dhaka, Bangladesh
DeepContext: Stateful Real-Time Detection of Multi-Turn Adversarial Intent Drift in LLMs
Feb 18, 2026Justin Albrethsen, Yash Datta, Kunal Kumar +1LLM Safety Jailbreak Defense
Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMs
Feb 17, 2026Anna Soligo, Vladimir Mikulik, William SaundersLanguage Model Safety Evaluation LLM Reliability
Imperial College London · Anthropic Fellows · Anthropic
Alignment as Iatrogenesis: Pastoral Power, Collective Pathology, and the Structural Limits of Monolingual Safety Evaluation
Feb 17, 2026Hiroki FukuiLLM Alignment Language Model Safety Evaluation
Department of Neuropsychiatry, Graduate School of Medicine, Kyoto University
NEST: Nascent Encoded Steganographic Thoughts
Feb 15, 2026Artem KarpovLanguage Model Safety Evaluation LLM Auditing
Independent.
AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation
Feb 4, 2026Kate H. Bentley, Luca Belli, Adam M. Chekroud +7Language Model Safety Evaluation Human-AI Agreement
Spring Health · UC Berkeley · Yale University
CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News Generation
Feb 4, 2026Zhao Tong, Chunlin Gong, Yiping Zhang +5Language Model Safety Evaluation LLM Interpretability
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Shanghai AI Laboratory +3
SalamahBench: Dialect and Category Level Safety Evaluation of Arabic Language Models
Feb 3, 2026Omar Abdelnasser, Fatemah Alharbi, Khaled Khasawneh +2LLM Safety Benchmarks Multilingual Language Model Evaluation
Compumacy for Artificial Intelligence Solutions, Cairo, Egypt · Computer Science Department, College of Computer Science and Engineering, Taibah University, Yanbu 46522, KSA · Electrical and Computer Engineering Department, George Mason University, VA, USA +2
Risky-Bench: Evaluating Agentic Safety Risks in Real-World Life-Assist Deployments
Feb 3, 2026Jingnan Zheng, Yanzhen Luo, Jingjun Xu +8Language Model Safety Evaluation LLM Agent Safety
National University of Singapore · University of Science and Technology of China · Southern University of Science and Technology +3
VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models
Jan 27, 2026Yuxiang Wang, Hongyu Liu, Dekun Chen +2Privacy-Preserving Language Models Language Model Safety Evaluation
The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute · Amphion Technology Co., Ltd. +1
Unknown Unknowns: Do Hidden Intentions in LLMs Evade Detection?
Jan 26, 2026Devansh Srivastav, David Pape, Lea SchönherrLanguage Model Safety Evaluation LLM Auditing
CISPA Helmholtz Center for Information Security, Saarbrücken, Germany
Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context
Jan 25, 2026Zhihao Zhang, Liting Huang, Guanghao Wu +3LLM Safety Benchmarks Language Model Safety Evaluation
Macquarie University1 · UTS2 · MBZUAI3 +1
SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care
Jan 23, 2026Dongshen Peng, Yi Wang, Austin Schoeffler +5LLM Sycophancy Language Model Safety Evaluation
UNC Chapel Hill · University of Waterloo · Stanford University
RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions
Jan 19, 2026Drishti Goel, Jeongah Lee, Qiuyue Joy Zhong +5Language Model Safety Evaluation LLM Safety Evaluation
University of Illinois Urbana-Champaign · University of Massachusetts Amherst · OSF HealthCare +1
In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement
Jan 19, 2026Anudeex Shetty, Aditya Joshi, Salil S. KanhereLanguage Model Safety Evaluation Adversarial Attacks on LLMs
School of Computer Science and Engineering, UNSW Sydney, Australia · School of Computing and Information System, the University of Melbourne, Australia
Preserving Fairness and Safety in Quantized LLMs Through Critical Weight Protection
Jan 17, 2026Muhammad Alif Al Hakim, Alfan Farizki Wicaksono, Fajri KotoLLM Quantization Multilingual Language Model Evaluation
Faculty of Computer Science, Universitas Indonesia · Mohamed bin Zayed University of Artificial Intelligence
Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignment
Jan 8, 2026Han Zhu, Jiale Chen, Chengkun Cai +8Language Model Safety Evaluation Multimodal Large Language Models
Hong Kong University of Science and Technology · University College London · AISpeech +1
Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
Jan 7, 2026Iago Alves Brito, Walcy Santos Rezende Rios, Julia Soares Dollis +2LLM Safety Benchmarks LLM Alignment
Advanced Knowledge Center for Immersive Technologies (AKCIT) Federal University of Goiás (UFG)
Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs
Jan 5, 2026Amirali Ebrahimzadeh, Seyyed M. SaliliLong-Context Retrieval LLM Hallucination Mitigation
Department of Electrical Engineering & Computer Science, University of Michigan, Ann Arbor, MI 48109, USA · Independent Researcher, Ann Arbor, MI 48106, USA
The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
Dec 12, 2025Erik LarsenLanguage Model Safety Evaluation LLM Refusal Behavior

Feb 27, 2026J/K move · Enter open · S save

Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking

Zhicheng Fang, Jingjie Zheng, Chenxu Fu, Wei Xu

Shanghai Qi Zhi Institute, Shanghai, China · University of Melbourne, Melbourne, Australia · Tsinghua University, Beijing, China

Jailbreak Attack Evaluation LLM Safety Benchmarks Language Model Safety Evaluation LLM Jailbreak Attacks

Open paper AskPDF arXiv

Jailbreak techniques for large language models (LLMs) evolve faster than benchmarks, making robustness estimates stale and difficult to compare across papers due to drift in datasets, harnesses, and judging protocols. We introduce JAILBREAK FOUNDRY (JBF), a system that addresses this gap via a multi-agent workflow to translate jailbreak papers into executable modules for immediate evaluation within a unified harness. JBF features three core components: (i) JBF-LIB for shared contracts and reusable utilities; (ii) JBF-FORGE for the multi-agent paper-to-module translation; and (iii) JBF-EVAL for standardizing evaluations. Across 30 reproduced attacks, JBF achieves high fidelity with a mean (reproduced-reported) attack success rate (ASR) deviation of +0.26 percentage points. By leveraging shared infrastructure, JBF reduces attack-specific implementation code by more than half relative to original repositories and achieves an 82.5% mean reused-code ratio. This system enables a standardized AdvBench evaluation of all 30 attacks across 10 victim models using a consistent GPT-4o judge. By automating both attack integration and standardized evaluation, JBF offers a scalable solution for creating living benchmarks that keep pace with the rapidly shifting security landscape.