Language Model Safety Evaluation

Ask about this topic

Overview

Language model safety evaluation asks whether models handle harmful, ambiguous, and high-stakes requests safely while still helping with legitimate ones. Researchers use benchmarks such as HarmBench, JailbreakBench, and XSTest, alongside static and adaptive red-teaming, expert review, and automated judges; newer tests also examine multi-turn, tool-using, speech, and multimodal systems. Psychometric methods, repeated sampling, and internal activation probes can reveal risks that single-run refusal scores miss, while localized and domain-specific evaluations test language, culture, and settings such as healthcare or finance. A central challenge is making results reliable and predictive of deployment: robustness can shift with wording, language, interaction length, or evaluator, and reducing harmful compliance can increase over-refusal or undermine useful performance.

Latest papers

CardsList

550 papers82 papers in the last four weeks, up 64% on the four weeks before. 0.5% of all new papers.

  1. An Independent Safety Evaluation of Kimi K2.5

    Apr 3, 2026Zheng-Xin Yong, Parv Mahajan, Andy Wang +12Language Model Safety EvaluationLLM Security

    Constellation · Brown University · Georgia Institute of Technology +7

  2. A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework

    Apr 3, 2026Chenyu Li, Zohaib Akhtar, Mingu Kwak +13LLM-as-a-JudgeClinical Language Model Evaluation

    Department of Biomedical Informatics, School of Medicine, University of Pittsburgh, Pittsburgh, PA, USA · Department of Health Information Management, School of Health and Rehabilitation Sciences, University of Pittsburgh, Pittsburgh, PA, USA · OpenCura, Health Innovation Consortium +6

  3. Blind Refusal: Language Models Refuse to Help Users Evade Unjust, Absurd, and Illegitimate Rules

    Apr 3, 2026Cameron Pattison, Lorenzo Manuali, Seth LazarRefusal Behavior in Language ModelsRefusal Mechanisms in Language Models

    Department of Philosophy, Vanderbilt University · Department of Philosophy, University of Michigan · School of Government and Policy, Johns Hopkins University

  4. Generalization Limits of Reinforcement Learning Alignment

    Apr 3, 2026Haruhi Shida, Koo Imai, Keigo KansaLLM AlignmentAdversarial Attacks on LLMs

    Aladdin Security Inc.

  5. Quantifying Self-Preservation Bias in Large Language Models

    Apr 2, 2026Matteo Migliarini, Joaquin Pereira Pizzini, Luca Moresca +3Language Model Safety EvaluationLLM Safety Benchmarks

    Sapienza University · ItalAI

  6. Hidden Reliability Risks in Large Language Models: Systematic Identification of Precision-Induced Output Disagreements

    Apr 2, 2026Yifei Wang, Tianlin Li, Xiaohan Zhang +4LLM ReliabilityLLM Evaluation

    Shanghai Jiao Tong University, China · Beihang University, China · Nanyang Technological University, Singapore +1

  7. Adversarial Moral Stress Testing of Large Language Models

    Apr 1, 2026Saeid Jamshidi, Foutse Khomh, Arghavan Moradi Dakhel +3Language Model Safety EvaluationAdversarial Robustness of Language Models

    are with the SWAT Laboratory, Polytechnique Montréal, Montréal, QC, Canada · is with the Huawei Distributed Scheduling and Data Engine Lab, Montréal, QC, Canada · is with the SæT Laboratory, Polytechnique Montréal, Montréal, QC, Canada

  8. Evaluation and Hardening of LLM System Instructions Against Extraction via Encoding Attacks

    Apr 1, 2026Anubhab Sahu, Diptisha Samanta, Reza SoosahabiLLM SecurityPrivacy Leakage in Language Models

    Keysight Technologies, India · Keysight Technologies, USA

  9. One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety

    Apr 1, 2026Samee Arif, Naihao Deng, Zhijing Jin +1LLM Jailbreak AttacksLanguage Model Safety Evaluation

    University of Michigan · University of Toronto

  10. Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundaries

    Mar 31, 2026Tanay GondilLanguage Model IntrospectionRefusal Behavior in Language Models

    Purdue University

  11. Kill-Chain Canaries: Stage-Level Tracking of Prompt Injection Across Attack Surfaces and Five Production LLMs

    Mar 30, 2026Haochuan Kevin Wang, Zechen ZhangMulti-Agent LLM SystemsLanguage Model Safety Evaluation

    Massachusetts Institute of Technology · University of Chicago

  12. Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs

    Mar 29, 2026Alexandre Cristovão MaioranoPrompt Injection DefenseLLM Guardrails

  13. The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams

    Mar 28, 2026Isaac Llorente-SaguerAnomaly DetectionLanguage Model Safety Evaluation

    Independent Researcher

  14. Squish and Release: Exposing Hidden Hallucinations by Making Them Surface as Safety Signals

    Mar 27, 2026Nathaniel Oh, Paul AttieHallucination in Language ModelsLLM Hallucination Mitigation

    School of Computer and Cyber Sciences Augusta University Augusta, Georgia 30912

  15. Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models

    Mar 26, 2026Xunguang Wang, Yuguang Zhou, Qingyue Wang +5LLM Safety MonitoringLLM Safety

    The Hong Kong University of Science and Technology, Hong Kong, China · Zhejiang University of Technology, Hangzhou, Zhejiang Province, China

  16. Evaluating Language Models for Harmful Manipulation

    Mar 26, 2026Canfer Akbulut, Rasmi Elasmar, Abhishek Roy +9Language Model Safety EvaluationPersuasion in Language Models

    Google DeepMind · Google

  17. Analysing the Safety Pitfalls of Steering Vectors

    Mar 25, 2026Yuxiao Li, Alina Fastowski, Efstratios Zaradoukas +2Activation SteeringLanguage Model Safety Evaluation

    Technical University of Munich, Munich Center for Machine Learning

  18. How Utilitarian Are OpenAI's Models Really? Replicating and Reinterpreting Pfeffer, Krügel, and Uhl (2025)

    Mar 24, 2026Johannes HimmelreichPrompt SensitivityMoral Reasoning in Language Models

  19. Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?

    Mar 23, 2026Richard J. YoungCoT FaithfulnessFaithfulness of Language Model Explanations

    University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA

  20. SecureBreak -- A dataset towards safe and secure models

    Mar 23, 2026Marco Arazzi, Vignesh Kumar Kembu, Antonino NoceraLLM SafetyLanguage Model Safety Evaluation

    Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy

  21. Structured Visual Narratives Undermine Safety Alignment in Multimodal Large Language Models

    Mar 23, 2026Rui Yang Tan, Yujia Hu, Roy Ka-Wei LeeMLLM SafetyAdversarial Attacks on VLMs

    Singapore University of Technology and Design. · University of British Colombia.

  22. The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialogues

    Mar 21, 2026Jocelyn Shen, Amina Luvsanchultem, Jessica Kim +6Opinion DynamicsLanguage Model Safety Evaluation

    ♣Massachusetts Institute of Technology, Cambridge, MA, USA · ♢Carnegie Mellon University, Pittsburgh, PA, USA

  23. Measuring Weak-to-Strong Legibility of Reasoning Models

    Mar 20, 2026Dani Roytburg, Shreya Sridhar, Daphne IppolitoLarge Reasoning ModelsLanguage Model Safety Evaluation

    Machine Learning Department, Carnegie Mellon University, Pittsburgh, United States · Language Technologies Institute, Carnegie Mellon University, Pittsburgh, United States

  24. AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?

    Mar 20, 2026Soorya Ram Shimgekar, Vipin Gunda, Jiwon Kim +3Mental Health Conversational AgentsAI Safety

    University of Illinois Urbana-Champaign

  25. Detection Is Cheap, Routing Is Learned: Why Refusal-Based Alignment Evaluation Fails

    Mar 18, 2026Gregory N. FrankLanguage Model Safety EvaluationRefusal Behavior in Language Models

    Independent Researcher

  26. IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia

    Mar 18, 2026Priyaranjan Pattnayak, Sanchari ChowdhuriLanguage Model Safety EvaluationLLM Safety Benchmarks

    Oracle America Inc.

  27. Harm or Humor: A Multimodal, Multilingual Benchmark for Overt and Covert Harmful Humor

    Mar 18, 2026Ahmed Sharshar, Hosam Elgendy, Saad El Dine Ahmed +2LLM Safety BenchmarksLanguage Model Safety Evaluation

    Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria

  28. Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generation

    Mar 18, 2026Jianan Chen, Zhifang Zhang, Shuo He +3LLM Safety AlignmentLarge Reasoning Models

    Southeast University, China · The University of Queensland, Australia · Nanyang Technological University, Singapore

  29. Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memory

    Mar 16, 2026Ce Zhang, Jinxi He, Junyi He +2Multimodal Large Language ModelsLanguage Model Safety Evaluation

    Robotics Institute, Carnegie Mellon University

  30. SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration

    Mar 16, 2026Yu Pan, Wenlong Yu, Tiejun Wu +4LLM Safety AlignmentJailbreak Defense

    Department College of Intelligence and Computing, Tianjin University · NSFOCUS Technologies Group Co., Ltd. · College of Management and Economics, Tianjin University +1