Language Model Safety Evaluation

Follow this topic Ask about this topic

Language model safety evaluation asks whether models handle harmful, ambiguous, and high-stakes requests safely while still helping with legitimate ones. Researchers use benchmarks such as HarmBench, JailbreakBench, and XSTest, alongside static and adaptive red-teaming, expert review, and automated judges; newer tests also examine multi-turn, tool-using, speech, and multimodal systems. Psychometric methods, repeated sampling, and internal activation probes can reveal risks that single-run refusal scores miss, while localized and domain-specific evaluations test language, culture, and settings such as healthcare or finance. A central challenge is making results reliable and predictive of deployment: robustness can shift with wording, language, interaction length, or evaluator, and reducing harmful compliance can increase over-refusal or undermine useful performance.

Latest papers

550 papers82 papers in the last four weeks, up 64% on the four weeks before. 0.5% of all new papers.

Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF
Apr 20, 2026Yuan Fang, Yiming Luo, Aimin Zhou +1Language Model Safety Evaluation Adversarial Scenario Generation
East China Normal University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China
The Granularity Gap: A Multi-Dimensional Longitudinal Audit of Sycophancy in Gemini Models
Apr 19, 2026Patrick KeoughLLM Sycophancy LLM Alignment
Independent Researcher Bremerton, WA, USA
Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding
Apr 18, 2026Yupeng Qi, Ziyu Lyu, Lixin Cui +2Language Model Safety Evaluation LLM Refusal Behavior
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University · School of Information, Central University of Finance and Economics · School of Artificial Intelligence, Beijing Normal University +1
Do LLMs Hold Their Values? MANTA: A Multi-Turn Adversarial Benchmark for Animal Welfare Reasoning
Apr 18, 2026Isabella Luong, Joyee Chen, Arturs Kanepajs +5LLM Evaluation Moral Reasoning in Language Models
SPAR · Compassion Aligned Machine Learning · NUS +2
Jailbreaking Large Language Models with Morality Attacks
Apr 18, 2026Ying Su, Mingen Zheng, Weili Diao +1Moral Reasoning in Language Models Language Model Safety Evaluation
South China University of Technology1 · HKUST2 · Beihang University3
Evaluating Multimodal LLMs for Inpatient Diagnosis: Real-World Performance, Safety, and Cost Across Ten Frontier Models
Apr 18, 2026Bruce A. Bassett, Amy Rouillard, Sitwala Mundia +8Language Model Safety Evaluation Healthcare
School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg, South Africa · Wits MIND Institute, University of the Witwatersrand, Johannesburg, South Africa · Grai Labs, Cape Town, South Africa +3
SafeDream: Safety World Model for Proactive Early Jailbreak Detection
Apr 18, 2026Bo Yan, Weikai Lin, Yada Zhu +1Language Model Safety Evaluation World Models
University of Central Florida · University of Rochester · IBM Research
Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs
Apr 17, 2026Jaechul Roh, Virat Shejwalkar, Amir HoumansadrLanguage Model Safety Evaluation LLM Fine-Tuning
University of Massachusetts Amherst
MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
Apr 17, 2026Manh Luong, Tamas Abraham, Junae Kim +6LLM Safety Benchmarks Language Model Safety Evaluation
Monash University, Australia · Defence Science and Technology Group, Australia
TWGuard: A Case Study of LLM Safety Guardrails for Localized Linguistic Contexts
Apr 17, 2026Hua-Rong Chu, Kuan-Chun Wang, Yao-Te HuangLanguage Model Safety Evaluation LLM Guardrails
Chunghwa Telecom Laboratories
Context Over Content: Exposing Evaluation Faking in Automated Judges
Apr 16, 2026Manan Gupta, Inderjeet Nair, Lu Wang +1LLM-as-a-Judge Language Model Safety Evaluation
1BITS Pilani, India · University of Michigan
VoxSafeBench: Not Just What Is Said, but Who, How, and Where
Apr 16, 2026Yuxiang Wang, Hongyu Liu, Yijiang Xu +9Language Model Safety Evaluation Privacy Leakage in Language Models
1The Chinese University of Hong Kong, Shenzhen · 2Shenzhen Loop Area Institute · 3Amphion Technology Co., Ltd.
The A-R Behavioral Space: Execution-Level Profiling of Tool-Using Language Model Agents in Organizational Deployment
Apr 13, 2026Shasha Yu, Fiona Carroll, Barry L. BentleyLLM Agent Evaluation Tool-Using Language Agents
Cardiff School of Technologies, Cardiff Metropolitan University, Cardiff, Wales CF5 2YB, UK · School of Professional Studies, Clark University, Worcester, MA 01610 USA · Harvard Medical School, Harvard University, Boston, MA 02115 USA
Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs
Apr 13, 2026Wenkai Li, Fan Yang, Shaunak A. Mehta +1Language Model Safety Evaluation LLM Safety
Carnegie Mellon University · Fujitsu Research of America Inc.
Principles Do Not Apply Themselves: A Hermeneutic Perspective on AI Alignment
Apr 12, 2026Behrooz RazeghiAI Alignment LLM Alignment
School of Engineering and Applied Sciences, Harvard University
Weird Generalization is Weirdly Brittle
Apr 11, 2026Miriam Wanner, Hannah Collison, William Jurayj +3LLM Alignment Emergent Misalignment in Language Models
Johns Hopkins University
The Alignment Floor: How Persona Customization Breaks Safety in Weakly-Aligned LLMs
Apr 10, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +4Language Model Safety Evaluation Personalized Language Model Alignment
AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China
Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
Apr 10, 2026Reetu Raj Harsh, Bhaskarjit Sarmah, Stefano PasqualiLanguage Model Safety Evaluation LLM Safety Benchmarks
Domyn, Gurugram, India · Domyn, New York, USA
SysAdmin: Measuring Instrumental Power-Seeking in Frontier AI
Apr 10, 2026Mana Azarm, Qiyao Wei, Rahul NambiarLanguage Model Safety Evaluation AI Alignment
Analytics & Information Systems University of San Francisco · Department of Applied Mathematics and Theoretical Physics University of Cambridge · Propensity Labs
Spectral Geometry of LoRA Adapters Encodes Training Objective and Predicts Harmful Compliance
Apr 10, 2026Roi PaulLLM Auditing Low-Rank Adaptation
Independent Researcher
Functional Emotions or Situational Contexts? A Discriminating Test from the Mythos Preview System Card
Apr 9, 2026Hiranya V. PeirisEmotion Representation in Language Models Language Model Auditing
Institute of Astronomy & Kavli Institute for Cosmology, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Cavendish Laboratory, Department of Physics, University of Cambridge, JJ Thomson Avenue, Cambridge, CB3 0HE, UK
Beyond Social Pressure: Benchmarking Epistemic Attack in Large Language Models
Apr 9, 2026Steven Au, Sujit NoronhaLLM Safety Benchmarks Language Model Safety Evaluation
Independent Researcher
TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories
Apr 8, 2026Yen-Shan Chen, Sian-Yao Huang, Cheng-Lin Yang +1Language Model Safety Evaluation LLM Guardrails
CyCraft AI Lab, Taiwan · National Taiwan University
When Safety Geometry Collapses: Fine-Tuning Vulnerabilities in Agentic Guard Models
Apr 8, 2026Ismail Hossain, Sai Puppala, Jannatul Ferdaus +4LLM Guardrails LLM Safety Alignment
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · Hanyang University, Seoul, South Korea +1
To Lie or Not to Lie? Investigating The Biased Spread of Global Lies by LLMs
Apr 8, 2026Zohaib Khan, Mustafa Dogan, Ifeoma Okoh +6Deception in Language Models Language Model Safety Evaluation
Fatima Fellowship · Stanford University · Georgia Institute of Technology
Swiss-Bench 003: Evaluating LLM Reliability and Adversarial Security for Swiss Regulatory Contexts
Apr 7, 2026Fatih UenalLLM Evaluation LLM Reliability
Department of Computer Science, University of Colorado Boulder
LLM Psychosis: A Theoretical and Diagnostic Framework for Reality-Boundary Failures in Large Language Models
Apr 7, 2026Ashutosh RajLLM Reliability Language Model Safety Evaluation
Founder & CEO, NeuraCare AI | IIT Ropar TBIF Incubatee, India
Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling
Apr 6, 2026Qingyang Xu, Yaling Shen, Stephanie Fong +7LLM Safety Adversarial Attacks on LLMs
Monash University · University of Liverpool
FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment
Apr 5, 2026Daniel Kuznetsov, Ofir Cohen, Karin Shistik +2LLM Safety Alignment Adversarial Attacks on LLMs
Computer and Information Science, Ben-Gurion University of the Negev, Beer-Sheva, Israel
Structural Rigidity and the 57-Token Predictive Window: A Physical Framework for Inference-Layer Governability in Large Language Models
Apr 4, 2026Gregory M. RuddellLanguage Model Safety Evaluation Inference-Time Language Model Alignment
SnailSafe.ai — Carson City, Nevada

Apr 20, 2026J/K move · Enter open · S save

Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF

Yuan Fang, Yiming Luo, Aimin Zhou, Fei Tan

East China Normal University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China

Language Model Safety Evaluation Adversarial Scenario Generation LLM Safety Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Ensuring the safety of large language models (LLMs) requires robust red teaming, yet the systematic synthesis of high-quality toxic data remains under-explored. We propose Reverse Constitutional AI (R-CAI), a framework for automated and controllable adversarial data generation that moves beyond isolated jailbreak prompts. By inverting a harmless constitution into a constitution of toxicity and iteratively refining model outputs through a critique--revision pipeline, R-CAI enables scalable synthesis of multi-dimensional adversarial data without human annotation. Optimizing solely for toxicity-related rewards, however, can lead to reward hacking and degraded semantic coherence. To address this challenge, we introduce probability clamping within reinforcement learning from AI feedback, which stabilizes adversarial optimization while preserving adversarial intent. Experiments demonstrate that R-CAI generates diverse, high-quality toxic data and that probability clamping substantially improves semantic coherence (15%) without sacrificing adversarial strength. Overall, R-CAI provides a fully automated framework for red teaming data generation and systematic safety evaluation of aligned language models.