LLM Safety Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, level with the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 177

All topics

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains
Apr 21, 2026Ishita Kakkar, Enze Zhang, Rheeya Uppaal +1AI Safety Evaluation LLM Safety Evaluation
Department of Computer Sciences, University of Wisconsin-Madison
Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safety
Apr 20, 2026Marcello Galisai, Susanna Cifani, Francesco Giarrusso +5LLM Safety Benchmarks Adversarial Attacks on LLMs
DEXAI – Icaro Lab · Sapienza University of Rome · Sant’Anna School of Advanced Studies
Using large language models for embodied planning introduces systematic safety risks
Apr 20, 2026Tao Zhang, Kaixian Qu, Zhibin Li +4Large Language Model-Based Robot Planning Robot Safety
ETH Zurich, Zurich, Switzerland. · University College London, London, United Kingdom. · Stanford University, Stanford, California, United States. +2
TPS-CalcBench: A Benchmark and Diagnostic Evaluation Framework for LLM Analytical Calculation Competence in Hypersonic Thermal Protection System Engineering
Apr 20, 2026Jinglai Zheng, Chuhan Qiao, Haiming HuangSynthetic Benchmark Generation LLM Safety Evaluation
School of Civil Engineering, Beijing Jiaotong University, Beijing 100044, China
MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models
Apr 20, 2026Suhyun Lee, Palakorn Achananuparp, Neemesh Yadav +2Mental Health Counseling LLM Evaluation
Department of Artificial Intelligence, Hanyang University, Seoul, Republic of Korea · School of Computing and Information Systems, Singapore Management University, Singapore
On Safety Risks in Experience-Driven Self-Evolving Agents
Apr 18, 2026Weixiang Zhao, Yichen Zhang, Yingshuo Wang +8LLM Safety Evaluation Over-Refusal
Harbin Institute of Technology · Singapore Management University · SERES
When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints
Apr 18, 2026Yuheng Chen, Zhiyu Wu, Bowen Cheng +2Adversarial Attacks on LLMs LLM Safety Alignment
Kagoshima University · Fudan University · China University of Petroleum-Beijing
HealthCraft: A Reinforcement Learning Safety Environment for Emergency Medicine
Apr 18, 2026Brandon DentHealthcare AI Agent Benchmarks
GOATnote Inc.
A Systematic Study of Training-Free Methods for Trustworthy Large Language Models
Apr 17, 2026Wai Man Si, Mingjie Li, Michael Backes +1LLM Alignment LLM Safety
CISPA Helmholtz Center for Information Security
Safety Training Modulates Harmful Misalignment Under On-Policy RL, But Direction Depends on Environment Design
Apr 14, 2026Leon Eshuijs, Shihan Wang, Antske FokkensReward Hacking RL for Language Models
Vrije Universiteit Amsterdam · Utrecht University
IatroBench: A Pre-Registered Benchmark of Clinical Omission in Language Models
Apr 9, 2026David GringrasLLM Safety Benchmarks Healthcare
Harvard T.H. Chan School of Public Health
Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations
Mar 31, 2026Yahan Li, Xinyi Jie, Wanjia Ruan +5Healthcare Clinical Reasoning
University of Southern California
Peer-Preservation in Frontier Models
Mar 30, 2026Yujin Potter, Nicholas Crispino, Vincent Siu +2AI Agent Safety LLM Safety Evaluation
University of California, Berkeley · University of California, Santa Cruz
SafeMath: Safe Solutions for Unsafe Math Word Problems
Mar 26, 2026Sagnik Basu, Subhrajit Mitra, Aman Juneja +3LLM Safety Alignment LLM Safety Evaluation
1Indian Institute of Technology Kharagpur · 2Cisco Systems · 3National University of Singapore
SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems
Mar 18, 2026Rima Hazra, Bikram Ghuku, Ilona Marchenko +5Intelligent Tutoring Systems AI Safety Evaluation
Eindhoven University of Technology, Netherlands (TU/e) · Indian Institute of Technology Kharagpur · Taras Shevchenko National University of Kyiv +3
Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services
Mar 11, 2026Fabrizio Dimino, Bhaskarjit Sarmah, Stefano PasqualiLanguage Model Safety Evaluation LLM Safety
1Domyn, New York, US · 2Domyn, New York, US · 3Domyn, New York, US
Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety
Mar 8, 2026David GringrasLLM Safety Benchmarks LLM Agent Evaluation
Harvard University
Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas
Jan 29, 2026Katherine Elkins, Jon ChunMoral Reasoning in Language Models LLM Auditing
Kenyon College Gambier, OH
Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions
Dec 26, 2025Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi +2Healthcare LLM Safety
Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. · Department of Anesthesiology and Intensive Care, Mashhad University of Medical Sciences, Mashhad, Iran. +2
Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data
Dec 25, 2025Yiwei Yan, Guanfeng LiuHealthcare Text Classification
School of Computing, Macquarie University, Australia
Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
Dec 21, 2025Zhang Wei, Hanxuan Chen, Peilu Hu +19Adversarial Robustness of Language Models LLM Safety Evaluation
Independent Researcher · Hunan University · Shenzhen Kaihong Digital Industry Development Co., Ltd. +12
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
Oct 17, 2025Hanbin Hong, Shuang Wu, Shuya Feng +6LLM Security LLM Jailbreak Attacks
University of Connecticut · University of Alabama at Birmingham · Independent
SafetyFlow: An Agent-Flow System for Automated LLM Safety Benchmarking
Aug 21, 2025Xiangyang Zhu, Yuan Tian, Chunyi Li +3LLM Safety Benchmarks Synthetic Benchmark Generation
Shanghai AI Lab · East China Normal University
HarmReduction: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs
Jul 29, 2025Kaixuan Wang, Chenxin Diao, Jason T. Jacques +2Healthcare Question Answering
University of Oxford, United Kingdom · University of Edinburgh, United Kingdom · University of St Andrews, United Kingdom +1
GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods
Feb 24, 2025Ruixuan Huang, Xunguang Wang, Zongjie Li +2Language Model Safety Evaluation LLM Jailbreak Attacks
The Hong Kong University of Science and Technology, Hong Kong SAR, China
CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models
Jan 24, 2025Guangzhi Sun, Xiao Zhan, Shutong Feng +2LLM Safety Benchmarks Language Model Safety Evaluation
Trinity College, University of Cambridge, Cambridge, United Kingdom · Department of Informatics, King’s College London, United Kingdom · Institut für Informatik, Heinrich-Heine-Universität Düsseldorf, Germany +1
Who Judges the Judges? A Chinese Safety QA Benchmark for Evaluating LLM Responses and Safety Judges
Date pendingRui Yang, Shuang Huang, Junhua Liu +7LLM Safety Benchmarks LLM-as-a-Judge
Anhui SparkShield Intelligent Technology · iFLYTEK · Peking University +1

Apr 21, 2026, cs.CLJ/K move · Enter open · S save

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains

Ishita Kakkar, Enze Zhang, Rheeya Uppaal, Junjie Hu

Department of Computer Sciences, University of Wisconsin-Madison

AI Safety Evaluation LLM Safety Evaluation Large Reasoning Models Reasoning Trace Analysis

Open paper AskPDF arXiv

Large reasoning models (LRMs) produce complex, multi-step reasoning traces, yet safety evaluation remains focused on final outputs, overlooking how harm emerges during reasoning. When jailbroken, harm does not appear instantaneously but unfolds through distinct behavioral steps such as suppressing refusal, rationalizing compliance, decomposing harmful tasks, and concealing risk. However, no existing benchmark captures this process at sentence-level granularity within reasoning traces -- a key step toward reliable safety monitoring, interventions, and systematic failure diagnosis. To address this gap, we introduce HarmThoughts, a benchmark for step-wise safety evaluation of reasoning traces. \ourdataset is built on our proposed harm taxonomy of 16 harmful reasoning behaviors across four functional groups that characterize how harm propagates rather than what harm is produced. The dataset consists of 56,931 sentences from 1,018 reasoning traces generated by four model families, each annotated with fine-grained sentence-level behavioral labels. Using HarmThoughts, we analyze harm propagation patterns across reasoning traces, identifying common behavioral trajectories and drift points where reasoning transitions from safe to unsafe. Finally, we systematically compare white-box and black-box detectors on the task of identifying harmful reasoning behaviours on HarmThoughts. Our results show that existing detectors struggle with fine-grained behavior detection in reasoning traces, particularly for nuanced categories within harm emergence and execution, highlighting a critical gap in process-level safety monitoring. HarmThoughts is available publicly at: https://huggingface.co/datasets/ishitakakkar-10/HarmThoughts

Momentum

Related topics

Latest papers 177