LLM Safety Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, level with the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 177

All topics

Muse Spark Safety & Preparedness Report
May 14, 2026Cristina Menghini, Peter Ney, Hamza Kwisaba +117Cybersecurity AI Safety Evaluation
Sail
Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks
May 14, 2026Enkelejda Kasneci, Gjergji KasneciLLM Sycophancy Intelligent Tutoring Systems
Technical University of Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany
The Great Pretender: A Stochasticity Problem in LLM Jailbreak
May 14, 2026Jean-Philippe Monteuuis, Cong Chen, Jonathan PetitLLM Safety Benchmarks LLM Evaluation
Qualcomm Technologies, Inc.
History Anchors: How Prior Behavior Steers LLM Decisions Toward Unsafe Actions
May 13, 2026Alberto G. Rodríguez SalgadoLLM Safety Evaluation AI Agent Safety Benchmarks
Independent Researcher
LLM-Based Persuasion Enables Guardrail Override in Frontier LLMs
May 13, 2026Rodrigo Nogueira, Thales Sales Almeida, Giovana Kerche Bonás +7Adversarial Prompt Generation LLM Guardrails
Maritaca AI · JusBrasil
VERA-MH: Validation of Ethical and Responsible AI in Mental Health
May 13, 2026Luca Belli, Kate H. Bentley, Josh Gieringer +6Healthcare Mental Health
Spring Health · Yale University
Confidently Deceptive: On the Relationship Between Confidence and Deception in LLMs
May 12, 2026Ali Asad, Stephen Obadinma, Anshul Pattoo +2Language Model Safety Evaluation Confidence Estimation in Language Models
Department of Electrical and Computer Engineering & Ingenuity Labs Research Institute Queen’s University, Kingston, Canada
IndustryBench: Probing the Industrial Knowledge Boundaries of LLMs
May 11, 2026Songlin Bai, Xintong Wang, Linlin Yu +12LLM Evaluation Question Answering
Multimodal and Industrial AI Team† Taobao&Tmall, Alibaba Group
Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editing
May 11, 2026Qinghua Mao, Xi Lin, Jinze Gu +3Knowledge Editing Knowledge Conflicts in Language Models
School of Computer Science · Shanghai Jiao Tong University
FragBench: Cross-Session Attacks Hidden in Benign-Looking Fragments
May 10, 2026Astha Mehta, Niruthiha Selvanayagam, Cedric Lam +10LLM Safety Benchmarks Language Model Safety Evaluation
Isabella · ÉTS Montreal / SPAR · Mila / Lida Safety
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
May 8, 2026Urchade Zaratiana, Mary Newhauser, George Hurn-Maloney +1LLM Guardrails Text Classification
Fastino Labs
Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity
May 7, 2026Florian A. D. Burnat, Brittany I. DavidsonLLM Safety Benchmarks LLM Evaluation
University of Bath
Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges
May 7, 2026Shihao Weng, Yang Feng, Xiaofei XieLLM-as-a-Judge AI Agent Safety
Nanjing University Nanjing, China · Singapore Management University Singapore
Chain of Risk: Safety Failures in Large Reasoning Models and Mitigation via Adaptive Multi-Principle Steering
May 7, 2026Xiaomin Li, Jianheng Hou, Zheyuan Deng +6LLM Safety LLM Safety Evaluation
Harvard University · University of Southern California · Brown University +6
XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity
May 7, 2026Dasol Choi, Eugenia Kim, Jaewon Noh +14LLM Safety Benchmarks Multilingual Language Model Evaluation
AIM Intelligence · Microsoft · Korea AISI +7
Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry
May 6, 2026Oriana Presacan, Andreea Grama, Larisa Irimină +6Selective Prediction Healthcare
National University of Science and Technology Politehnica Bucharest · Psychiatric Hospital Doctor Gheorghe Preda · Oslo Metropolitan University +3
Safety and accuracy follow different scaling laws in clinical large language models
May 5, 2026Sebastian Wind, Tri-Thien Nguyen, Jeta Sopa +9LLM Safety Benchmarks Retrieval-Augmented Generation
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Institute of Radiology, University Hospital Erlangen, Erlangen, Germany · Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany +3
Code World Model Preparedness Report
May 1, 2026Daniel Song, Peter Ney, Cristina Menghini +21AI Safety Evaluation LLM Safety Evaluation
MSL Preparedness Team, AI Security Team
ARMOR 2025: A Military-Aligned Benchmark for Evaluating Large Language Model Safety Beyond Civilian Contexts
Apr 30, 2026Sydney Johns, Heng Jin, Chaoyu Zhang +2LLM Safety Benchmarks LLM Safety Alignment
Virginia Polytechnic Institute and State University, Virginia, United States
Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations
Apr 29, 2026Mingqian Zheng, Malia Morgan, Liwei Jiang +2LLM Safety Benchmarks Language Model Safety Evaluation
Carnegie Mellon University · Allen Institute for AI · University of Washington
Benchmarking the Safety of Large Language Models for Robotic Health Attendant Control
Apr 29, 2026Mahiro Nakao, Kazuhiro TakemotoLanguage Model-Based Control Healthcare
Kyushu Institute of Technology Iizuka, Fukuoka, Japan
Tatemae: Detecting Alignment Faking via Tool Selection in LLMs
Apr 29, 2026Matteo Leonesi, Francesco Belardinelli, Flavio Corradini +1Deception in Language Models LLM Safety Alignment
Department of Computing Imperial College London
From Prompt Risk to Response Risk: Paired Analysis of Safety Behavior of Large Language Models
Apr 28, 2026Mengya Hu, Qiong Wei, Sandeep AtluriLanguage Model Safety Evaluation LLM Safety
Responsible AI, Microsoft
Grammar-Constrained Refinement of Safety Operational Rules Using Language in the Loop: What Could Go Wrong
Apr 26, 2026Khouloud Gaaloul, Zaid Ghazal, Madhu Latha Pulimi +1Cyber-Physical Systems Safety-Critical Control
University of Michigan–Dearborn Dearborn, MI, USA
AI Safety Training Can be Clinically Harmful
Apr 25, 2026Suhas BN, Andrew M. Sherrill, Rosa I. Arriaga +2Mental Health Counseling LLM Alignment
College of Information Sciences and Technology, Penn State University, USA · Department of Psychiatry and Behavioral Sciences, Emory University, USA · School of Interactive Computing, Georgia Tech, USA +1
Evaluating Jailbreaking Vulnerabilities in LLMs Deployed as Assistants for Smart Grid Operations: A Benchmark Against NERC Standards
Apr 25, 2026Taha Hammadia, Lucas Rea, Ahmad Mohammad Saber +2LLM Jailbreak Attacks LLM Safety Evaluation
ECE Department, University of Toronto, Toronto, ON Canada · CIISE, Concordia University, Montréal, QC, Canada
Discovering Agentic Safety Specifications from 1-Bit Danger Signals
Apr 25, 2026Víctor GallegoAgentic Reasoning LLM Safety Evaluation
Komorebi AI
SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs
Apr 24, 2026Sihang Zhao, Kangrui Yu, Youliang Yuan +2Intelligent Tutoring Systems Generative AI in Education
Center for Data Science, New York University Shanghai · Courant Institute of Mathematical Sciences, New York University · School of Data Science, The Chinese University of Hong Kong, Shenzhen
Large Language Models Outperform Humans in Fraud Detection and Resistance to Motivated Investor Pressure
Apr 22, 2026Nattavudh PowdthaveeFinancial Fraud Detection LLM Decision-Making
Nanyang Technological University
Omission Constraints Decay While Commission Constraints Persist in Long-Context LLM Agents
Apr 22, 2026Yeran GamageAI Agent Monitoring LLM Security
University of South Florida · Independent AI Security Researcher
When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains
Apr 21, 2026Ishita Kakkar, Enze Zhang, Rheeya Uppaal +1AI Safety Evaluation LLM Safety Evaluation
Department of Computer Sciences, University of Wisconsin-Madison
Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safety
Apr 20, 2026Marcello Galisai, Susanna Cifani, Francesco Giarrusso +5LLM Safety Benchmarks Adversarial Attacks on LLMs
DEXAI – Icaro Lab · Sapienza University of Rome · Sant’Anna School of Advanced Studies
Using large language models for embodied planning introduces systematic safety risks
Apr 20, 2026Tao Zhang, Kaixian Qu, Zhibin Li +4Large Language Model-Based Robot Planning Robot Safety
ETH Zurich, Zurich, Switzerland. · University College London, London, United Kingdom. · Stanford University, Stanford, California, United States. +2
TPS-CalcBench: A Benchmark and Diagnostic Evaluation Framework for LLM Analytical Calculation Competence in Hypersonic Thermal Protection System Engineering
Apr 20, 2026Jinglai Zheng, Chuhan Qiao, Haiming HuangSynthetic Benchmark Generation LLM Safety Evaluation
School of Civil Engineering, Beijing Jiaotong University, Beijing 100044, China
MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models
Apr 20, 2026Suhyun Lee, Palakorn Achananuparp, Neemesh Yadav +2Mental Health Counseling LLM Evaluation
Department of Artificial Intelligence, Hanyang University, Seoul, Republic of Korea · School of Computing and Information Systems, Singapore Management University, Singapore
On Safety Risks in Experience-Driven Self-Evolving Agents
Apr 18, 2026Weixiang Zhao, Yichen Zhang, Yingshuo Wang +8LLM Safety Evaluation Over-Refusal
Harbin Institute of Technology · Singapore Management University · SERES
When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints
Apr 18, 2026Yuheng Chen, Zhiyu Wu, Bowen Cheng +2Adversarial Attacks on LLMs LLM Safety Alignment
Kagoshima University · Fudan University · China University of Petroleum-Beijing
HealthCraft: A Reinforcement Learning Safety Environment for Emergency Medicine
Apr 18, 2026Brandon DentHealthcare AI Agent Benchmarks
GOATnote Inc.
A Systematic Study of Training-Free Methods for Trustworthy Large Language Models
Apr 17, 2026Wai Man Si, Mingjie Li, Michael Backes +1LLM Alignment LLM Safety
CISPA Helmholtz Center for Information Security
Safety Training Modulates Harmful Misalignment Under On-Policy RL, But Direction Depends on Environment Design
Apr 14, 2026Leon Eshuijs, Shihan Wang, Antske FokkensReward Hacking RL for Language Models
Vrije Universiteit Amsterdam · Utrecht University
IatroBench: A Pre-Registered Benchmark of Clinical Omission in Language Models
Apr 9, 2026David GringrasLLM Safety Benchmarks Healthcare
Harvard T.H. Chan School of Public Health
Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations
Mar 31, 2026Yahan Li, Xinyi Jie, Wanjia Ruan +5Healthcare Clinical Reasoning
University of Southern California
Peer-Preservation in Frontier Models
Mar 30, 2026Yujin Potter, Nicholas Crispino, Vincent Siu +2AI Agent Safety LLM Safety Evaluation
University of California, Berkeley · University of California, Santa Cruz
SafeMath: Safe Solutions for Unsafe Math Word Problems
Mar 26, 2026Sagnik Basu, Subhrajit Mitra, Aman Juneja +3LLM Safety Alignment LLM Safety Evaluation
1Indian Institute of Technology Kharagpur · 2Cisco Systems · 3National University of Singapore
SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems
Mar 18, 2026Rima Hazra, Bikram Ghuku, Ilona Marchenko +5Intelligent Tutoring Systems AI Safety Evaluation
Eindhoven University of Technology, Netherlands (TU/e) · Indian Institute of Technology Kharagpur · Taras Shevchenko National University of Kyiv +3
Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services
Mar 11, 2026Fabrizio Dimino, Bhaskarjit Sarmah, Stefano PasqualiLanguage Model Safety Evaluation LLM Safety
1Domyn, New York, US · 2Domyn, New York, US · 3Domyn, New York, US
Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety
Mar 8, 2026David GringrasLLM Safety Benchmarks LLM Agent Evaluation
Harvard University
Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas
Jan 29, 2026Katherine Elkins, Jon ChunMoral Reasoning in Language Models LLM Auditing
Kenyon College Gambier, OH
Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions
Dec 26, 2025Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi +2Healthcare LLM Safety
Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. · Department of Anesthesiology and Intensive Care, Mashhad University of Medical Sciences, Mashhad, Iran. +2
Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data
Dec 25, 2025Yiwei Yan, Guanfeng LiuHealthcare Text Classification
School of Computing, Macquarie University, Australia
Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
Dec 21, 2025Zhang Wei, Hanxuan Chen, Peilu Hu +19Adversarial Robustness of Language Models LLM Safety Evaluation
Independent Researcher · Hunan University · Shenzhen Kaihong Digital Industry Development Co., Ltd. +12
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
Oct 17, 2025Hanbin Hong, Shuang Wu, Shuya Feng +6LLM Security LLM Jailbreak Attacks
University of Connecticut · University of Alabama at Birmingham · Independent
SafetyFlow: An Agent-Flow System for Automated LLM Safety Benchmarking
Aug 21, 2025Xiangyang Zhu, Yuan Tian, Chunyi Li +3LLM Safety Benchmarks Synthetic Benchmark Generation
Shanghai AI Lab · East China Normal University
HarmReduction: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs
Jul 29, 2025Kaixuan Wang, Chenxin Diao, Jason T. Jacques +2Healthcare Question Answering
University of Oxford, United Kingdom · University of Edinburgh, United Kingdom · University of St Andrews, United Kingdom +1
GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods
Feb 24, 2025Ruixuan Huang, Xunguang Wang, Zongjie Li +2Language Model Safety Evaluation LLM Jailbreak Attacks
The Hong Kong University of Science and Technology, Hong Kong SAR, China
CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models
Jan 24, 2025Guangzhi Sun, Xiao Zhan, Shutong Feng +2LLM Safety Benchmarks Language Model Safety Evaluation
Trinity College, University of Cambridge, Cambridge, United Kingdom · Department of Informatics, King’s College London, United Kingdom · Institut für Informatik, Heinrich-Heine-Universität Düsseldorf, Germany +1
Who Judges the Judges? A Chinese Safety QA Benchmark for Evaluating LLM Responses and Safety Judges
Date pendingRui Yang, Shuang Huang, Junhua Liu +7LLM Safety Benchmarks LLM-as-a-Judge
Anhui SparkShield Intelligent Technology · iFLYTEK · Peking University +1

May 14, 2026, cs.CYJ/K move · Enter open · S save

Muse Spark Safety & Preparedness Report

Cristina Menghini, Peter Ney, Hamza Kwisaba, Zifan+116

Sail

Cybersecurity AI Safety Evaluation LLM Safety Evaluation AI Safety

Open paper AskPDF arXiv

Muse Spark is the latest large language model developed by Meta. In this report, we first present evaluations for catastrophic risk domains under Meta's Advanced AI Scaling Framework, along with the evidence that informed our launch decision. We then discuss additional considerations, such as Muse Spark's broader content safety and behavioral profile, that are relevant to overall safety but fall outside the catastrophic risk domains governed by the Framework. Our preparedness results covering Chemical and Biological, Cybersecurity, and Loss of Control risks assess Muse Spark's deployment within Meta AI as presenting acceptable levels of residual risks under our Advanced AI Scaling Framework. We conducted a broad set of evaluations targeting dual-use and high-risk capabilities across these catastrophic risk domains. Those evaluations identified elevated risks prior to mitigations, with Chemical and Biological capabilities assessed as likely reaching the "high risk" category under the Advanced AI Scaling Framework before safeguards were applied. We have implemented a multi-layered set of mitigations that address the identified risks, and Muse Spark demonstrates state-of-the-art refusal across a range of benchmarks related to hazardous workflows in chemistry and biology. We therefore release Muse Spark as the underlying model of Meta AI.

Momentum

Related topics

Latest papers 177