Language Model Safety Evaluation

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 58% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 347

All topics

SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing
Jun 29, 2026Jiacheng Zhang, Haoyu He, Sen Zhang +5LLM Safety Benchmarks Language Model Safety Evaluation
1ByteDance · 2The University of Melbourne
CAREBench: A Child-Safety Risk Benchmark for Language Models
Jun 29, 2026Kaavya Krishna-Kumar, Elaine Lau, Vaughn Robinson +6LLM Safety Benchmarks Language Model Safety Evaluation
Handshake AI · University of California, Los Angeles · McGill University
Mechanistically Eliciting Latent Behaviors in Language Models
Jun 28, 2026Andrew Mack, Nina Panickssery, Alexander Matt TurnerLLM Alignment Language Model Safety Evaluation
†Principles of Intelligence · ‡Anthropic · §Independent
The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning
Jun 27, 2026Will Hawkins, Kaivalya Rawal, Jonathan Rystrøm +8LLM Safety Benchmarks Multilingual Language Model Evaluation
Oxford Internet Institute, University of Oxford · Hasso Plattner Institute, University of Potsdam · Chinese University of Hong Kong +2
Agent Safety Is Action Alignment
Jun 27, 2026Shawn Li, Yue ZhaoLanguage Model Safety Evaluation AI Alignment
University of Southern California · Los Angeles, California, USA
RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots
Jun 26, 2026Nima DorzhievLanguage Model Safety Evaluation Prompt Injection Defense
Pennsylvania State University, State College, USA
ToxiREX: A Dataset on Toxic REasoning in ConteXt
Jun 26, 2026Stefan F. Schouten, Ilia Markov, Piek VossenLanguage Model Safety Evaluation Toxicity Detection
Vrije Universiteit Amsterdam, the Netherlands
The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report
Jun 25, 2026Kwan Soo Shin, In Seok Kang, Yunkyung Min +2Language Model Safety Evaluation AI Safety Evaluation
PolymathMinds Lab, Seoul, Republic of Korea · Department of Chemical Engineering, Pohang University of Science and Technology (POSTECH), Pohang, Republic of Korea · aSSIST University (Seoul School of Integrated Sciences & Technologies), Seoul, Republic of Korea +3
Manipulation Is Task-Dependent: A Multi-Axis, Multi-Environment Evaluation of Frontier LLMs
Jun 24, 2026Adeeb Zaman, Erik Nordby, Fred HeidingLLM Evaluation Language Model Safety Evaluation
Cambridge Boston Alignment Initiative (CBAI), Cambridge, US · The AI and Cybersecurity Institute (TAICI), Cambridge, US
Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation
Jun 24, 2026Han Jeon, Shiv Medler, Joseph Voyles +1LLM Safety Benchmarks LLM-as-a-Judge
PricewaterhouseCoopers, U.S.A.
How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring
Jun 24, 2026Yang GaoLLM Evaluation LLM-as-a-Judge
Veyon Solutions
Long-Term Simulation Exposes Cognitive-Developmental Risks in AI Companions
Jun 24, 2026Kaicheng Shen, Lingyu Li, Wen Wu +3Language Model Safety Evaluation Long-Horizon Agent Evaluation
Shanghai Institute of Artificial Intelligence for Education, East China Normal University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · School of Computer Science and Technology, East China Normal University, Shanghai, China
A Survey of Toxicity Detection and Mitigation Strategies for Multilingual Language Models
Jun 24, 2026Soham Dan, Himanshu Beniwal, Thomas HartvigsenMultilingual Language Models LLM Alignment
Scale AI · Indian Institute of Technology Gandhinagar · University of Virginia
Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion
Jun 23, 2026Sahil KadadekarLanguage Model Safety Evaluation Speculative Decoding
Independent Researcher
Can LLMs Reliably Self-Report Adversarial Prefills, and How?
Jun 22, 2026Quang Minh Nguyen, Uzair Ahmed, Taegyoon KimLanguage Model Safety Evaluation Adversarial Attacks on LLMs
KAIST
Evaluation Awareness Is Not One Capability: Evidence from Open Language Models
Jun 22, 2026Nilesh Nayan, Aishwarya Sampath Kumar, Rishiraj Girmal +5Language Model Safety Evaluation LLM Interpretability
University of Massachusetts, Amherst · 2Microsoft Corp.
Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts
Jun 22, 2026Arthur Wuhrmann, Gaetan Stein, Daniel Brunner +1Multilingual Language Model Evaluation Legal NLP
Swiss Federal Supreme Court · HES-SO Valais-Wallis
SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning
Jun 22, 2026SingGuard TeamLanguage Model Safety Evaluation LLM Guardrails
1AI Security Lab, Ant Group
IndicGuard: A Multilingual Safety Guard Model and Dataset for Indic Languages
Jun 22, 2026Parth Bramhecha, Smit Deshmukh, Sairaj Bodhale +2Multilingual Language Models Language Model Safety Evaluation
L3Cube-Labs, Pune
FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming
Jun 18, 2026Chaeyun Kim, Daeyoung Park, Junghwan Kim +4LLM Safety Benchmarks Financial Services
DATUMO INC., Republic of Korea · Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · Financial Security Institute (FSI), Republic of Korea
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Jun 17, 2026Naihao Deng, Yiming Feng, Chimaobi Okite +4LLM Grounding LLM Alignment
University of Michigan · University of Cambridge · University of Aberdeen
A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models
Jun 16, 2026Nicola FrancoLanguage Model Safety Evaluation Neural Network Robustness
Head of AI Security Lab The Italian Institute of Artificial Intelligence (AI4I) Corso Castelfidardo 22, 10129 Turin, Italy
Tail-Shape Estimation in LLM Evaluation Is Fragile: A Protocol for Diagnosing False Positives
Jun 15, 2026Luca ZhouLLM Evaluation Language Model Safety Evaluation
Sapienza University of Rome
AuAu: A Benchmark for Auditing Authoritarian Alignment in Large Language Models
Jun 15, 2026Andreas Einwiller, Max Klabunde, Florian LemmerichLanguage Model Safety Evaluation LLM Auditing
University of Passau, Germany
Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness
Jun 14, 2026Evan DuanLanguage Model Safety Evaluation Representation Drift
University of Michigan
On Defining Erasure Harms for NLP
Jun 14, 2026Yu Lu Liu, Arnav Goel, Jackie Chi Kit Cheung +3Language Model Safety Evaluation
1Johns Hopkins University · 2Carnegie Mellon University · 3Mila – Québec Artificial Intelligence Institute +2
Constitutional Value Potentials: reading and steering internal priority margins in language models
Jun 13, 2026Tong Che, Rui WuLanguage Model Safety Evaluation LLM Safety
Rutgers University
CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment
Jun 13, 2026Wenbo Yu, Bohua Wang, Hao Fang +10LLM Safety Benchmarks Language Model Safety Evaluation
Tsinghua University · Beijing Normal University · South China University of Technology +2
Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds
Jun 13, 2026Ömer Veysel Çağatan, Xuandong ZhaoReward Hacking Language Model Safety Evaluation
KUIS AI Center, Koc¸ University · Istanbul, Türkiye · University of California, Berkeley +1
I'm Sorry Driver, I'm Afraid I Can't Do That: Appraising the Safety of LLMs within Automotive Contexts
Jun 12, 2026Shaun Feakins, Ibrahim Habli, Kim Littler +1Language Model-Based Control Language Model Safety Evaluation
UKRI AI Centre for Doctoral Training in Safe Artificial Intelligence Systems (SAINTS), University of York, York, UK · Jaguar Land Rover, Coventry, UK

Jun 29, 2026, cs.AIJ/K move · Enter open · S save

SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing

Jiacheng Zhang, Haoyu He, Sen Zhang, Shen Wang+4

1ByteDance · 2The University of Melbourne

LLM Safety Benchmarks Language Model Safety Evaluation LLM Guardrails

Open paper AskPDF arXiv

In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on predefined risk taxonomies. In this work, we study this setting under the paradigm of in-context policy guardrailing, where guardrails predict safety violations based on policy specifications provided in context. To systematically evaluate this capability, we introduce SafePyramid, a safety benchmark comprising 1,000 multi-turn conversations across 10 domains and 3,000 corresponding application-specific policies, which together contain 61,699 distinct natural-language rules. SafePyramid organizes the evaluation into three difficulty levels: L0 evaluates individual-rule understanding, L1 evaluates reasoning over rule dependencies, and L2 evaluates adaptation of full novel policy frameworks defined in context. To ensure benchmark quality, we employ a rigorous multi-stage pipeline to construct and validate the benchmark. Using SafePyramid, we evaluate 10 frontier LLMs and 5 policy-configurable guardrails and find that in-context policy guardrailing remains highly challenging: even the best-performing model, GPT-5.5, exactly identifies the full set of violated rules in only 54.0%, 35.3%, and 12.9% cases on L0, L1, and L2, respectively. These results highlight the limitations of current guardrails and call for stronger in-context policy guardrails that can reliably execute policies, resolve rule dependencies, and adapt to novel policy frameworks.

Momentum

Related topics

Latest papers 347