Language Model Safety Evaluation

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 58% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 347

All topics

TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages
May 31, 2026Victor Akinode, Senyu Li, Wassim Hamidouche +3LLM Safety Benchmarks Multilingual Language Model Evaluation
Mila - Quebec AI Institute · McGill University · Microsoft AI for Good Research Lab +1
MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks an dClassifier-Based Defenses for Medical AI Safety
May 30, 2026Anushka Sheoran, Yiduo HaoLanguage Model Safety Evaluation Adversarial Attacks on LLMs
University of Pennsylvania
Quantifying the Salience of Geo-Cultural Values for Pluralistic Safety Alignment
May 29, 2026Arkadiy Saakyan, Charvi Rastogi, Lora AroyoLanguage Model Safety Evaluation Cultural Alignment
Columbia University, New York, NY, USA. · Google DeepMind, New York, NY, USA.
LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories
May 29, 2026Krishnapriya Vishnubhotla, Sowmya Vajjala, Akriti Vij +1Multilingual Language Model Evaluation LLM-as-a-Judge
National Research Council, Canada · IMDA, Singapore
RealityTest: How People Probe AI Identity and Whether Models Disclose It
May 29, 2026Anna Gausen, Sarenne Wallbridge, Bessie O'Dell +2LLM Evaluation Language Model Safety Evaluation
AI Security Institute, London, UK
EUDAIMONIA: Evaluating Undesirable Dynamics in AI
May 28, 2026Jun Rui Huang, Wang Bill Zhu, Ziyi Liu +3LLM Alignment Language Model Safety Evaluation
University of Southern California
Evaluating using Mock Tool Calls to Quarantine Untrusted Prompt Inputs
May 28, 2026David Gros, Adam GleaveLanguage Model Safety Evaluation Adversarial Attacks on LLMs
Configurable Reward Model for Balanced Safety Alignment
May 28, 2026Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj +5Reward Modeling Language Model Safety Evaluation
Johns Hopkins University · Meta · Meta Superintelligence Labs
BioRefusalAudit: Auditing Biosecurity Refusal Depth Using General and Domain-Fine-Tuned Sparse Autoencoders
May 28, 2026Caleb DeLeeuwLanguage Model Safety Evaluation Sparse Autoencoders
Independent researcher
Beyond English and Evasion: A Human-Annotated Multi-Domain Benchmark for High-Stakes LLM Safety Evaluation in Chinese
May 28, 2026Wajdi Zaghouani, Kholoud K. Aldous, Yicheng GaoMultilingual Language Model Evaluation LLM Evaluation
Northwestern University in Qatar
Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content
May 28, 2026Ihor Stepanov, Aleksandr SmechovLanguage Model Safety Evaluation LLM Guardrails
1Knowledgator, Kyiv, Ukraine · 2Wordcab, New York, USA
FinGuard: Detecting Financial Regulatory Non-Compliance in LLM Interactions
May 28, 2026Huaixia Dou, Jie Zhu, Minghao Wu +5Language Model Safety Evaluation LLM Guardrails
Qwen DianJin Team, Alibaba Cloud Computing · Tongyi Lab, Alibaba Group · School of Computer Science and Technology, Soochow University
When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception
May 28, 2026Vahideh ZolfaghariLinear Probing Language Model Safety Evaluation
Algoverse AI Research, Algoverse AI Research Mentorship Program, USA. · Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
Position: Retire the "Positive Backdoor" Label -- Secret Alignment Requires Strict and Systematic Evaluation
May 27, 2026Jianwei Li, Jung-Eun KimLLM Backdoor Attacks Language Model Safety Evaluation
Department of Computer Science, North Carolina State University, Raleigh, USA.
Models That Know How Evaluations Are Designed Score Safer
May 27, 2026Katharina Deckenbach, Haritz Puerto, Jonas Geiping +1Language Model Safety Evaluation AI Safety Evaluation
ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems, Tübingen AI Center
KSAFE-MM: A Multimodal Safety Benchmark via Localized Contextualization for Korean Cultural Risks
May 27, 2026Yongwoo Kim, Sojung An, Yunjin Park +8Adversarial Attacks on VLMs Language Model Safety Evaluation
Korea University · KT Corporation
Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models
May 27, 2026Himanshu Beniwal, Mayank SinghLanguage Model Safety Evaluation LLM Interpretability
Indian Institute of Technology Gandhinagar
Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
May 27, 2026Sachin KumarLLM Evaluation Language Model Safety Evaluation
LexisNexis, USA
The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
May 27, 2026Eric Onyame, Runtao Zhou, Kowshik Thopalli +2CoT Faithfulness Multilingual Language Model Evaluation
University of Virginia · Lawrence Livermore National Laboratory
When Context Flips, Safety Breaks: Diagnosing Brittle Safety in Aligned Language Models
May 27, 2026Dasol Choi, Alex KwonLLM Safety Benchmarks Counterfactual Evaluation
AIM Intelligence · Independent Researcher
A Paired Testing Protocol for Batch-Conditioned Refusal Robustness in LLM Serving
May 26, 2026Sahil KadadekarLLM Inference Language Model Safety Evaluation
Independent Researcher.
Efficient Safety Benchmarking via Item Response Theory
May 26, 2026Fabio Spagliardi, Mírian Silva, Ayan Datta +3LLM Safety Benchmarks Language Model Safety Evaluation
Supervised Program for Alignment Research (SPAR) · Yale University · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs
May 26, 2026Zhe Yu, Wenpeng Xing, Chen Ye +4Language Model Safety Evaluation Knowledge Conflicts in Language Models
Binjiang Institute of Zhejiang University · Zhejiang University · Hangzhou Dianzi University +1
Prompt Injection Detection is Regime-Dependent: A Deployment-Aware Evaluation with Interpretable Structural Signals
May 26, 2026Akindoyin Akinrele, Shreyank N GowdaLanguage Model Safety Evaluation Prompt Injection Attacks on LLMs
School of Computer Science, University of Nottingham, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, United Kingdom
AlbanianLLMSafety: A Safety Evaluation Dataset for Large Language Models in Albanian
May 26, 2026Wajdi Zaghouani, Kholoud K. Aldous, Isra FejzullajLLM Safety Benchmarks Language Model Safety Evaluation
Northwestern University in Qatar
KZ-SafetyPrompts: A Kazakh Safety Evaluation Prompt Dataset for Large Language Models
May 26, 2026Wajdi Zaghouani, Shimaa Amer Ibrahim, Aruzhan Muratbek +2LLM Safety Benchmarks Language Model Safety Evaluation
Northwestern University in Qatar
$D^2$ -Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation Signals
May 25, 2026Aoxi Liu, Yupeng Chen, James Oldfield +5Language Model Safety Evaluation LLM Safety
Torr Vision Group, University of Oxford · The Chinese University of Hong Kong, Shenzhen
StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMs
May 25, 2026Yang Luo, Xinran Liu, Tiantian Ji +3Adversarial Attacks on VLMs Language Model Safety Evaluation
Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications · Institute of Computing Technology, Chinese Academy of Sciences
The Age of Curiosity Meets the Age of AI: Benchmarking Child Safety in Large Language Models
May 25, 2026Samee Arif, Angana Borah, Rada MihalceaLLM Safety Benchmarks Language Model Safety Evaluation
University of Michigan
SomaliBench Eval: Measuring English-to-Somali Refusal Gaps in Open-Weight Language Models
May 25, 2026Khalid Yusuf DahirLLM Safety Benchmarks Multilingual Language Model Evaluation
Independent researcher

May 31, 2026, cs.CLJ/K move · Enter open · S save

TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages

Victor Akinode, Senyu Li, Wassim Hamidouche, Waqas Zamir+2

Mila - Quebec AI Institute · McGill University · Microsoft AI for Good Research Lab+1

LLM Safety Benchmarks Multilingual Language Model Evaluation Language Model Safety Evaluation LLM Jailbreak Attacks Low-Resource Language Processing Refusal Behavior in Language Models

Open paper AskPDF arXiv

Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We introduce TUKABENCH, a jailbreak benchmark for seven African languages that extends JailbreakBench (JBB) beyond direct translation through four settings: human translation of JBB prompts, English adaptation to African contexts followed by human translation, human-curated prompts validated through interactions with GPT-5.2, and code-switched prompts combining English and African languages, isolating the effect of language, cultural grounding, and prompt evasiveness on model safety. Across closed and open models, prompting in African languages reduces refusal relative to English, with culturally adapted prompts leading to least refusal. The evaluation also surfaces two structural limitations: model comprehension failures and reduced LLM-as-a-judge reliability in LRLs. To capture the first, we introduce Deflection alongside Refused and Jailbroken; to assess the second, we validate outputs with human annotations, showing that judge-human agreement drops in lower-resource languages and less commonly supported scripts.

Momentum

Related topics

Latest papers 347