Language Model Safety Evaluation

Follow this topic Ask about this topic

Momentum

45 papers in the last four weeks, down 15% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 396

All topics

Harder to Defend: Towards Chinese Toxicity Attacks via Implicit Enhancement and Obfuscation Rewriting
May 21, 2026Jingyi Kang, Junyu Lu, Bo Xu +4Language Model Safety Evaluation Toxicity Detection
1Dalian University of Technology · 2The University of Tokyo · 3Singapore University of Technology and Design
On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation
May 20, 2026Andy Han, Kristina Fujimoto, Avidan Shah +5Language Model Safety Evaluation LLM Safety Alignment
1New York University
Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs
May 20, 2026Dylan Feng, Pragya Srivastava, Anca Dragan +1Language Model Safety Evaluation LLM Guardrails
University of California, Berkeley, USA · 3Google DeepMind, India.
RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts
May 20, 2026Lukas Weidener, Marko Brkić, Mihailo Jovanović +2LLM Safety Benchmarks Language Model Safety Evaluation
Applied Scientific Intelligence, Inc.
Do No Harm? Hallucination and Actor-Level Abuse in Web-Deployed Medical Large Language Models
May 20, 2026Sunday Oyinlola Ogundoyin, Muhammad Ikram, Rahat MasoodLanguage Model Safety Evaluation Healthcare
Macquarie University, Sydney, Australia · The University of New South Wales, Sydney, Australia
Measuring Safety Alignment Effects in Autonomous Security Agents
May 19, 2026Isaac David, Arthur GervaisLanguage Model Safety Evaluation LLM Safety Alignment
University College London
DECOR: Auditing LLM Deception via Information Manipulation Theory
May 19, 2026Linyue Cai, Samuel Yeh, Jwala Dhamala +2Language Model Safety Evaluation LLM Auditing
Department of Computer Sciences, University of Wisconsin-Madison · Amazon
Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamics
May 18, 2026Maciej Chrabąszcz, Aleksander Szymczyk, Marcin Sendera +2Language Model Safety Evaluation LLM Interpretability
NASK - National Research Institute, Poland · Faculty of Electronics and Information Technology, Warsaw University of Technology, Poland · Faculty of Mathematics and Computer Science, Jagiellonian University, Poland +3
Multilingual jailbreaking of LLMs using low-resource languages
May 18, 2026Dylan Marx, Marcel DunaiskiLanguage Model Safety Evaluation LLM Jailbreak Attacks
Computer Science Division, Mathematical Sciences Department Stellenbosch University Stellenbosch, South Africa
Geometry-Lite: Interpretable Safety Probing via Layer-Wise Margin Geometry
May 18, 2026Woo Seob Sim, Yu Rang ParkLanguage Model Safety Evaluation LLM Safety
Department Biomedical Systems Informatics Yonsei University College of Medicine, Seoul, Republic of Korea · Yonsei University
Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications
May 17, 2026Isaac David, Arthur GervaisLanguage Model Safety Evaluation Cybersecurity
University College London
LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
May 17, 2026Nanxi Li, Zhengyue Zhao, Chaowei XiaoLanguage Model Safety Evaluation LLM Guardrails
Johns Hopkins University
Why Do Safety Guardrails Degrade Across Languages?
May 16, 2026Max Zhang, Ameen Patel, Sang T. Truong +1Multilingual Language Model Evaluation Language Model Safety Evaluation
Stanford University
New Wide-Net-Casting Jailbreak Attacks Risk Large Models
May 16, 2026Qiuchi Xiang, Haoxuan Qu, Hossein Rahmani +1Language Model Safety Evaluation Jailbreak Attacks
1Lancaster University.
The Point of No Return: Counterfactual Localization of Deceptive Commitment in Language-Model Reasoning
May 16, 2026Scott Merrill, Shashank SrivastavaLanguage Model Safety Evaluation LLM Interpretability
University of North Carolina at Chapel Hill
Training ML Models with Predictable Failures
May 14, 2026Will Schwarzer, Scott NiekumLanguage Model Safety Evaluation LLM Fine-Tuning
University of Massachusetts Amherst
A Few Steps Further: Why Defenses Against Malicious Finetuning Erode Under Continued Training
May 14, 2026Itay Zloczower, Eyal Lenga, Gilad Gressel +1Language Model Safety Evaluation Adversarial Attacks on LLMs
Ben-Gurion University of the Negev · Amrita Vishwa Vidyapeetham, Amritapuri
NodeSynth: Socially Aligned Synthetic Data for AI Evaluation
May 14, 2026Qazi Mamunur Rashid, Xuan Yang, Zhengzhe Yang +5Language Model Generation Evaluation Language Model Safety Evaluation
Google Research · Google · Google Deepmind
ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety
May 13, 2026Michael S. Lee, Yash Maurya, Drew Rein +13Multilingual Language Model Evaluation Language Model Safety Evaluation
1Scale AI · 2Korea AI Safety Institute
Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Study
May 13, 2026Mokshit Surana, Archit Rathod, Akshaj SatishkumarLanguage Model Safety Evaluation Toxicity Detection
University of Illinois Chicago
AERIC: Anticipatory Hidden-State Monitoring for Implicit Harmful Dialogue
May 13, 2026Jihyung Park, Saleh Afroogh, Junfeng JiaoLanguage Model Safety Evaluation LLM Guardrails
The University of Texas at Austin
Quantifying LLM Safety Degradation Under Repeated Attacks Using Survival Analysis
May 13, 2026Zvi TopolLanguage Model Safety Evaluation LLM Jailbreak Attacks
MuyVentive, LLC
Before the Last Token: Diagnosing Final-Token Safety Probe Failures
May 12, 2026Shravan DodaLanguage Model Safety Evaluation Safety Filtering
Kipo AI
DisaBench: A Participatory Evaluation Framework for Disability Harms in Language Models
May 12, 2026Eugenia Kim, Ioana Tanase, Christina MallonLLM Safety Benchmarks LLM Evaluation
Microsoft
Classifier Context Rot: Monitor Performance Degrades with Context Length
May 12, 2026Sam Martin, Fabien RogerLanguage Model Safety Evaluation Long-Context Language Model Evaluation
Anthropic Fellows Program · Anthropic
Confidently Deceptive: On the Relationship Between Confidence and Deception in LLMs
May 12, 2026Ali Asad, Stephen Obadinma, Anshul Pattoo +2Language Model Safety Evaluation Confidence Estimation in Language Models
Department of Electrical and Computer Engineering & Ingenuity Labs Research Institute Queen’s University, Kingston, Canada
To Whom Do Language Models Align? Measuring Principal Hierarchies Under High-Stakes Competing Demands
May 12, 2026Fangyi Yu, Nabeel Seedat, Jonathan Richard Schwarz +1LLM Alignment Language Model Safety Evaluation
Thomson Reuters Foundational Research, 5 Canada Square, London · Imperial College London · University of Oxford
Safety-Oriented Evaluation of Language Understanding Systems for Air Traffic Control
May 12, 2026Yujing Chang, Yash Guleria, Duc-Thinh Pham +4LLM Grounding Air Traffic Management
1ATMRI, Nanyang Technological University (NTU), Singapore · School of Management, Indian Institute of Technology Mandi, India · Centre of AI Research, VinUniversity, Vietnam
Beyond Red-Teaming: Formal Guarantees of LLM Guardrail Classifiers
May 11, 2026Nikita Kezins, Urbas Ekka, Pascal Berrang +1Neural Network Verification Language Model Safety Evaluation
Delft University of Technology · University of Birmingham & Zeroth Research
RUBEN: Rule-Based Explanations for Retrieval-Augmented LLM Systems
May 11, 2026Joel Rorseth, Parke Godfrey, Lukasz Golab +2Retrieval-Augmented Generation Language Model Safety Evaluation
University of Waterloo · York University · AT&T Chief Data Office

May 21, 2026, cs.CLJ/K move · Enter open · S save

Harder to Defend: Towards Chinese Toxicity Attacks via Implicit Enhancement and Obfuscation Rewriting

Jingyi Kang, Junyu Lu, Bo Xu, Hongbo Wang+3

1Dalian University of Technology · 2The University of Tokyo · 3Singapore University of Technology and Design

Language Model Safety Evaluation Toxicity Detection Adversarial Robustness of Language Models Adversarial Text Attacks

Open paper AskPDF arXiv

Large language models (LLMs) require robust toxicity evaluation beyond explicit wording. This setting remains underexplored in Chinese, where toxicity may combine semantic indirectness with surface obfuscation. We introduce Chinese Implicit Toxicity Attack (CITA), a controlled red-team evaluation and defense-data generation framework, not a deployable evasion tool. CITA uses three stages: (i) Harmful Intent Learning, (ii) Implicit Toxicity Enhancement, and (iii) Obfuscation Variant Rewriting, to preserve harmful intent, increase implicitness, and add controlled surface variants. On CITA-generated evaluation samples, the seven tested detectors exhibit substantial missed-detection risks, reaching an average ASR of 69.48%; human evaluation further confirms preserved harmfulness and increased implicitness/evasiveness. As a downstream defense application, we fine-tune a Chinese Implicit Toxicity Defense model (CITD) with CITA-generated red-team data, showing that such data can improve robustness through additional training.

Momentum

Related topics

Latest papers 396