Adversarial Robustness of Language Models

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 123

All topics

SoK: Robustness in Large Language Models against Jailbreak Attacks
May 6, 2026Feiyue Xu, Hongsheng Hu, Chaoxiang He +9Language Model Safety Evaluation LLM Security
Shanghai Jiao Tong University, China · Microsoft Corporation
Are LLMs Ready for Conflict Monitoring? Empirical Evidence from West Africa
May 5, 2026Hoffmann Muki, Olukunle OwolabiText Classification Adversarial Robustness of Language Models
University of Maryland, College Park, USA · Independent Researcher, Seattle, USA
Exposing LLM Safety Gaps Through Mathematical Encoding:New Attacks and Systematic Analysis
May 5, 2026Haoyu Zhang, Mohammad Zandsalimy, Shanu SushmitaLanguage Model Safety Evaluation Adversarial Attacks on LLMs
† Northeastern University · ‡ Ansys
RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs
May 3, 2026Sadia Asif, Mohammad Mohammadi AmiriFine-Tuning LLM Fine-Tuning
Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA
The Power of Order: Fooling LLMs with Adversarial Table Permutations
May 1, 2026Xinshuai Dong, Haifeng Chen, Xuyuan Liu +5Table QA Adversarial Attacks on LLMs
CMU & NEC Labs · NEC Labs · Dartmouth College & NEC Labs +1
Dynamic Adversarial Fine-Tuning Reorganizes Refusal Geometry
Apr 29, 2026Wenhao Lan, Shan Li, Xinhua Lai +4Adversarial Training Language Model Safety Evaluation
University of Chinese Academy of Sciences, Beijing, China · Inner Mongolia University of Technology, Inner Mongolia, China · Tsinghua University, Beijing, China +1
SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Prompts
Apr 29, 2026Yuan Xin, Yixuan Weng, Minjun Zhu +5Adversarial Training Adversarial Prompt Generation
1CISPA · 2Westlake University · 3Southern University of Science and Technology +1
Structured Security Auditing and Robustness Enhancement for Untrusted Agent Skills
Apr 28, 2026Lijia Lv, Xuehai Tang, Jie Wen +2AI Agent Auditing AI Agent Security
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences
SafeRedirect: Defeating Internal Safety Collapse via Task-Completion Redirection in Frontier LLMs
Apr 22, 2026Chao Pan, Yu Wu, Xin YaoLLM Guardrails LLM Safety Alignment
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China · The Hong Kong Polytechnic University, Hong Kong, China · George Washington University +1
Towards Understanding the Robustness of Sparse Autoencoders
Apr 20, 2026Ahson Saiyed, Sabrina Sadiekh, Chirag AgarwalAdversarial Attacks on LLMs Jailbreak Robustness
University of Virginia · Independent Researcher
Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks
Apr 20, 2026Jin Zhao, Marta Knežević, Tanja KäserLLM Evaluation Intelligent Tutoring Systems
†Work done during exchange at EPFL.
Do LLMs Hold Their Values? MANTA: A Multi-Turn Adversarial Benchmark for Animal Welfare Reasoning
Apr 18, 2026Isabella Luong, Joyee Chen, Arturs Kanepajs +5LLM Evaluation Moral Reasoning in Language Models
SPAR · Compassion Aligned Machine Learning · NUS +2
The Cognitive Penalty: Ablating System 1 and System 2 Reasoning in Edge-Native SLMs for Decentralized Consensus
Apr 18, 2026Syed Muhammad Aqdas RizviSmall Language Models Adversarial Robustness of Language Models
Independent Researcher Karachi, Pakistan
SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models
Apr 17, 2026Noor Islam S. Mohammad, Uluğ BayazıtLLM Hallucination Mitigation Privacy-Preserving Language Models
Istanbul Technical University
Stochasticity in Tokenisation Improves Robustness
Apr 17, 2026Sophie Steger, Rui Li, Sofiane Ennadir +4Adversarial Attacks on LLMs Neural Network Robustness
Institute of Signal Processing and Speech Communication, Graz University of Technology, Graz, Austria · ELLIS Institute Finland & Aalto University, Espoo, Finland · King AI Labs, Microsoft Gaming +2
On the Robustness of LLM-Based Dense Retrievers: A Systematic Analysis of Generalizability and Stability
Apr 17, 2026Yongkang Li, Panagiotis Eustratiadis, Yixing Fan +1LLM Evaluation Retrieval Robustness
University of Amsterdam, The Netherlands · Chinese Academy of Sciences, China
Segment-Level Coherence for Robust Harmful Intent Probing in LLMs
Apr 16, 2026Xuanli He, Bilgehan Sel, Faizan Ali +3LLM Safety LLM Jailbreak Attacks
University College London · Virginia Tech · Anthropic
Routing-Aware Safety Alignment for Mixture-of-Experts Models
Feb 4, 2026Jiacheng Liang, Yuhui Wang, Tanqiu Jiang +1Mixture-of-Experts Language Models LLM Alignment
Department of Computer Science Stony Brook University
SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care
Jan 23, 2026Dongshen Peng, Yi Wang, Austin Schoeffler +5LLM Sycophancy Language Model Safety Evaluation
UNC Chapel Hill · University of Waterloo · Stanford University
AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains
Jan 21, 2026Adam Szelestey, Sofie van Engelen, Tianhao Huang +3Factual Consistency Evaluation LLM Reliability
Eindhoven University of Technology Eindhoven, Netherlands
OI-Bench: An Option Injection Benchmark for Evaluating LLM Susceptibility to Directive Interference
Jan 19, 2026Yow-Fu Liou, Yu-Chien Tang, Yu-Hsiang Liu +1LLM Evaluation Adversarial Robustness of Language Models
Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan
Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions
Dec 26, 2025Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi +2Healthcare LLM Safety
Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. · Department of Anesthesiology and Intensive Care, Mashhad University of Medical Sciences, Mashhad, Iran. +2
Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
Dec 21, 2025Zhang Wei, Hanxuan Chen, Peilu Hu +19Adversarial Robustness of Language Models LLM Safety Evaluation
Independent Researcher · Hunan University · Shenzhen Kaihong Digital Industry Development Co., Ltd. +12
CluCERT: Certifying LLM Robustness via Clustering-Guided Denoising Smoothing
Dec 1, 2025Zixia Wang, Gaojie Jin, Jia Hu +1Randomized Smoothing Adversarial Robustness of Language Models
University of Exeter
destroR: A Benchmark and Adversarial-Training Defense for Bangla Transfer Models under Meaning-Preserving Attacks
Nov 13, 2025Saadat Rafid Ahmed, Rubayet Shareen, Radoan Sharkar +3Adversarial Training Sentiment Analysis
Dept. of Computer Science and Engineering, BRAC University
Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content
Sep 16, 2025Shaz Furniturewala, Arkaitz ZubiagaMechanistic Interpretability Toxicity Detection
Center for Data Science, New York University · Queen Mary University of London
Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming
Jul 30, 2025Jiazhen Pan, Bailiang Jian, Paul Hager +20LLM Safety Benchmarks Language Model Safety Evaluation
Cherise · School of Computation, Information and Technology, Technical University of Munich (TUM), Munich, Germany · Department of Engineering Science, University of Oxford, Oxford, United Kingdom +9
Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents
Jul 3, 2025Sizhe Chen, Arman Zharmagambetov, David Wagner +1LLM Agent Security AI Agent Security Benchmarks
FAIR at Meta · UC Berkeley
Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models
Apr 25, 2025Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín +1LLM Fine-Tuning RL for Language Model Reasoning
Department of Computer Systems, Technical University of Madrid, Madrid, Spain · United Arab Emirates University, Al Ain, United Arab Emirates
Unified Enhancement of the Generalization and Robustness of Language Models via Bi-Stage Optimization
Mar 19, 2025Yudao Sun, Juan Yin, Juan Zhao +3Adversarial Training Neural Network Generalization
Department of New Networks, Peng Cheng Laboratory, Shenzhen 518055, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore 117576 · School of Data Science and Artificial Intelligence, Chang’an University, Shaanxi, 710064, China

May 6, 2026, cs.CRJ/K move · Enter open · S save

SoK: Robustness in Large Language Models against Jailbreak Attacks

Feiyue Xu, Hongsheng Hu, Chaoxiang He, Sheng Hang+8

Shanghai Jiao Tong University, China · Microsoft Corporation

Language Model Safety Evaluation LLM Security LLM Jailbreak Attacks Adversarial Robustness of Language Models Jailbreak Defense

Open paper AskPDF arXiv

Large Language Models (LLMs) have achieved remarkable success but remain highly susceptible to jailbreak attacks, in which adversarial prompts coerce models into generating harmful, unethical, or policy-violating outputs. Such attacks pose real-world risks, eroding safety, trust, and regulatory compliance in high-stakes applications. Although a variety of attack and defense methods have been proposed, existing evaluation practices are inadequate, often relying on narrow metrics like attack success rate that fail to capture the multidimensional nature of LLM security. In this paper, we present a systematic taxonomy of jailbreak attacks and defenses and introduce Security Cube, a unified, multi-dimensional framework for comprehensive evaluation of these techniques. We provide detailed comparison tables of existing attacks and defenses, highlighting key insights and open challenges across the literature. Leveraging Security Cube, we conduct benchmark studies on 13 representative attacks and 5 defenses, establishing a clear view of the current landscape encompassing jailbreak attacks, defenses, automated judges, and LLM vulnerabilities. Based on these evaluations, we distill critical findings, identify unresolved problems, and outline promising research directions for enhancing LLM robustness against jailbreak attacks. Our analysis aims to pave the way towards more robust, interpretable, and trustworthy LLM systems. Our code is available at Code.

Momentum

Related topics

Latest papers 123