Adversarial Robustness of Language Models

Ask about this topic

Overview

The field examines whether language models preserve safe, accurate, and consistent behavior when facing jailbreaks, prompt injections, meaning-preserving edits, misleading evidence, or pressure across multiple turns. Researchers test models with red-teaming and adversarial benchmarks, then strengthen them through adversarial training, preference and reinforcement learning, parameter-efficient updates, and inference-time guards or steering. Mechanistic methods also inspect and modify internal representations, while certification aims to provide guarantees for defined threat models. A central challenge is that defenses that work on known attacks may fail against adaptive or out-of-distribution ones, and stronger protection can reduce utility or cause over-refusal; evaluations therefore increasingly measure both robustness and capability across diverse settings.

Latest papers

CardsList

192 papers17 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. AdvJudge-Zero: Binary Decision Flips in LLM-as-a-Judge via Adversarial Control Tokens

    Dec 19, 2025Tung-Ling Li, Yuhao Wu, Hongliang LiuLLM-as-a-JudgeAdversarial Attacks on LLMs

    Palo Alto Networks

  2. CluCERT: Certifying LLM Robustness via Clustering-Guided Denoising Smoothing

    Dec 1, 2025Zixia Wang, Gaojie Jin, Jia Hu +1Randomized SmoothingSemantic Clustering

    University of Exeter

  3. destroR: A Benchmark and Adversarial-Training Defense for Bangla Transfer Models under Meaning-Preserving Attacks

    Nov 13, 2025Saadat Rafid Ahmed, Rubayet Shareen, Radoan Sharkar +3Adversarial Text AttacksAdversarial Robustness of Language Models

    Dept. of Computer Science and Engineering, BRAC University

  4. Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content

    Sep 16, 2025Shaz Furniturewala, Arkaitz ZubiagaMechanistic InterpretabilityToxicity Detection

    Center for Data Science, New York University · Queen Mary University of London

  5. Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming

    Jul 30, 2025Jiazhen Pan, Bailiang Jian, Paul Hager +20LLM Safety BenchmarksLanguage Model Safety Evaluation

    Cherise · School of Computation, Information and Technology, Technical University of Munich (TUM), Munich, Germany · Department of Engineering Science, University of Oxford, Oxford, United Kingdom +9

  6. Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents

    Jul 3, 2025Sizhe Chen, Arman Zharmagambetov, David Wagner +1LLM Agent SecurityAI Agent Security Benchmarks

    FAIR at Meta · UC Berkeley

  7. Safety Alignment via Constrained Knowledge Unlearning

    May 24, 2025Zesheng Shi, Yucheng Zhou, Jing LiLLM Safety AlignmentLLM Unlearning

    Harbin Institute of Technology, Shenzhen, China · University of Macau, China

  8. Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models

    Apr 25, 2025Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín +1LLM Fine-TuningRL for Language Model Reasoning

    Department of Computer Systems, Technical University of Madrid, Madrid, Spain · United Arab Emirates University, Al Ain, United Arab Emirates

  9. Unified Enhancement of the Generalization and Robustness of Language Models via Bi-Stage Optimization

    Mar 19, 2025Yudao Sun, Juan Yin, Juan Zhao +3Adversarial TrainingNeural Network Generalization

    Department of New Networks, Peng Cheng Laboratory, Shenzhen 518055, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore 117576 · School of Data Science and Artificial Intelligence, Chang’an University, Shaanxi, 710064, China

  10. DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization

    Dec 10, 2024Xiaoqun Liu, Weiming Qi, Qiben YanAdversarial Robustness of Language ModelsLLM Jailbreak Attacks

    Michigan State University, East Lansing, MI 48824, USA

  11. TSCheater: Generating High-Quality Tibetan Adversarial Texts via Visual Similarity

    Dec 3, 2024Xi Cao, Quzong Gesang, Yuan Sun +2Adversarial Robustness of Language ModelsAdversarial Text Attacks

    Minzu University of China, Beijing, China · National Language Resource Monitoring & Research Center Minority Languages Branch, Beijing, China · Tibet University, Lhasa, China +1

  12. Learning diverse attacks on large language models for robust red-teaming and safety tuning

    May 28, 2024Seanie Lee, Minsu Kim, Lynn Cherif +8Adversarial Attacks on LLMsLLM Red Teaming

    KAIST · Mila – Québec AI Institute · Université de Montréal +3