Adversarial Attacks on LLMs

LLM: Large Language Model

Overview

This field studies how attackers can make large language models produce unsafe or unreliable outputs, leak information, manipulate judgments, or misuse tools. Common approaches include black-box prompt search and multi-turn red teaming, gradient-based suffix optimization, reinforcement-learning attackers, and interventions on model activations or weights. Researchers evaluate these attacks on benchmarks such as HarmBench, AdvBench, and JailbreakBench, as well as in multilingual, multimodal, and tool-using agent settings. A central challenge is that adaptive attacks often expose weaknesses missed by static tests, while defenses must improve robustness without excessive false refusals, lost capability, or new system-level vulnerabilities.

Latest papers

CardsList

385 papers30 papers in the last four weeks, down 40% on the four weeks before. 0.2% of all new papers.

  1. Diffusion LLMs are Natural Adversaries for any LLM

    Oct 31, 2025David Lüdke, Tom Wollschläger, Paul Ungermann +2Adversarial Prompt GenerationAdversarial Attacks on LLMs

  2. Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks

    Oct 16, 2025Trilok Padhi, Pinxian Lu, Abdulkadir Erol +5Adversarial Attacks on LLMsAI Agent Security Benchmarks

  3. Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization

    Oct 8, 2025Tiancheng Xing, Jerry Li, Yixuan Du +1Adversarial Attacks on LLMsLLM Reranking

  4. NonTextual Target Attack

    Oct 3, 2025Xinzhe Huang, Wenjing Hu, Tianhang Zheng +6Adversarial Prompt GenerationAdversarial Attacks on LLMs

  5. There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning

    Aug 6, 2025Roman Belaire, Arunesh Sinha, Pradeep VarakanthamLLM Red TeamingAdversarial Attacks on LLMs

  6. Data Security in Large Language Models: Risks, Defense, and Directions

    Aug 4, 2025Kang Chen, Xiuze Zhou, Yuanhui Yu +4Adversarial Attacks on LLMsLLM Security

  7. Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models

    Jul 7, 2025Ziqi Miao, Lijun Li, Yuan Xiong +7Adversarial Attacks on LLMsLLM Jailbreak Attacks

  8. In-Context Bias Propagation in LLM-Based Tabular Data Generation

    Jun 11, 2025Pol G. Recasens, Alberto Gutierrez, Jordi Torres +4Synthetic Data GenerationSynthetic Tabular Data Generation

  9. Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models

    Jun 9, 2025Mickel Liu, Liwei Jiang, Yancheng Liang +4Self-Play RLAdversarial Attacks on LLMs

  10. Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

    Jun 8, 2025Ren-Jian Wang, Ke Xue, Zeyu Qin +7LLM Red TeamingAdversarial Prompt Generation

  11. HauntAttack: When Attack Follows Reasoning as a Shadow

    Jun 8, 2025Jingyuan Ma, Rui Li, Zheng Li +4Adversarial Attacks on LLMsJailbreak Attacks

  12. BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage

    Jun 3, 2025Kalyan Nakka, Nitesh SaxenaJailbreak AttacksLLM Jailbreak Attacks

  13. Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models

    Mar 8, 2025Thomas Winninger, Boussad Addad, Katarzyna KapustaAdversarial Prompt GenerationAdversarial Attacks on LLMs

  14. UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models

    Feb 18, 2025Huawei Lin, Yingjie Lao, Tony Geng +2LLM Backdoor AttacksAdversarial Attacks on LLMs

  15. Towards the Worst-case Robustness of Large Language Models

    Jan 31, 2025Huanran Chen, Zeming Wei, Yao Huang +2Adversarial Robustness of Language ModelsAdversarial Attacks on LLMs

  16. Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models

    Dec 21, 2024Yanxu Mao, Peipei Liu, Tiehan Cui +3Adversarial Attacks on LLMsMultimodal Large Language Models

  17. Could Mobile AI be Exploited?: Understanding the Exploitations of Trust and Ethics Gap in On-Device Small Language Models

    Jun 8, 2024Kalyan Nakka, Jimmy Dani, Nitesh SaxenaLanguage Model Safety EvaluationLLM Safety

  18. Learning diverse attacks on large language models for robust red-teaming and safety tuning

    May 28, 2024Seanie Lee, Minsu Kim, Lynn Cherif +8Adversarial Attacks on LLMsLLM Red Teaming

  19. On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex

    Jan 30, 2023Terry Yue Zhuo, Zhuang Li, Yujin Huang +4Adversarial Robustness of Language ModelsSemantic Parsing