Adversarial Attacks on LLMs

LLM: Large Language Model

Overview

This field studies how attackers can make large language models produce unsafe or unreliable outputs, leak information, manipulate judgments, or misuse tools. Common approaches include black-box prompt search and multi-turn red teaming, gradient-based suffix optimization, reinforcement-learning attackers, and interventions on model activations or weights. Researchers evaluate these attacks on benchmarks such as HarmBench, AdvBench, and JailbreakBench, as well as in multilingual, multimodal, and tool-using agent settings. A central challenge is that adaptive attacks often expose weaknesses missed by static tests, while defenses must improve robustness without excessive false refusals, lost capability, or new system-level vulnerabilities.

Latest papers

CardsList

385 papers30 papers in the last four weeks, down 40% on the four weeks before. 0.2% of all new papers.

  1. Hardening Agentic Web Populations Through Multi-Criterion Competitive Co-Evolution: Lessons from Wargame Strategy Optimization

    Nov 14, 2026Shashank Raj, Kalyanmoy DebAI Agent SecurityAdversarial Attacks on LLMs

  2. Adversarially Attacking Symbolic Vocabulary Vulnerabilities In LLM Planners

    Nov 13, 2026Stephen Obadinma, Xiaodan ZhuLLM PlanningAdversarial Attacks on LLMs

  3. One Fabrication Is Enough: A Fake News Article Changes the Forecasts of LLM Forecasting Systems

    Nov 11, 2026Zeyu Zhang, Bradly C. StadieRAG Poisoning AttacksAdversarial Attacks on LLMs

  4. Chatter Attack: Resource Consumption Attack for Large Language Models

    Nov 8, 2026Yibo Miao, Yichuan Cao, Xiao-Shan Gao +1Adversarial Attacks on LLMsLLM Security

  5. A Layer-wise Study of How Style Rewriting Breaks the Calibration of Zero-Shot LLM Fake-News Detectors

    Nov 8, 2026Prabhjot Singh, Praney Goyal, Raksha SharmaLanguage Model RobustnessLanguage Model Calibration

  6. Beyond the Prompt: Leveraging Pre-Decoding States for Jailbreak Detection in dLLMs

    Nov 6, 2026Adam Hazimeh, Amel Abdelraheem, Ke Wang +4Jailbreak DetectionLLM Safety Monitoring

  7. META-IR: Meta-Introspective Reasoning for Adaptive Safety Alignment in LLM

    Nov 6, 2026Mahabubur Rahman Miraj, Hongyu Huang, Nankun Mu +2LLM Safety AlignmentAdversarial Robustness of Language Models

  8. Logit Laundering: Evading Data-Use Auditing in Large Language Models

    Nov 6, 2026Ruihan Hu, Wei Luo, Yu-Ming Shang +5LLM AuditingLanguage Model Auditing

  9. Debate to Deceive: Poisoning LLM's Fact-Checking via Game-Theoretic Multi-Agent Debate

    Nov 5, 2026Liting Huang, Huizhi Liang, Yi Lei +2Adversarial Attacks on LLMsDeception in Language Models

  10. RL-ATTACK: A Generalized Black-Box Adversarial Attack on Language Models via Reinforcement Learning

    Nov 5, 2026Taikun Chen, Yue Zhang, Wei Qian +1Adversarial Attacks on LLMsBlack-Box Adversarial Attacks

  11. Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models

    Nov 4, 2026Hamid Dashtbani, Alexandra DmitrienkoLLM Safety AlignmentSafety-Preserving Fine-Tuning of Language Models

  12. JevBreak: Your Model, Their Decision

    Nov 2, 2026Nitin Aravind Birur, Divyanshu Kumar, Sahil Agarwal +1Adversarial AttacksAdversarial Text Attacks

  13. The Customer Never Sees the Goal: Measuring and Detecting Vulnerability-Targeted Manipulation in LLM Assistants

    Nov 2, 2026Bohan Jiang, Dawei Li, Yasin Silva +1LLM AuditingPersuasion in Language Models

  14. Understanding and Mitigating Role Spoofing in LLMs

    Oct 29, 2026Ezra Edelman, Benjamin Edelman, Surbhi Goel +1Adversarial Attacks on LLMsPrompt Injection Attacks on LLMs

  15. Adversarial Self-Distillation

    Oct 28, 2026Mohammad Sadegh Akhondzadeh, Soroush H. Zargarbashi, Aleksandar BojchevskiAdversarial Robustness of Language ModelsSafety-Preserving Fine-Tuning of Language Models

  16. SEMA-Auto: Sighted and Efficient Multi-Turn Attack with Autonomy

    Oct 26, 2026Mingqian Feng, Xiaodong Liu, Weiwei Yang +8Adversarial Attacks on LLMsJailbreak Attacks

  17. Do AI-for-Science Agents Preserve Model Safety?

    Oct 23, 2026Michael Yew Chuan Lee, Manya Sriram, N. Benjamin ErichsonLLM Agent SafetyLLM Safety Evaluation

  18. Diffusion Language Models for Perplexity-Aware Black-Box Attacks

    Oct 22, 2026Ali Al-Lawati, Elsayed Eshra, Dongwon Lee +1Adversarial Attacks on LLMsLLM Safety Evaluation

  19. After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents

    Oct 20, 2026Zhiyue Xu, Xukai Wang, Zhiyuan Huang +3Adversarial Attacks on LLMsPrompt Injection Attacks on AI Agents

  20. Polarity Blindness in LLM Safety: A Systematic Study of Negation, Antonym Substitution, and Adversarial Framing

    Oct 20, 2026Diego Miguel Lozano, Erik Derner, Juan Antonio Pérez-Ortiz +1Language Model Safety EvaluationAdversarial Robustness of Language Models