Adversarial Attacks on LLMs

LLM: Large Language Model

Ask about this topic

Overview

This field studies how attackers can make large language models produce unsafe or unreliable outputs, leak information, manipulate judgments, or misuse tools. Common approaches include black-box prompt search and multi-turn red teaming, gradient-based suffix optimization, reinforcement-learning attackers, and interventions on model activations or weights. Researchers evaluate these attacks on benchmarks such as HarmBench, AdvBench, and JailbreakBench, as well as in multilingual, multimodal, and tool-using agent settings. A central challenge is that adaptive attacks often expose weaknesses missed by static tests, while defenses must improve robustness without excessive false refusals, lost capability, or new system-level vulnerabilities.

Latest papers

CardsList

358 papers51 papers in the last four weeks, up 89% on the four weeks before. 0.3% of all new papers.

  1. Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion

    Apr 11, 2026Vishal Pramanik, Maisha Maliha, Susmit Jha +1Adversarial Attacks on LLMsLLM Jailbreak Attacks

    Department of Computer & Information Science & Engineering, University of Florida, Gainesville, FL 32611, USA · School of Computer Science, University of Oklahoma, Norman, OK 73019, USA · Computer Science Laboratory, SRI International, Menlo Park, CA 94025, USA

  2. Conflicts Make Large Reasoning Models Vulnerable to Attacks

    Apr 10, 2026Honghao Liu, Chengjin Xu, Xuhui Jiang +5LLM SafetyLarge Reasoning Models

    International Digital Economy Academy · The Hong Kong University of Science and Technology (Guangzhou) · DataArc Tech +1

  3. Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation

    Apr 9, 2026Wenpeng Xing, Moran Fang, Guangtai Wang +2Adversarial Attacks on LLMsJailbreak Attacks

    Zhejiang University · Binjiang Institute of Zhejiang University

  4. Beyond Social Pressure: Benchmarking Epistemic Attack in Large Language Models

    Apr 9, 2026Steven Au, Sujit NoronhaLLM Safety BenchmarksLanguage Model Safety Evaluation

    Independent Researcher

  5. Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelines

    Apr 8, 2026Tanzim Ahad, Ismail Hossain, Md Jahangir Alam +4LLM Agent SecurityLLM Agent Safety

    Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · Hanyang University, Seoul, South Korea +1

  6. Gradient-Controlled Decoding: A Safety Guardrail for LLMs with Dual-Anchor Steering

    Apr 6, 2026Purva Chiniya, Kevin Scaria, Sagar ChaturvediLLM GuardrailsLLM Safety

    Amazon Alexa · Amazon AGI

  7. Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling

    Apr 6, 2026Qingyang Xu, Yaling Shen, Stephanie Fong +7LLM SafetyAdversarial Attacks on LLMs

    Monash University · University of Liverpool

  8. Towards Unveiling Vulnerabilities of Large Reasoning Models in Machine Unlearning

    Apr 5, 2026Aobo Chen, Chenxu Zhao, Chenglin Miao +1Machine UnlearningLLM Unlearning

    Iowa State University, Ames IA 50011, USA

  9. FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment

    Apr 5, 2026Daniel Kuznetsov, Ofir Cohen, Karin Shistik +2LLM Safety AlignmentAdversarial Attacks on LLMs

    Computer and Information Science, Ben-Gurion University of the Negev, Beer-Sheva, Israel

  10. CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks

    Apr 5, 2026Siyuan Li, Zehao Liu, Xi Lin +6LLM SecurityAdversarial Robustness of Language Models

    School of Computer Science, Shanghai Jiao Tong University · Department of Computer Science, University of Illinois Urbana-Champaign · Big Data Institute, Central South University

  11. Generalization Limits of Reinforcement Learning Alignment

    Apr 3, 2026Haruhi Shida, Koo Imai, Keigo KansaLLM AlignmentAdversarial Attacks on LLMs

    Aladdin Security Inc.

  12. The Persistent Vulnerability of Aligned AI Systems

    Mar 31, 2026Aengus LynchAI Agent SafetyAI Alignment

    Department of Computer Science, University College London

  13. Security in LLM-as-a-Judge: A Comprehensive SoK

    Mar 31, 2026Aiman Al Masoud, Antony Anju, Marco Arazzi +6LLM-as-a-JudgeLLM Security

    Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy · Department of Computer Applications, Cochin University of Science and Technology, Kerala, India

  14. CivicShield: A Cross-Domain Defense-in-Depth Framework for Securing Government-Facing AI Chatbots Against Multi-Turn Adversarial Attacks

    Mar 30, 2026KrishnaSaiReddy PatilLLM SecurityCybersecurity

  15. Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuning

    Mar 30, 2026Bilgehan Sel, Xuanli He, Alwin Peng +2Adversarial Attacks on LLMsLLM Safety

    Anthropic Fellows Program · Virginia Tech · University College London, UK +1

  16. Exploring LLM biases to manipulate AI search overview

    Mar 30, 2026Roman SmirnovAdversarial Attacks on LLMsLanguage Model Bias Evaluation

  17. H-Node Attack and Defense in Large Language Models

    Mar 27, 2026Eric Yocam, Varghese Vaidyan, Yong WangLLM Hallucination MitigationHallucination in Language Models

    Department of Computer Science and Software Engineering, California Polytechnic State University, San Luis Obispo, CA 93407, USA. · Beacom College of Computer and Cyber Sciences, Dakota State University, Madison, SD 57042, USA. · Department of Computer Science, University of Idaho, Moscow, ID 83844, USA.

  18. Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models

    Mar 26, 2026Xunguang Wang, Yuguang Zhou, Qingyue Wang +5LLM Safety MonitoringLLM Safety

    The Hong Kong University of Science and Technology, Hong Kong, China · Zhejiang University of Technology, Hangzhou, Zhejiang Province, China

  19. Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs

    Mar 25, 2026Alexander Panfilov, Peter Romov, Igor Shilov +3Adversarial Attacks on LLMsAdversarial Prompt Generation

    MATS · ELLIS Institute Tübingen & Max Planck Institute for Intelligent Systems · Tübingen AI Center +1

  20. Adaptively Robust LLM Monitoring via Activation Watermarking

    Mar 24, 2026Toluwani Aremu, Daniil Ognev, Samuele Poppi +1LLM AuditingLLM Watermarking

    Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)

  21. Structured Visual Narratives Undermine Safety Alignment in Multimodal Large Language Models

    Mar 23, 2026Rui Yang Tan, Yujia Hu, Roy Ka-Wei LeeMLLM SafetyAdversarial Attacks on VLMs

    Singapore University of Technology and Design. · University of British Colombia.

  22. When AI Agrees Too Much: Manipulating Automated Code Review via Bias Injection

    Mar 19, 2026Georgios Alexopoulos, Nikolaos Alexopoulos, Diomidis Spinellis +1Automated Code ReviewAdversarial Attacks on LLMs

    University of Athens · National Infrastructures for Research and Technology · Athens University of Economics and Business +1

  23. Security Assessment and Mitigation Strategies for Large Language Models: A Comprehensive Defensive Framework

    Mar 17, 2026Taiwo Onitiju, Iman VakiliniaLLM SecurityAdversarial Attacks on LLMs

    School of Computing, University of North Florida

  24. Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure

    Mar 17, 2026Caglar YildirimLLM Agent SafetyLLM Agent Evaluation

    Khoury College of Computer Sciences Northeastern University Boston, MA 02115, USA

  25. Structured Semantic Cloaking for Jailbreak Attacks on Large Language Models

    Mar 17, 2026Xiaobing Sun, Perry Lam, Shaohua Li +4Adversarial Attacks on LLMsLLM Jailbreak Attacks

    Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore. · Information Systems Technology and Design Pillar, Singapore University of Technology and Design.

  26. Resource Consumption Threats in Large Language Models

    Mar 17, 2026Yuanhe Zhang, Xinyue Wang, Zhican Chen +8LLM SecurityLLM Inference Efficiency

    Beijing University of Posts and Telecommunications · Hangzhou Dianzi University · Nanyang Technological University +1

  27. Re-Mask and Redirect: Exploiting Denoising Irreversibility in Diffusion Language Models

    Mar 17, 2026Arth SinghAdversarial Attacks on LLMsAdversarial Attacks on Diffusion Models

    AIM Intelligence · National Institute of Technology Agartala

  28. Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graph

    Mar 16, 2026Zhenheng Tang, Xiang Liu, Qian Wang +3LLM AlignmentLLM Safety

    The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou) · New York University +1

  29. Amplification Effects in Test-Time Reinforcement Learning: Safety and Reasoning Vulnerabilities

    Mar 16, 2026Vanshaj Khattar, Md Rafi ur Rashid, Moumita Choudhury +4Test-Time RLLLM Safety

    Virginia Tech · Penn State University · University of Massachusetts Amherst +1

  30. SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration

    Mar 16, 2026Yu Pan, Wenlong Yu, Tiejun Wu +4LLM Safety AlignmentJailbreak Defense

    Department College of Intelligence and Computing, Tianjin University · NSFOCUS Technologies Group Co., Ltd. · College of Management and Economics, Tianjin University +1