Adversarial Attacks on LLMs

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 311% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 238

All topics

When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints
Apr 18, 2026Yuheng Chen, Zhiyu Wu, Bowen Cheng +2Adversarial Attacks on LLMs LLM Safety Alignment
Kagoshima University · Fudan University · China University of Petroleum-Beijing
Stochasticity in Tokenisation Improves Robustness
Apr 17, 2026Sophie Steger, Rui Li, Sofiane Ennadir +4Adversarial Attacks on LLMs Neural Network Robustness
Institute of Signal Processing and Speech Communication, Graz University of Technology, Graz, Austria · ELLIS Institute Finland & Aalto University, Espoo, Finland · King AI Labs, Microsoft Gaming +2
Conjunctive Prompt Attacks in Multi-Agent LLM Systems
Apr 17, 2026Nokimul Hasan Arif, Qian Lou, Mengxin ZhengMulti-Agent LLM Systems Adversarial Attacks on LLMs
University of Central Florida, USA
Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover
Mar 11, 2026Indranil Halder, Annesya Banerjee, Cengiz PehlevanAdversarial Attacks on LLMs Prompt Injection Attacks on LLMs
John A. Paulson School of Engineering And Applied Sciences, Harvard University · Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology · Speech and Hearing Bioscience and Technology, Harvard Medical School +2
Prompt Injection as Role Confusion
Feb 22, 2026Charles Ye, Jasmine Cui, Dylan Hadfield-MenellAdversarial Attacks on LLMs Prompt Injection Attacks
*Equal contribution 1Independent · 2Massachusetts Institute of Technology, Cambridge, MA, United States
Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges
Feb 14, 2026Ruomeng Ding, Yifei Pang, He Sun +3LLM Alignment LLM-as-a-Judge
University of North Carolina at Chapel Hill · 2Carnegie Mellon University · 3Yale University +1
Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model
Feb 8, 2026Tianyi Wang, Huawei Fan, Yuanchao Shu +2LLM Security Adversarial Attacks on LLMs
Zhejiang University
Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
Jan 29, 2026Xiang Zheng, Yutao Wu, Hanxun Huang +5Prompt Injection AI Coding Agents
City University of Hong Kong · Deakin University · The University of Melbourne +3
In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement
Jan 19, 2026Anudeex Shetty, Aditya Joshi, Salil S. KanhereLanguage Model Safety Evaluation Adversarial Attacks on LLMs
School of Computer Science and Engineering, UNSW Sydney, Australia · School of Computing and Information System, the University of Melbourne, Australia
Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
Jan 7, 2026Iago Alves Brito, Walcy Santos Rezende Rios, Julia Soares Dollis +2LLM Safety Benchmarks LLM Alignment
Advanced Knowledge Center for Immersive Technologies (AKCIT) Federal University of Goiás (UFG)
Hidden State Poisoning Attacks against Mamba-based Language Models
Jan 5, 2026Alexandre Le Mercier, Chris Develder, Thomas DemeesterAdversarial Attacks on LLMs Language Modeling
IDLab–T2K, Ghent University–imec
Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety
Dec 8, 2025Max Zhang, Derek Liu, Kai Zhang +2Adversarial Attacks on LLMs LLM Safety Alignment
Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrieval
Nov 30, 2025Hao Wu, Prateek SaxenaAdversarial Attacks on LLMs LLM Security
National University of Singapore
Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks
Oct 16, 2025Trilok Padhi, Pinxian Lu, Abdulkadir Erol +5Adversarial Attacks on LLMs AI Agent Security Benchmarks
1Georgia State University · 2Georgia Institute of Technology · University of California, Berkeley
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Oct 11, 2025Wentian Zhu, Zhen Xiang, Wei Niu +1Adversarial Attacks on LLMs LLM Security
University of Georgia
Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization
Oct 8, 2025Tiancheng Xing, Jerry Li, Yixuan Du +1Adversarial Attacks on LLMs LLM Reranking
National University of Singapore · University of Southern California · Georgetown University +1
NonTextual Target Attack
Oct 3, 2025Xinzhe Huang, Wenjing Hu, Tianhang Zheng +6Adversarial Prompt Generation Adversarial Attacks on LLMs
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou 310007, China · Hangzhou High-Tech Zone, Binjiang Institute of Blockchain and Data Security, Hangzhou 310053, China · School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing 210094, China +3
Data Security in Large Language Models: Risks, Defense, and Directions
Aug 4, 2025Kang Chen, Xiuze Zhou, Yuanhui Yu +4Adversarial Attacks on LLMs LLM Security
School of Computer Engineering, Jimei University, Xiamen, 361021, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China · School of Informatics, Xiamen University, Xiamen, 361102, China +1
Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness
Jun 30, 2025Tung-Ling Li, Hongliang LiuLLM Alignment Language Model Safety Evaluation
Palo Alto Networks
Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
Jun 9, 2025Mickel Liu, Liwei Jiang, Yancheng Liang +4Self-Play RL Adversarial Attacks on LLMs
Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA · Department of Computer Science, Stanford University, Stanford, CA, USA
HauntAttack: When Attack Follows Reasoning as a Shadow
Jun 8, 2025Jingyuan Ma, Rui Li, Zheng Li +4Adversarial Attacks on LLMs Jailbreak Attacks
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · StepFun · Department of Computing, The Hong Kong Polytechnic University +1
Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models
Mar 8, 2025Thomas Winninger, Boussad Addad, Katarzyna KapustaAdversarial Prompt Generation Adversarial Attacks on LLMs
*Work started during internship at Thales.
UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models
Feb 18, 2025Huawei Lin, Yingjie Lao, Tony Geng +2LLM Backdoor Attacks Adversarial Attacks on LLMs
Rochester Institute of Technology · Tufts University · University of Rochester +1
Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-Based Robots
Dec 21, 2024Xuancun Lu, Zhengxian Huang, Xinfeng Li +3Large Language Model-Based Robot Planning Adversarial Attacks on LLMs
Zhejiang University
Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models
Dec 21, 2024Yanxu Mao, Peipei Liu, Tiehan Cui +3Adversarial Attacks on LLMs Multimodal Large Language Models
School of Computer Science and Technology, Shanghai Jiao Tong University, Shanghai, China · Zhongguancun Laboratory, Beijing, China · School of Software, Henan University, Kaifeng, China +1
Learning diverse attacks on large language models for robust red-teaming and safety tuning
May 28, 2024Seanie Lee, Minsu Kim, Lynn Cherif +8Adversarial Attacks on LLMs LLM Red Teaming
KAIST · Mila – Québec AI Institute · Université de Montréal +3
A False Average: Pooled CoT-Monitor Accuracy Conceals a Reasoning-Dependent Fragility
Date pendingShikhar Shiromani, Leo RichterReward Hacking Adversarial Attacks on LLMs
Pivotal Research · University College London
Whitewashing Hate, Smearing Harmless Content: Annotator-Style Rebuttal Attacks on LLM-Based Moderation
Date pendingJunyu Lu, Kaiyuan Liu, Kaichun Wang +8Content Moderation Adversarial Attacks on LLMs
Dalian University of Technology · Zhejiang University · Tencent +1

Apr 18, 2026, cs.CLJ/K move · Enter open · S save

When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints

Yuheng Chen, Zhiyu Wu, Bowen Cheng, Yihang Wu+1

Kagoshima University · Fudan University · China University of Petroleum-Beijing

Adversarial Attacks on LLMs LLM Safety Alignment LLM Safety Evaluation Refusal Behavior in Language Models

Open paper AskPDF arXiv

We identify and systematically characterize a class of task-structural alignment failures in large language models (LLMs): even when the harmful intent remains unchanged, changing the task presentation and output constraints can substantially alter model safety behavior. Specifically, when a harmful request is reformulated as a forced-choice multiple-choice question (MCQ) in which all options are harmful and no refusal option is provided, some models that refuse the equivalent open-ended query instead select, prefer, or justify a harmful option. We evaluate 14 proprietary and open-source models on a bilingual Chinese-English human-authored dataset covering five harm categories, together with 900 model-generated Chinese adversarial MCQs. On human-authored data, attack success rate (ASR) increases sharply as prompts shift from open-ended queries to explicit forced-choice formats, typically peaking under intermediate levels of choice constraint. Model-generated Chinese MCQs further weaken or eliminate the recovery regime observed on human-authored data, driving ASR close to saturation for multiple models. The observed transfer patterns are consistent with stronger generators producing more difficult or boundary-adjacent MCQs, although other properties of the generated inputs may also contribute. We also find that adding an explicit refusal option or a safety preamble substantially reduces ASR for several high-capability models, often to near-zero levels, although their effectiveness varies across target models. These findings suggest that safety evaluations centered on open-ended generation may underestimate risks in structured deployment settings, and that task structure should be treated as an important and diagnosable dimension of safety evaluation and alignment training.

Momentum

Related topics

Latest papers 238