Adversarial Attacks on LLMs

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 311% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 238

All topics

One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails
Oct 8, 2026Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud PedramLLM Guardrails LLM Agent Security
University of Southern California (USC), USA
BRANCH: Bypassing Multi-Scanner AI Guardrails
Oct 7, 2026William Hackett, Peter GarraghanLLM Guardrails Adversarial Attacks on LLMs
Mindgard
Quad-State Safety Evaluation of Open-Weight Large Language Models on Non-Canonical Inputs
Oct 6, 2026Pavan MaddulaLanguage Model Safety Evaluation Adversarial Robustness of Language Models
Independent Researcher
How Fragile Is On-Device Language Model Safety? Localizing Safety-Critical Parameters for Sparse Fault Analysis
Oct 6, 2026Muhammad Zeeshan Karamat, Christiana Chamon GarciaLLM Safety Language Model Safety Evaluation
Virginia Tech
Secure Speculative Decoding for Large Language Models
Oct 6, 2026Yichi Zhang, Zhiqi Wang, Neil Gong +1Adversarial Attacks on LLMs Speculative Decoding
The Pennsylvania State University · Duke University
Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge
Oct 5, 2026Wonjun Lee, Kyungsik Yang, Gaeun Ji +5Adversarial Attacks on LLMs LLM Security
UNC Chapel Hill · Kyung Hee University · AIM Intelligence +1
Jailbreaking Open-Weight LLMs via Random Embedding Perturbations
Oct 5, 2026Abhinav Sudhakar Dubey, Scott Sirri, Vaggos Chatziafratis +1Adversarial Attacks Adversarial Attacks on LLMs
University of California Santa Cruz
Reward Stealing Attack on Large Language Models
Oct 5, 2026Jiaming Qian, Pengyang Zhou, Jiahe Xu +1LLM Alignment Model Extraction Attacks
Zhejiang University
Don't Judge an LLM Only by Its Activations: Discovering Suppressed Safety Features via Counterfactual Activation Potential
Oct 4, 2026Swadesh Swain, Sanghamitra DuttaAdversarial Attacks on LLMs Mechanistic Interpretability
University of Maryland, College Park
Reflections and Fragments: Securing LLMs Against Sequential Mosaic Attacks
Oct 4, 2026Emanuele La Malfa, Saar Cohen, Gabriele La Malfa +4Adversarial Attacks on LLMs LLM Safety
Department of Computer Science, University of Oxford · Institute for Decentralized AI (IDAI) · Department of Informatics, King’s College London +2
Evaluating and Improving the Robustness of Large Language Models to Input Sequence Variations
Oct 1, 2026Narek MaloyanLanguage Model Robustness Adversarial Attacks on LLMs
TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety
Oct 1, 2026Fengpeng Li, Kemou Li, Qizhou Wang +3LLM Fine-Tuning Adversarial Attacks on LLMs
PRADA Lab, King Abdullah University of Science and Technology · State Key Laboratory of Internet of Things for Smart City, University of Macau · Imperfect Information Learning Team, RIKEN Center for Advanced Intelligence Project +1
CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion
Sep 30, 2026Zhen Liang, Hai Huang, Wentao ChenAdversarial Attacks on LLMs Jailbreak Attacks
School of Computer Science and Technology, Zhejiang Sci-Tech University · Zhejiang Key Laboratory of Digital Fashion and Data Governance, Zhejiang Sci-Tech University · China Academy of Information and Communications Technology
Hiding in Plain Sight: Decoupling Pretext from Actuation for Skill Poisoning in LLM Agents
Sep 30, 2026Wenxin Wu, Lingyong Yan, Lei Sha +2Adversarial Attacks on LLMs LLM Agent Security
Beihang University · Baidu Inc · Wilfrid Laurier University
MADBench: Benchmarking the Security of Multi-Agent Debate
Sep 30, 2026Yuwan Liu, Jiaming Zhang, Yue Huang +1Adversarial Attacks on LLMs Multi-Agent Debate
Tsinghua University
Breaking the Illusion of Review Reliability under Static Evaluation: SCOPE Fuzzing for LLM-based Scientific Reviewers
Sep 29, 2026Zhuo Chen, Hao Zeng, Jiawei Liu +6LLM Evaluation Adversarial Attacks on LLMs
Wuhan University · East China Normal University
ACTR: Aligning Thoughts and Responses for Multilingual Safety in Reasoning LLMs
Sep 29, 2026Xianhui Zhang, Jian Yu, Chengyu Xie +6Adversarial Attacks on LLMs LLM Safety Alignment
Nanjing University of Science and Technology, Nanjing, China · National University of Singapore, Singapore · Beihang University, Beijing, China
Controlled Decoding Attacks on Black-Box LLMs
Sep 29, 2026Jesson Wang, Shawn Li, Wei Yang +4Constrained Decoding Adversarial Attacks on LLMs
University of Southern California · Adobe · Amazon
Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue
Sep 29, 2026Omar Sheta, Rinku Deuja, Hadi Masoudi +1Language Model Safety Evaluation Adversarial Attacks on LLMs
University of Louisville Louisville, Kentucky, USA
LLMs Learn to Evade Latent Monitors from Prior Feedback Alone
Sep 29, 2026Hugo Lyons Keenan, Christopher Leckie, Sarah ErfaniLLM Auditing Adversarial Attacks on LLMs
School of Computing and Information Systems The University of Melbourne, Victoria, Australia
Guard Models Are Overconfident Where Base Models Are Uncertain
Sep 29, 2026Jonghyun Hong, MinJae Jung, Minwoo KimLanguage Model Safety Evaluation Adversarial Attacks on LLMs
DATUMO INC.
Epistemic Policy Divergence in Multi-Turn LLM Contamination: A Protocol-Gradient Investigation
Sep 28, 2026Fahrell Giovanny, Geby Bayuningtyas, Sahrul Mukharom +1Knowledge Conflicts in Language Models Adversarial Attacks on LLMs
Independent Researcher, Tokyo, Japan · Independent Researcher, Jakarta, Indonesia · Department of Informatics, Institut Teknologi Sumatera, Lampung, Indonesia
One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs
Sep 27, 2026Sen Nie, Jie Zhang, Zhongqi Wang +2Adversarial Attacks on VLMs Adversarial Attacks on LLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, and the University of Chinese Academy of Sciences.
SafeMol: Dual-Modality Safety Alignment for Molecular Multimodal Models
Sep 27, 2026Xinmiao Wang, Ruijie Wang, Menghui Wang +5Adversarial Attacks on LLMs LLM Safety Alignment
Beihang University · China CITIC Bank · Stable AI +1
Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learning
Sep 27, 2026Chenlong Yin, Xiaolong Jin, Wei Zou +2Adversarial Attacks on LLMs LLM Red Teaming
The Pennsylvania State University · Purdue University
JevOut: Natural Context Can Flip Decision Models
Sep 24, 2026Zixiang Xu, Zirui Song, Chiyu Zhang +4Adversarial Attacks on LLMs LLM Decision-Making
University of Southern California
Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs
Sep 24, 2026Lukáš Brůna, Robert Bridges, Adam EkAdversarial Attacks on LLMs Prompt Injection Attacks
Uppsala University · AI Sweden
The Tokens Remember: When Tokenization Bypasses Knowledge Editing and Unlearning
Sep 24, 2026Manit Baser, Aditya Nawal, Dinil Mon Divakaran +1Adversarial Attacks on LLMs Privacy Leakage in Language Models
National University of Singapore · A*STAR Institute of Advanced Intelligence and Computing, Singapore
Recovering Agentic Sovereignty: Mitigating the Consensus Paradox via Contrastive Epistemic Decoding
Sep 22, 2026Dahlia Shehata, Ming LiLLM Sycophancy Adversarial Attacks on LLMs
University of Waterloo Canada
ALIBI: Adversarial Legitimacy Injection in Binary Input against LLM Malware Analyzers
Sep 17, 2026Hyeongjun Choi, Wonyoung Jung, Haehoon Seo +1Adversarial Attacks on LLMs LLM Security
78ResearchLab Seoul, South Korea

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails

Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram

University of Southern California (USC), USA

LLM Guardrails LLM Agent Security Adversarial Attacks on LLMs Language Model Safety Evaluation Prompt Injection Attacks on AI Agents

Open paper AskPDF arXiv

A typed decision model reads a piece of text and returns a probability over caller-defined options, each with a short written definition, generating no text. Recent work places these models in agent systems as guardrails: the component that reads a proposed tool call or incoming message and decides whether to allow it. We evaluate seven open-weight models in that role and report the two error directions separately: a fail-open error allows a prohibited action and is a vulnerability; a fail-closed error blocks a permitted one and is only a cost. On prompt-injection, jailbreak and toxic-content screening, accuracy at the allow-or-block decision ranges from 36% to 72% against a chance level of 50%. A low error rate in one direction only reflects which answer a model defaults to: one allows nearly everything, another blocks nearly everything. On a synthetic suite of agent tool calls, six lines of server log text that say nothing about the policy raise a gate's fail-open rate from 0% to 63% on a policy it otherwise decides correctly. Giving the permissive option a misleading name, with its definition and the judged text untouched, raises that rate to between 93% and 100% on the four models that place the label in their input. Every defense we tested is defeated, either by an attacker who targets its mechanism or by attacker-controlled text. Escalating the least confident decisions does not help either: a decision an attack has reversed is no less confident than the one it replaced. Parsing each policy field into a typed value does eliminate one attack, but it also makes the model unnecessary: a deterministic rule over those values reaches 100% accuracy on all six policies. These models can reduce how many cases reach a reviewer, but on this evidence they should not be the component that decides. Code is available at https://github.com/ArminAzizi98/option-channel-attack.

Momentum

Related topics

Latest papers 238