Adversarial Attacks on LLMs

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 311% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 238

All topics

Test-Time Training Undermines Safety Guardrails
May 21, 2026Simone Antonelli, Sadegh Akhondzadeh, Aleksandar BojchevskiLanguage Model Safety Evaluation Adversarial Attacks on LLMs
CISPA Helmholtz Center for Information Security · University of Cologne
Latent-space Attacks for Refusal Evasion in Language Models
May 20, 2026Giorgio Piras, Raffaele Mura, Fabio Brau +4Adversarial Attacks on LLMs LLM Safety
University of Cagliari · University of Genova
How Far Will They Go? Red-Teaming Online Influence with Large Language Models
May 20, 2026Daniel C. Ruiz, Anna Serbina, Ashwin Rao +2Language Model Steering LLM Auditing
Information Sciences Institute University of Southern California Los Angeles, CA, USA
Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes
May 19, 2026Mohammed Alshaalan, Miguel R. D. RodriguesAdversarial Attacks on LLMs Jailbreak Detection
Department of Electronic and Electrical Engineering, University College London, London, United Kingdom.
Attention-Guided Reward for Reinforcement Learning-based Jailbreak against Large Reasoning Models
May 19, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +2Adversarial Attacks on LLMs LLM Jailbreak Attacks
Xidian University · Xi’an Jiaotong University
Adaptive Probe-based Steering for Robust LLM Jailbreaking
May 19, 2026Junxi Chen, Junhao Dong, Xiaohua XieLanguage Model Steering Adversarial Attacks on LLMs
School of Computer Science and Engineering, Sun Yat-Sen University, China · Nanyang Technological University, Singapore.
When Efficiency Backfires: Cascading LLMs Trigger Cascade Failure under Adversarial Attack
May 17, 2026Zehan Sun, Dingfan Chen, Songze LiAdversarial Attacks on LLMs LLM Security
1Southeast · 2Max Planck Institute for Intelligent Systems
Propagating Unsafe Actions in LLM Controlled Multi-Robot Collaboration via Single Robot Compromise
May 15, 2026Zhen Huang, Zhihuang Liu, Mengxuan Luo +2Large Language Model-Based Robot Planning Adversarial Attacks on LLMs
College of Computer Science and Technology, National University of Defense Technology
Widening the Gap: Exploiting LLM Quantization via Outlier Injection
May 14, 2026Xiaohua Zhan, Kazuki Egashira, Robin Staab +2LLM Quantization LLM Backdoor Attacks
ETH Zurich
Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks
May 14, 2026Karthik Raghu Iyer, Yazdan Jamshidi, Nicholas Bray +1LLM Safety Benchmarks Benchmark Auditing
Palo Alto Networks
A Few Steps Further: Why Defenses Against Malicious Finetuning Erode Under Continued Training
May 14, 2026Itay Zloczower, Eyal Lenga, Gilad Gressel +1Language Model Safety Evaluation Adversarial Attacks on LLMs
Ben-Gurion University of the Negev · Amrita Vishwa Vidyapeetham, Amritapuri
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
May 13, 2026Shuoyang Sun, Chang Dai, Hao Fang +6Adversarial Attacks on LLMs Speculative Decoding
Harbin Institute of Technology, Shenzhen · South China University of Technology · Tsinghua Shenzhen International Graduate School, Tsinghua University +1
Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution
May 13, 2026Xiaozhe Zhang, Chaozhuo Li, Hui Liu +4Continual Learning for LLMs Adversarial Attacks on LLMs
City University of Hong Kong · Beijing University of Posts and Telecommunications · Wuhan University +2
Inducing Overthink: Hierarchical Genetic Algorithm-based DoS Attack on Black-Box Large Language Reasoning Models
May 13, 2026Shuqiang Wang, Wei Cao, Jiaqi Weng +4Overthinking in Language Models Adversarial Attacks on LLMs
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Alibaba Group
Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models
May 13, 2026Yejin Lee, Ungsik Kim, Yo-Sub HanAdversarial Attacks on LLMs Diffusion Language Model Inference
Department of Computer Science Yonsei University Seoul, Republic of Korea
Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
May 13, 2026Adarsh Kumarappan, Ananya MujooAI Agent Reliability Multi-Agent LLM Systems
1California Institute of Technology · 2Evergreen Valley College
REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
May 12, 2026Buyun Liang, Jinqi Luo, Liangzu Peng +6Adversarial Prompt Generation Adversarial Attacks
University of Pennsylvania
CoT-Guard: Small Models for Strong Monitoring
May 12, 2026Nirav Diwan, Han Wang, Berkcan Kapusuzoglu +6LLM Auditing Adversarial Attacks on LLMs
†University of Illinois Urbana-Champaign · ♢Capital One
Metaphor Is Not All Attention Needs
May 12, 2026Olga Sorokoletova, Francesco Giarrusso, Giacomo De Luca +6Adversarial Attacks on LLMs Jailbreak Attacks
Sapienza University of Rome Department of Computer, Control and Management Engineering Sapienza University of Rome Via Ariosto 25, Rome 00185, Italy · DEXAI – Icaro Lab · University of Rome Tor Vergata +1
Seirênes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning
May 12, 2026Chi Zhang, Haibo Qiu, Qiming Zhang +3RL for Language Model Reasoning Adversarial Attacks on LLMs
School of Computer Science, Wuhan University · Meituan Inc · Independent Researchers +1
Comment and Control: Hijacking Agentic Workflows via Context-Grounded Evolution
May 11, 2026Neil Fendley, Zhengyu Liu, Aonan Guan +2Adversarial Attacks on LLMs Agentic Workflows
Johns Hopkins University Maryland, USA · Wyze Labs Washington, USA
PRISM: Generation-Time Detection and Mitigation of Secret Leakage in Multi-Agent LLM Pipelines
May 11, 2026Riya Tapwal, Abhishek Kumar, Carsten MapleData Leakage Multi-Agent LLM Systems
School of Computing and Electrical Engineering · Indian Institute of Technology, Mandi (IIT) · The Alan Turing Institute, London +1
Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization
May 11, 2026Huilin Zhou, Jian Zhao, Yilu Zhong +7Inference-Time Optimization Adversarial Attacks on LLMs
University of Science and Technology of China · Institute of Artificial Intelligence (TeleAI), China Telecom
FragBench: Cross-Session Attacks Hidden in Benign-Looking Fragments
May 10, 2026Astha Mehta, Niruthiha Selvanayagam, Cedric Lam +10LLM Safety Benchmarks Language Model Safety Evaluation
Isabella · ÉTS Montreal / SPAR · Mila / Lida Safety
LLM-Agnostic Semantic Representation Attack
May 9, 2026Jiawei Lian, Jianhong Pan, Lefan Wang +4Adversarial Prompt Generation Adversarial Attacks on LLMs
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR · Hong Kong Institute of AI for Science, City University of Hong Kong, Hong Kong SAR
OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents
May 9, 2026Xinyu Li, Ronghui Mu, Lin Li +2Adversarial Attacks on LLMs LLM Red Teaming
Department of Computer Science, University of Exeter · Department of Computer Science, University of Oxford · Department of Mathematics and Computer Science, Eindhoven University of Technology
A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models
May 8, 2026Hamid Kazemi, Atoosa Chegini, Maria SafiLLM Interpretability Adversarial Attacks on LLMs
Apple · Work done during an internship at Apple. Present affiliation: University of Maryland, College Park.
The Attacker in the Mirror: Breaking Self-Consistency in Safety via Anchored Bipolicy Self-Play
May 8, 2026Gabriele La Malfa, Emanuele La Malfa, Saar Cohen +4Self-Play RL Language Model Safety Evaluation
Department of Informatics, King’s College London · Department of Computer Science, University of Oxford · Institute for Decentralized AI +1
A Systematic Investigation of RL-Jailbreaking in LLMs
May 7, 2026Montaser Mohammedalamen, Kevin Roice, Reginald McLean +1Language Model Safety Evaluation Adversarial Attacks on LLMs
Alberta Machine Intelligence Institute (Amii), Edmonton, Canada.
LoopTrap: Termination Poisoning Attacks on LLM Agents
May 7, 2026Huiyu Xu, Zhibo Wang, Wenhui Zhang +4Adversarial Attacks on LLMs LLM Red Teaming
The State Key Laboratory of Blockchain and Data Security Zhejiang University Hangzhou, Zhejiang, P. R. China · School of Cyber Science and Engineering Southeast University Nanjing, Jiangsu, P. R. China

May 21, 2026, cs.LGJ/K move · Enter open · S save

Test-Time Training Undermines Safety Guardrails

Simone Antonelli, Sadegh Akhondzadeh, Aleksandar Bojchevski

CISPA Helmholtz Center for Information Security · University of Cologne

Language Model Safety Evaluation Adversarial Attacks on LLMs Jailbreak Attacks Test-Time Training LLM Safety

Open paper AskPDF arXiv

Test-Time Training (TTT) is an emerging paradigm that enables models to adapt their parameters during inference, improving performance on tasks such as few-shot learning, retrieval-augmented generation, and complex reasoning. However, this dynamic adaptation introduces new vulnerabilities that adversaries can exploit to jailbreak models. We identify three threat models for TTT and demonstrate how attackers can leverage them to bypass safety filters. Our results show that TTT can significantly increase the Attack Success Rate (ASR) and the ASR over 10 generation trials (ASR@10). For example, under LoRA, the few-shot and generation-phase threat models achieve an average ASR@10 of 95% and 93% respectively, across models from different families and scales. These vulnerabilities transfer to production fine-tuning APIs. We also show that TTT-induced overfitting can produce degenerate outputs that inflate ASR under standard judges, and propose a validity-aware evaluation to correct for this. Our findings suggest that TTT exposes a new attack surface, strengthens attacks, and undermines existing safety guardrails. As a first step toward defense, we propose a lightweight provider-side detector that flags TTT requests via the perplexity shift on a private harmful holdout, but robust deployment will ultimately require dynamic alignment.

Momentum

Related topics

Latest papers 238