LLM Jailbreak Attacks

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, down 12% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 137

All topics

Mitigating Many-shot Jailbreak Attacks with One Single Demonstration
May 8, 2026Kejia Chen, Jiawen Zhang, Boheng Li +6Jailbreak Attacks In-Context Learning
Zhejiang University · Nanyang Technological University · University of Science and Technology of China +2
A Systematic Investigation of RL-Jailbreaking in LLMs
May 7, 2026Montaser Mohammedalamen, Kevin Roice, Reginald McLean +1Language Model Safety Evaluation Adversarial Attacks on LLMs
Alberta Machine Intelligence Institute (Amii), Edmonton, Canada.
On the Hardness of Junking LLMs
May 6, 2026Marco Rando, Samuel VaiterLLM Backdoor Attacks LLM Jailbreak Attacks
Université Côte d’Azur, Inria, CNRS, LJAD, Nice, France · CNRS & Université Côte d’Azur, LJAD, Nice, France
SoK: Robustness in Large Language Models against Jailbreak Attacks
May 6, 2026Feiyue Xu, Hongsheng Hu, Chaoxiang He +9Language Model Safety Evaluation LLM Security
Shanghai Jiao Tong University, China · Microsoft Corporation
Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimization
May 6, 2026Zheng Fang, Xiaosen Wang, Shenyi Zhang +2Deep Learning Optimization Audio-Language Models
1Wuhan University · 2Huazhong University of Science and Tech · Institute for Math & AI, Wuhan University +2
Self-Mined Hardness for Safety Fine-Tuning
May 4, 2026Prakhar Gupta, Garv Shah, Donghua ZhangHard Negative Mining LLM Fine-Tuning
University of Michigan
ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming
May 4, 2026Mario Rodríguez Béjar, Francisco J. Cortés-Delgado, S. Braghin +1Evolutionary Optimization LLM Red Teaming
Universidad de Murcia · IBM Research
Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment
May 3, 2026Jiajia Li, Xiaoyu Wen, Zhongtian Ma +3LLM Alignment LLM Safety Alignment
Northwestern Polytechnical University · Shanghai AI Laboratory · Shanghai Jiao Tong University +1
MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
May 3, 2026Jialin Song, Xiaodong Liu, Weiwei Yang +4LLM Safety Benchmarks Adversarial Prompt Generation
Simon Fraser University · Microsoft Research
SRTJ: Self-Evolving Rule-Driven Training-Free LLM Jailbreaking
May 1, 2026Jindong Li, Ying Liu, Yali Fu +4LLM Jailbreak Attacks
1HKUST (GZ) · 2Jilin University · 3Yale University
RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs
May 1, 2026Zhiyuan Xu, Joseph Gardiner, Sana Belguith +1Mixture-of-Experts Language Models Adversarial Attacks on LLMs
University of Bristol Bristol, United Kingdom
Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models
Apr 30, 2026Shubham Kumar, Narendra AhujaLLM Interpretability Causal Reasoning in Language Models
University of Illinois Urbana-Champaign
TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning
Apr 30, 2026Bowen Sun, Chaozhuo Li, Yaodong Yang +2Jailbreak Attacks LLM Security
Johns Hopkins University · Microsoft Research Asia · Peking University +1
Cross-Lingual Jailbreak Detection via Semantic Codebooks
Apr 28, 2026Shirin Alanova, Bogdan Minko, Sabrina Sadiekh +1Multilingual Language Model Evaluation LLM Security
AI Talent Hub, ITMO University · HiveTraceLab
Adaptive Prompt Embedding Optimization for LLM Jailbreaking
Apr 27, 2026Miles Q. Li, Benjamin C. M. Fung, Boyang Li +2Adversarial Prompt Generation Adversarial Attacks on LLMs
School of Information Studies, McGill University, Montreal, QC, Canada · Department of Computer Science, Kean University, Union, NJ, United States · Mila - Quebec AI Institute, Montreal, QC, Canada +1
Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models
Apr 27, 2026Nay Myat Min, Long H. Pham, Jun SunLLM Backdoor Attacks Adversarial Attacks on LLMs
Singapore Management University
Jailbreaking Frontier Foundation Models Through Intention Deception
Apr 27, 2026Xinhe Wang, Katia Sycara, Yaqi XieJailbreak Attacks LLM Safety
Carnegie Mellon University
Evaluating Jailbreaking Vulnerabilities in LLMs Deployed as Assistants for Smart Grid Operations: A Benchmark Against NERC Standards
Apr 25, 2026Taha Hammadia, Lucas Rea, Ahmad Mohammad Saber +2LLM Jailbreak Attacks LLM Safety Evaluation
ECE Department, University of Toronto, Toronto, ON Canada · CIISE, Concordia University, Montréal, QC, Canada
From Concept-Aligned Tokens to Vulnerable Features: Mechanistic Localization of Jailbreaks
Apr 25, 2026Nilanjana Das, Mathew Dawit, Aman Chadha +1LLM Interpretability Sparse Autoencoders
UMBC, Maryland, U.S.A. · Apple, California, U.S.A.
AVISE: Framework for Evaluating the Security of AI Systems
Apr 22, 2026Mikko Lempinen, Joni Kemppainen, Niklas RaesalmiLanguage Model Safety Evaluation Adversarial Attacks on LLMs
University of Oulu
One Jailbreak, Many Tongues: Learning Language-Insensitive Intention Representations for Multilingual Jailbreak Detection
Apr 22, 2026Shuyu Jiang, Kaiyu Xu, Xingshu Chen +5Multilingual Language Models LLM Jailbreak Attacks
School of Cyber Science and Engineering, Sichuan University, Chengdu, 610065, China · School of Computer Science and Engineering, Nanyang Technological University · School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China
EvoJail: Evolutionary Diverse Jailbreak Prompt Generation for Large Language Models
Apr 22, 2026Rui Tang, Kaiyu Xu, Pengsen Cheng +3Adversarial Prompt Generation LLM Jailbreak Attacks
aSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China · dCollege of Computer Science, Sichuan University, Chengdu 610065, China · aSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China bKey Laboratory of Data Protection and Intelligent Management (Sichuan University), Ministry of Education, Chengdu 610065, China
HarDBench: A Benchmark for Draft-Based Co-Authoring Jailbreak Attacks for Safe Human-LLM Collaborative Writing
Apr 21, 2026Euntae Kim, Soomin Han, Buru ChangLLM Safety Benchmarks Language Model Safety Evaluation
Korea University · Sogang University
STAR-Teaming: A Strategy-Response Multiplex Network Approach to Automated LLM Red Teaming
Apr 21, 2026MinJae Jung, YongTaek Lim, Chaeyun Kim +3LLM Security LLM Jailbreak Attacks
DATUMO INC.
Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks
Apr 20, 2026Md Rysul Kabir, Zoran TiganjLanguage Model Safety Evaluation LLM Interpretability
Department of Computer Science Luddy School of Informatics, Computing, and Engineering Indiana University Bloomington
Jailbreaking Large Language Models with Morality Attacks
Apr 18, 2026Ying Su, Mingen Zheng, Weili Diao +1Moral Reasoning in Language Models Language Model Safety Evaluation
South China University of Technology1 · HKUST2 · Beihang University3
SafeDream: Safety World Model for Proactive Early Jailbreak Detection
Apr 18, 2026Bo Yan, Weikai Lin, Yada Zhu +1Language Model Safety Evaluation World Models
University of Central Florida · University of Rochester · IBM Research
Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framing
Apr 17, 2026Zehao Wang, Lanjun WangAdversarial Prompt Generation Jailbreak Attacks
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China. · School of New Media and Communication, Tianjin University, Tianjin 300072, China. · Shanghai Key Laboratory of Data Science,[cs.LG] Shanghai 200438, China.
Segment-Level Coherence for Robust Harmful Intent Probing in LLMs
Apr 16, 2026Xuanli He, Bilgehan Sel, Faizan Ali +3LLM Safety LLM Jailbreak Attacks
University College London · Virginia Tech · Anthropic
How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Models
Apr 6, 2026Gregory N. FrankLLM Interpretability LLM Safety
Chief Scientist, Molt AI Corp.

May 8, 2026, cs.CRJ/K move · Enter open · S save

Mitigating Many-shot Jailbreak Attacks with One Single Demonstration

Kejia Chen, Jiawen Zhang, Boheng Li, Pengcheng Li+5

Zhejiang University · Nanyang Technological University · University of Science and Technology of China+2

Jailbreak Attacks In-Context Learning LLM Safety LLM Jailbreak Attacks Multi-Turn Jailbreak Attacks Jailbreak Defense

Open paper AskPDF arXiv

Many-shot jailbreaking (MSJ) causes safety-aligned language models to answer harmful queries by preceding them with many harmful question-answer demonstrations. We study why this attack becomes stronger as the number of demonstrations increases. Empirically, we find that MSJ induces a progressive activation drift: the representation of a fixed harmful query moves step by step away from the safety-aligned region as more harmful demonstrations are added. Theoretically, we show that this drift can be interpreted as implicit malicious fine-tuning: conditioning on N harmful demonstrations induces SGD-style updates equivalent to optimizing on the corresponding N harmful samples. This view turns the attack mechanism into a defense principle. We append a fixed one-shot safety demonstration at inference time, which induces a counteracting safety-oriented update and restores refusal behavior. The resulting method improves the model's robustness to MSJ without modifying its parameters or requiring white-box access at deployment. Code is available at https://github.com/Thecommonirin/SafeEnd.

Momentum

Related topics

Latest papers 137