LLM Jailbreak Attacks

LLM: Large Language Model

LLM jailbreak research examines how prompts and interactions can circumvent safety alignment, and how to make models resist those attempts. Attacks range from roleplay, obfuscation, and multi-turn persuasion to automated prompt optimization, internal activation steering, and attacks on audio, code, and tool-using agents. Defenses include input filters, intent detectors, safety fine-tuning, representation-level interventions, and safeguards that track risk across a conversation; evaluations increasingly test diverse languages, adaptive attacks, and harmful actions that actually execute. A central challenge is improving robustness without rejecting benign requests, while inconsistent judges, budgets, and test settings make reported attack success difficult to compare.

Latest papers

207 papers25 papers in the last four weeks, up 79% on the four weeks before. 0.1% of all new papers.

Discovering Jailbreaks via Curriculum of Models
Oct 28, 2026Michał Burzyński, Noah T. Rossi, Shomit Basu +1LLM Security LLM Jailbreak Attacks
University of Warsaw · Independent · Northeastern University
Rethinking LLM Jailbreak Evaluation: Continuous Risk Landscapes Beyond Binary Outcomes
Oct 28, 2026Yukai Zhou, Wenjie WangLanguage Model Safety Evaluation Jailbreak Attack Evaluation
ShanghaiTech University
Poisoned Skills: Propagating Jailbreaks Through Agent Markdown Files
Oct 27, 2026Kashyap R Manjusha, Murali EmaniPrompt Injection Attacks on AI Agents LLM Jailbreak Attacks
University of Illinois, Urbana-Champaign, IL, USA · Argonne National Laboratory, Lemont, IL, USA
SEMA-Auto: Sighted and Efficient Multi-Turn Attack with Autonomy
Oct 26, 2026Mingqian Feng, Xiaodong Liu, Weiwei Yang +8Adversarial Attacks on LLMs Jailbreak Attacks
University of Rochester · Microsoft Research · Microsoft Azure
Identically Trained Language Models Break on Different Jailbreaks
Oct 25, 2026Chi-Wei Chang, Mu-Tien Kuo, Kuan-Wei Lee +1Language Model Safety Evaluation LLM Jailbreak Attacks
Cerulean Labs · National Taiwan University, Taipei, Taiwan · National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Harmful Responses Are Jailbreak Templates: Transferring Autoregressive Jailbreaks to Diffusion Language Models
Oct 22, 2026Jinwei Gan, Zifeng Cheng, Zhiwei Jiang +3LLM Jailbreak Attacks Adversarial Attacks on Diffusion Models
State Key Laboratory for Novel Software Technology, Nanjing University · Nanjing University of Posts and Telecommunications
Budgeted Attack Scheduling for LLM Red-Teaming: The Roles of Static Priors, Semantic Guidance, and Online Feedback
Oct 19, 2026Zhicheng Fang, Han Sun, Jiachen Lei +5LLM Red Teaming Language Model Safety Evaluation
Shanghai Qi Zhi Institute · East China Normal University · The Chinese University of Hong Kong +3
Pair-Aware Optimization: Coordinating Exploration and Refinement for Jailbreaking LLM Agents
Oct 17, 2026Juanyang Xu, Zheng Wang, Xingyu Zhao +2Adversarial Attacks on LLMs Jailbreak Attacks
University of Macau · WMG, University of Warwick · Wuhan University
ECHO: Replacing the Compliance Proxy with Response Harmfulness in Optimization-Based Jailbreak Attack
Oct 16, 2026Yuanwei Wu, Lingdong Huang, Yunlong Mao +3LLM Jailbreak Attacks
Nanjing University · University of the Chinese Academy of Sciences · University of Macau
RedDarwin: Evolving Jailbreak Skills against Frontier LLMs
Oct 14, 2026Zhouyuan Ma, Yutao Wu, Hanxun Huang +3LLM Jailbreak Attacks LLM Red Teaming
Fudan University · Deakin University · University of Melbourne +2
Did the LLM Answer the Question? A Semantic Reconstruction Game for Jailbreak Evaluation
Oct 13, 2026Liam Davis, Romain Emil-Popescu, Suman JanaJailbreak Attack Evaluation LLM Safety Evaluation
Amherst College · Columbia University
SpecCloak: Probing Harmful Code Generation through Engineering Specifications
Oct 13, 2026Wenbin Chen, Jiexin Wang, Guanting Wu +2LLM Security LLM Code Generation
South China University of Technology
Narrative Paradoxes in LLM Inference: Shaping Reasoning Trajectories toward Misalignment
Oct 12, 2026Haoming Yang, Ke Ma, Yangbangyan Jiang +6Adversarial Attacks on LLMs LLM Safety
School of Electronic, Electrical and Communication Engineering, UCAS · School of Computer Science and Technology, UCAS · Nanyang Technological University, Singapore +3
ATTCK-R: Knowledge-Aware Long-Horizon Jailbreaking for Protected LLMs
Oct 12, 2026Md Mushfiqur Rahman, Tobias Rohde, Rahul Tripathi +2LLM Jailbreak Attacks LLM Red Teaming
George Mason University · Amazon
Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Models
Oct 10, 2026Lass Najmuldeen, Seyedhamidreza MousaviAdversarial Robustness of Language Models LLM Jailbreak Attacks
School of Innovation, Design and Engineering, Mälardalen University, Västerås, Sweden
How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defense
Oct 7, 2026Zhankai Ye, Yanning Wang, Yukai Jin +5Refusal Behavior in Language Models LLM Safety Alignment
Florida State University · Texas Christian University · University of Pennsylvania +1
PatchBench: Measuring Collateral Damage in Activation Patching
Oct 7, 2026Alexi Canesse, Mathis Le Bail, Maël Jenny +3Jailbreak Attack Evaluation Language Model Safety Evaluation
LIX (École Polytechnique, IP Paris, CNRS), France · AMIAD (Agence Ministérielle pour l’IA de Défense), France
From Expected Harmfulness to Likelihood: A Probabilistic Reformulation of Jailbreaking LLM Agents
Oct 7, 2026Juanyang Xu, Zheng Wang, Xingyu Zhao +2Jailbreak Attacks LLM Jailbreak Attacks
University of Macau · WMG, University of Warwick · Wuhan University
Secure Speculative Decoding for Large Language Models
Oct 6, 2026Yichi Zhang, Zhiqi Wang, Neil Gong +1Adversarial Attacks on LLMs Speculative Decoding
The Pennsylvania State University · Duke University
Jailbreaking Open-Weight LLMs via Random Embedding Perturbations
Oct 5, 2026Abhinav Sudhakar Dubey, Scott Sirri, Vaggos Chatziafratis +1Adversarial Attacks Adversarial Attacks on LLMs
University of California Santa Cruz
Benchmarking Jailbreak Guardrails for Embodied Agents
Oct 5, 2026Xunguang Wang, Qingyue Wang, Yuguang Zhou +3LLM Guardrails AI Agent Safety
Department of Computer Science and Engineering The Hong Kong University of Science and Technology Clear Water Bay, Kowloon, Hong Kong · School of Information Renmin University of China Beijing, China
Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models
Oct 4, 2026Tongyan Hu, Hao Li, Xiaogeng Liu +8Test-Time Training LLM Jailbreak Attacks
Johns Hopkins University · National University of Singapore · Washington University in St. Louis +2
Reactivating Alignment: Defending LLMs from Jailbreaks via Intention-Aware Input-Output Matching
Oct 3, 2026Luoyu Chen, Weiqi Wang, Chenhan Zhang +2Jailbreak Defense LLM Safety Alignment
University of Technology Sydney, Sydney, NSW, Australia · Xi’an Jiaotong University, Xi’an, Shaanxi, China · Southeast University, Nanjing, Jiangsu, China
Target-free Latent Safety Alignment
Oct 3, 2026Luoyu Chen, Weiqi Wang, Chenhan Zhang +3LLM Safety Alignment Adversarial Robustness of Language Models
University of Technology Sydney, Sydney, NSW, Australia · Xi’an Jiaotong University, Xi’an, Shaanxi, China · Southeast University, Nanjing, Jiangsu, China +1
MLCommons Jailbreak Benchmark v1.0
Oct 2, 2026Carsten Maple, Cagatay Yucel, Isaac Holeman +35Jailbreak Attack Evaluation LLM Safety Benchmarks
Victor · MLCommons · University of Warwick +18
Intent-Hiding Jailbreaks: An Information-Theoretic Framework for Compositional Attacks
Oct 1, 2026Fengwei Tian, Ravi TandonJailbreak Attacks LLM Security
School of Electrical, Computing and Software Engineering, University of Arizona, Tucson, AZ 85721, USA.
CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion
Sep 30, 2026Zhen Liang, Hai Huang, Wentao ChenAdversarial Attacks on LLMs Jailbreak Attacks
School of Computer Science and Technology, Zhejiang Sci-Tech University · Zhejiang Key Laboratory of Digital Fashion and Data Governance, Zhejiang Sci-Tech University · China Academy of Information and Communications Technology
Controlled Decoding Attacks on Black-Box LLMs
Sep 29, 2026Jesson Wang, Shawn Li, Wei Yang +4Constrained Decoding Adversarial Attacks on LLMs
University of Southern California · Adobe · Amazon
Constitutional adapters: Inference-time interventions for misalignment and misuse
Sep 29, 2026Adam S. Lowet, Mark KurzejaLLM Safety Alignment LLM Jailbreak Attacks
Anthropic Fellows Program · Independent
Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking
Sep 26, 2026Marco BiroliJailbreak Attacks LLM Jailbreak Attacks
Physical Science Division Chicago, IL 60637, USA

Oct 28, 2026J/K move · Enter open · S save

Discovering Jailbreaks via Curriculum of Models

Michał Burzyński, Noah T. Rossi, Shomit Basu, Andy Arditi

University of Warsaw · Independent · Northeastern University

LLM Security LLM Jailbreak Attacks Curriculum RL

Open paper AskPDF arXiv

Automated jailbreak discovery aims to find prompts that elicit prohibited behaviors from safety-trained language models, helping developers identify vulnerabilities before deployment. One class of approaches uses reinforcement learning to train an investigator model to generate prompts that elicit a specified behavior from a target model. However, training directly against a safety-trained target is difficult because nearly all prompts are refused, leaving the investigator with a sparse reward signal. Prior work addresses this with the propensity bound (PRBO), a proxy reward estimated by sampling responses from a more compliant target, and then adjusting for how likely those responses would have been under the original target. We introduce Curriculum of Models (CoM), an alternative method that trains the investigator against a sequence of modified versions of the target, progressing gradually from a version that readily exhibits the specified behavior to the original target. For jailbreak discovery, training begins against a target version that rarely refuses harmful requests, and advances to the next, harder version once the investigator reliably elicits the behavior from the current one. We evaluate CoM on elicitation tasks involving CBRN materials, explosives, and illicit drugs, using two open-weight targets, gemma-4-31b-it and gpt-oss-120b. We compare against PRBO and direct training on the original target without a curriculum. When the investigator is trained to elicit a single behavior, CoM finds successful jailbreaks more reliably than either baseline. When trained on multiple behaviors simultaneously, the CoM-trained investigator can elicit held-out behaviors, though PRBO-trained investigators achieve higher success rates. Our results suggest that a simple curriculum over target models offers a promising approach for addressing sparse rewards in rare-behavior elicitation.