LLM Jailbreak Attacks

LLM: Large Language Model

LLM jailbreak research examines how prompts and interactions can circumvent safety alignment, and how to make models resist those attempts. Attacks range from roleplay, obfuscation, and multi-turn persuasion to automated prompt optimization, internal activation steering, and attacks on audio, code, and tool-using agents. Defenses include input filters, intent detectors, safety fine-tuning, representation-level interventions, and safeguards that track risk across a conversation; evaluations increasingly test diverse languages, adaptive attacks, and harmful actions that actually execute. A central challenge is improving robustness without rejecting benign requests, while inconsistent judges, budgets, and test settings make reported attack success difficult to compare.

Latest papers

207 papers25 papers in the last four weeks, up 79% on the four weeks before. 0.1% of all new papers.

Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking
Feb 27, 2026Zhicheng Fang, Jingjie Zheng, Chenxu Fu +1Jailbreak Attack Evaluation LLM Safety Benchmarks
Shanghai Qi Zhi Institute, Shanghai, China · University of Melbourne, Melbourne, Australia · Tsinghua University, Beijing, China
Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Search
Feb 26, 2026Xun Huang, Simeng Qin, Xiaoshuang Jia +6LLM Jailbreak Attacks Adversarial Prompt Generation
Nanyang Technological University, Singapore · BraneMatrix AI, China · Nanjing University of Science and Technology, China +6
Analysis of LLMs Against Prompt Injection and Jailbreak Attacks
Feb 24, 2026Piyush Jaiswal, Aaditya Pratap, Shreyansh Saraswati +2Prompt Injection Attacks on LLMs Language Model Safety Evaluation
NIT Trichy, Tiruchirappalli, India · NCE Chandi, Nalanda, India · Bishop Cotton Boys’ School, Bengaluru, India +2
When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models
Feb 21, 2026Zafir Shamsi, Nikhil Chekuru, Zachary Guzman +1Adversarial Attacks on LLMs Language Model Safety Evaluation
Algoverse AI Research
Recursive language models for jailbreak detection: a procedural defense for tool-augmented agents
Feb 18, 2026Doron ShavitJailbreak Detection LLM Security
Intent Laundering: AI Safety Datasets Are Not What They Seem
Feb 17, 2026Shahriar Golchin, Marc WetterLLM Safety Evaluation LLM Safety Benchmarks
Applied Machine Learning Research
Boundary Point Jailbreaking of Black-Box LLMs
Feb 16, 2026Xander Davies, Giorgi Giglemiani, Edmund Lau +3Adversarial Attacks on LLMs Jailbreak Attacks
UK AI Security Institute · OATML, University of Oxford
Sparse Autoencoders are Capable LLM Jailbreak Mitigators
Feb 12, 2026Yannick Assogba, Jacopo Cortellazzi, Javier Abad +3Sparse Autoencoders LLM Jailbreak Attacks
1Apple · 2ETH Zurich
Causal Front-Door Adjustment for Robust Jailbreak Attacks on LLMs
Feb 5, 2026Yao Zhou, Zeen Song, Wenwen Qiang +2Adversarial Attacks on LLMs LLM Jailbreak Attacks
Institute of Software Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement
Jan 19, 2026Anudeex Shetty, Aditya Joshi, Salil S. KanhereLanguage Model Safety Evaluation Adversarial Attacks on LLMs
School of Computer Science and Engineering, UNSW Sydney, Australia · School of Computing and Information System, the University of Melbourne, Australia
Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
Jan 7, 2026Iago Alves Brito, Walcy Santos Rezende Rios, Julia Soares Dollis +2LLM Safety Benchmarks LLM Alignment
Advanced Knowledge Center for Immersive Technologies (AKCIT) Federal University of Goiás (UFG)
One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs
Dec 14, 2025Yixin Tan, Zhe Yu, Rui Wen +1LLM Security Adversarial Attacks on LLMs
Institute of Science Tokyo, Tokyo, Japan · RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan
MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs
Nov 10, 2025Liang Shan, Kaicheng Shen, Wen Wu +8LLM Safety LLM Safety Alignment
School of Computer Science and Technology, East China Normal University · Shanghai AI Lab, Shanghai Innovation Institute
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
Oct 17, 2025Hanbin Hong, Shuang Wu, Shuya Feng +6LLM Security LLM Jailbreak Attacks
University of Connecticut · University of Alabama at Birmingham · Independent
Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks
Oct 16, 2025Trilok Padhi, Pinxian Lu, Abdulkadir Erol +5Adversarial Attacks on LLMs AI Agent Security Benchmarks
1Georgia State University · 2Georgia Institute of Technology · University of California, Berkeley
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Oct 11, 2025Wentian Zhu, Zhen Xiang, Wei Niu +1Adversarial Attacks on LLMs LLM Security
University of Georgia
NonTextual Target Attack
Oct 3, 2025Xinzhe Huang, Wenjing Hu, Tianhang Zheng +6Adversarial Prompt Generation Adversarial Attacks on LLMs
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou 310007, China · Hangzhou High-Tech Zone, Binjiang Institute of Blockchain and Data Security, Hangzhou 310053, China · School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing 210094, China +3
Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
Oct 1, 2025Shoumik Saha, Jifan Chen, Sam Mayers +3AI Coding Agents AI Agent Security
University of Maryland – College Park · AWS AI Labs · Meta Superintelligence Labs
Context Misleads LLMs: The Role of Context Filtering in Maintaining Safe Alignment of LLMs
Aug 9, 2025Jinhwa Kim, Ian G. HarrisLLM Safety Alignment LLM Safety
Department of Computer Science University of California, Irvine
LLMs Encode Harmfulness and Refusal Separately
Jul 16, 2025Jiachen Zhao, Jing Huang, Zhengxuan Wu +2LLM Interpretability LLM Refusal Behavior
Northeastern University · Stanford University
HauntAttack: When Attack Follows Reasoning as a Shadow
Jun 8, 2025Jingyuan Ma, Rui Li, Zheng Li +4Adversarial Attacks on LLMs Jailbreak Attacks
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · StepFun · Department of Computing, The Hong Kong Polytechnic University +1
Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models
May 12, 2025Haoran Gu, Handing Wang, Yi Mei +2LLM Safety Alignment LLM Safety
Xidian University · Victoria University of Wellington · Westlake University
Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models
Mar 8, 2025Thomas Winninger, Boussad Addad, Katarzyna KapustaAdversarial Prompt Generation Adversarial Attacks on LLMs
*Work started during internship at Thales.
GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods
Feb 24, 2025Ruixuan Huang, Xunguang Wang, Zongjie Li +2Language Model Safety Evaluation LLM Jailbreak Attacks
The Hong Kong University of Science and Technology, Hong Kong SAR, China
Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-Based Robots
Dec 21, 2024Xuancun Lu, Zhengxian Huang, Xinfeng Li +3Large Language Model-Based Robot Planning Adversarial Attacks on LLMs
Zhejiang University
DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization
Dec 10, 2024Xiaoqun Liu, Weiming Qi, Qiben YanAdversarial Robustness of Language Models LLM Jailbreak Attacks
Michigan State University, East Lansing, MI 48824, USA
Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge
Apr 8, 2024Weikai Lu, Ziqian Zeng, Jianwei Wang +5LLM Safety Alignment LLM Jailbreak Attacks
South China University of Technology, China · Pazhou Laboratory, China

Feb 27, 2026J/K move · Enter open · S save

Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking

Zhicheng Fang, Jingjie Zheng, Chenxu Fu, Wei Xu

Shanghai Qi Zhi Institute, Shanghai, China · University of Melbourne, Melbourne, Australia · Tsinghua University, Beijing, China

Jailbreak Attack Evaluation LLM Safety Benchmarks Language Model Safety Evaluation LLM Jailbreak Attacks

Open paper AskPDF arXiv

Jailbreak techniques for large language models (LLMs) evolve faster than benchmarks, making robustness estimates stale and difficult to compare across papers due to drift in datasets, harnesses, and judging protocols. We introduce JAILBREAK FOUNDRY (JBF), a system that addresses this gap via a multi-agent workflow to translate jailbreak papers into executable modules for immediate evaluation within a unified harness. JBF features three core components: (i) JBF-LIB for shared contracts and reusable utilities; (ii) JBF-FORGE for the multi-agent paper-to-module translation; and (iii) JBF-EVAL for standardizing evaluations. Across 30 reproduced attacks, JBF achieves high fidelity with a mean (reproduced-reported) attack success rate (ASR) deviation of +0.26 percentage points. By leveraging shared infrastructure, JBF reduces attack-specific implementation code by more than half relative to original repositories and achieves an 82.5% mean reused-code ratio. This system enables a standardized AdvBench evaluation of all 30 attacks across 10 victim models using a consistent GPT-4o judge. By automating both attack integration and standardized evaluation, JBF offers a scalable solution for creating living benchmarks that keep pace with the rapidly shifting security landscape.