Jailbreak Attacks

Follow this topic Ask about this topic

Jailbreak attacks study how to make AI systems produce harmful or otherwise restricted outputs despite safety safeguards, using methods such as adversarial prompt optimization, encoded instructions, multi-turn persuasion, and search over attack strategies. Research now tests these methods not only on text-only language models but also on image, audio, and video systems, tool-using agents, and robots, with benchmarks including AdvBench, HarmBench, and JailbreakBench. Defenses include input classifiers, tracking intent across turns or tools, and interventions in model activations, but they must preserve useful responses to benign requests. A central challenge is that measured success depends on attack budgets, prompt variants, and evaluation judges, while defenses may fail when harmful intent is disguised by style, language, modality, or context.

Latest papers

76 papers11 papers in the last four weeks, up 57% on the four weeks before. 0.1% of all new papers.

Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation
Apr 9, 2026Wenpeng Xing, Moran Fang, Guangtai Wang +2Adversarial Attacks on LLMs Jailbreak Attacks
Zhejiang University · Binjiang Institute of Zhejiang University
JailWAM: Jailbreaking World Action Models in Robot Control
Apr 7, 2026Hanqing Liu, Songping Wang, Jiahuan Long +9Jailbreak Attacks Robot Safety
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China · Defense Innovation Institute, Chinese Academy of Military Science, Beijing, China · PR Lab, Nanjing University, Suzhou, China
Generalization Limits of Reinforcement Learning Alignment
Apr 3, 2026Haruhi Shida, Koo Imai, Keigo KansaLLM Alignment Adversarial Attacks on LLMs
Aladdin Security Inc.
Evolving Jailbreaks: Automated Multi-Objective Long-Tail Attacks on Large Language Models
Mar 20, 2026Wenjing Hong, Zhonghua Rong, Li Wang +5Jailbreak Attacks LLM Jailbreak Attacks
School of Artificial Intelligence Shenzhen Univeristy, China · Brain-Inspired Technology Co.,Ltd, China · Department of Computer Science and Engineering Southern University of Science and Technology, China +1
Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs
Mar 14, 2026Zijian Ling, Pingyi Hu, Xiuyong Gao +6Adversarial Attacks LLM Security
School of Cyber Science and Engineering, Huazhong University of Science and Technology · State Key Laboratory of Internet Architecture, Tsinghua University · Hubei Key Laboratory of Distributed System Security, Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology +1
Accelerating Suffix Jailbreak attacks with Prefix-Shared KV-cache
Mar 12, 2026Xinhai Wang, Shaopeng Fu, Shu Yang +3Jailbreak Attacks LLM Jailbreak Attacks
King Abdullah University of Science and Technology (KAUST) · Zhejiang University
Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models
Mar 11, 2026Xiangwen Wang, Ananth Balashankar, Varun ChandrasekaranJailbreak Attacks LLM Jailbreak Attacks
University of Illinois Urbana-Champaign · Google DeepMind
Efficient Refusal Ablation in LLM through Optimal Transport
Mar 4, 2026Geraldin Nanfack, Eugene Belilovsky, Elvis DohmatobAdversarial Attacks on LLMs Jailbreak Attacks
Concordia University, Montreal, Canada · Mila – Quebec AI Institute, Montreal, Canada
TAO-Attack: Toward Advanced Optimization-Based Jailbreak Attacks for Large Language Models
Mar 3, 2026Zhi Xu, Jiaqi Li, Xiaotong Zhang +2Adversarial Attacks on LLMs Jailbreak Attacks
Dalian University of Technology, Dalian, China
MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models
Mar 3, 2026Zhongxi Wang, Yueqian Lin, Jingyang Zhang +3LLM Safety Benchmarks LLM Safety
1Duke University · 2Virtue AI
Ignore All Previous Instructions: Jailbreaking as a de-escalatory peace building practise to resist LLM social media bots
Mar 2, 2026Huw Day, Adrianna Jezierska, Jessica WoodgateJailbreak Attacks Computational Social Science
School of Computer Science, University of Bristol
Boundary Point Jailbreaking of Black-Box LLMs
Feb 16, 2026Xander Davies, Giorgi Giglemiani, Edmund Lau +3Adversarial Attacks on LLMs Jailbreak Attacks
UK AI Security Institute · OATML, University of Oxford
When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Models
Feb 10, 2026Jiacheng Hou, Yining Sun, Ruochong Jin +4Jailbreak Attacks Multimodal Jailbreak Attacks
Tsinghua University, China · Pengcheng Laboratory, Shenzhen, China · Central South University, Changsha, China.
NonTextual Target Attack
Oct 3, 2025Xinzhe Huang, Wenjing Hu, Tianhang Zheng +6Adversarial Prompt Generation Adversarial Attacks on LLMs
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou 310007, China · Hangzhou High-Tech Zone, Binjiang Institute of Blockchain and Data Security, Hangzhou 310053, China · School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing 210094, China +3
HauntAttack: When Attack Follows Reasoning as a Shadow
Jun 8, 2025Jingyuan Ma, Rui Li, Zheng Li +4Adversarial Attacks on LLMs Jailbreak Attacks
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · StepFun · Department of Computing, The Hong Kong Polytechnic University +1
The Linear Geometry of Interpretable Tokens: Jailbreaking Attacks and Defenses for Unlearned Diffusion Models
Apr 30, 2025Siyi Chen, Yimeng Zhang, Sijia Liu +1Jailbreak Attacks Diffusion Model Unlearning
University of Michigan · Michigan State University

Apr 9, 2026J/K move · Enter open · S save

Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation

Wenpeng Xing, Moran Fang, Guangtai Wang, Changting Lin+1

Zhejiang University · Binjiang Institute of Zhejiang University

Adversarial Attacks on LLMs Jailbreak Attacks LLM Guardrails LLM Safety Causal Interventions in Language Models

Open paper AskPDF arXiv

While Large Language Models (LLMs) have achieved remarkable performance, they remain vulnerable to jailbreak attacks that circumvent safety constraints. Existing strategies, ranging from heuristic prompt engineering to computationally intensive optimization, often face significant trade-offs between effectiveness and efficiency. In this work, we propose Contextual Representation Ablation (CRA), a novel inference-time intervention framework designed to dynamically silence model guardrails. Predicated on the geometric insight that refusal behaviors are mediated by specific low-rank subspaces within the model's hidden states, CRA identifies and suppresses these refusal-inducing activation patterns during decoding without requiring expensive parameter updates or training. Empirical evaluation across multiple safety-aligned open-source LLMs demonstrates that CRA significantly outperforms baselines. These results expose the intrinsic fragility of current alignment mechanisms, revealing that safety constraints can be surgically ablated from internal representations, and underscore the urgent need for more robust defenses that secure the model's latent space.