Dynamic Deep Prompt Optimization for Defending Against Jailbreak Attacks on LLMs

Doniyorkhon Obidov; Honggang Yu; Xiaolong Guo; Kaichen Yang

cs.CRAug 10, 2026

Dynamic Deep Prompt Optimization for Defending Against Jailbreak Attacks on LLMs

Authors: Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo, Kaichen Yang

Organizations: Michigan Technological University · Miami University · Kansas State University

Abstract

Large Language Models (LLMs) demonstrate impressive capabilities across many applications but remain vulnerable to jailbreak attacks, which elicit harmful or unintended content. While model fine-tuning is an option for safety alignment, it is costly and prone to catastrophic forgetting. Prompt optimization has emerged as a promising alternative, yet existing prompt-based defenses typically rely on static modifications (e.g., fixed prefixes or suffixes) that cannot adapt to diverse and evolving attacks. We propose Dynamic Deep Prompt Optimization (DDPO), the first jailbreak defense based on deep prompt optimization. DDPO uses the target LLM's own intermediate layers as feature extractors to dynamically generate defensive embeddings via a lightweight multilayer perceptron. These tailored embeddings are then injected into a subsequent intermediate layer, enabling an input-dependent defense without modifying the LLM's weights. This design ensures high adaptability with minimal computational overhead. Experiments on a diverse set of models and attacks demonstrate that DDPO significantly outperforms static prompt optimization methods, particularly on weakly aligned models and when handling semantically ambiguous benign prompts, successfully distinguishing them from genuinely harmful requests.

Figures & tables

Figure 1: Prompt tuning strategies against jailbreak attacks (none update the target LLM’s weights). Attack tokens are in red and defensive tokens/embeddings in green. Existing methods use static hard or soft prompts. A naive dynamic approach requires a large auxiliary LLM and double input processing. DDPO instead uses the target LLM’s early‑layer features and injects dynamically generated defensive embeddings (via a two‑layer MLP, negligible compared to the LLM size) into the inputs of later layers, adding minimal overhead.

Figure 2: DDPO directly inserts a defensive embedding into an intermediate LLM layer, generated by a lightweight MLP using features from earlier layers. A masked placeholder preserves positional embeddings during earlier processing. The MLP is called once to produce the embedding, which is reused for output generation, minimizing overhead.

		Attack Success Rate (%) $\downarrow$										Utility (%) $\uparrow$
Model	Defense	PH	PEZ	UAT	GCG	AP	GBDA	GCG-M	PAIR	HC	Average	Benign	MMLU
Llama3	None	1.29	3.12	5.54	7.25	6.92	4.02	7.14	5.06	20.62	6.77	94.45	65.46
	PAT	0.00	2.17	2.61	4.35	4.78	3.48	5.65	4.78	23.48	5.70	75.80	42.81
	RPO	0.00	1.30	1.74	2.17	1.74	2.17	0.87	3.04	10.00	2.56	70.60	47.65
	DRO	0.00	1.74	1.74	1.30	0.87	1.30	0.00	0.87	6.96	1.64	92.40	47.15
	DDPO	2.61	1.30	0.00	0.43	0.00	0.43	0.43	0.87	0.87	0.77	97.80	63.53

Table 1: A comparison of defense performance for various models against multiple jailbreak attacks. The Attack Success Rate (ASR) is evaluated on 230 unseen prompts for each attack type. Benign utility is evaluated on 500 unseen prompts that are designed to seem harmful but are in fact benign. DDPO (ours) was trained on 50 examples of GCG and human-crafted prompts. PH denotes plain harmful prompts, AP denotes AutoPrompt, and HC denotes human-crafted jailbreak prompts.

	SDPO (Static)		DDPO (Dynamic)
Model	ASR $\downarrow$	BPR $\uparrow$	ASR $\downarrow$	BPR $\uparrow$
Llama3	12.32	98.60	0.77	97.80
Deepseek	22.90	86.80	0.39	94.60
Vicuna	47.20	96.00	0.68	95.80
Llama2	9.52	70.00	1.40	93.20
Openchat	79.32	99.00	0.97	95.60

Table 2: Ablation study comparing our dynamic method (DDPO) against a static variant (SDPO). ASR is averaged over 2,070 attack prompts (230 prompts for each of 9 attacks), while BPR is measured on 500 benign prompts. Both methods were trained on identical data and used the same intermediate layers. ASR and BPR values are given in (%).

Model	Metric	$K=1$	$K=5$	$K=10$
Llama3	ASR	0.77	1.50	0.72
	BPR	97.8	98.0	97.4
Deepseek	ASR	0.39	1.26	0.34
	BPR	94.6	96.2	96.4
Vicuna	ASR	0.68	1.35	0.97
	BPR	95.8	95.4	96.6

Table 3: Ablation study on the number of defensive embeddings (

K

) across several models. The Attack Success Rate (ASR) is averaged over 2,070 attack prompts (230 for each of 9 attack types), while the Benign Pass Rate (BPR) is measured on 500 benign prompts. The default DDPO configuration uses

K=1

Figure 3: The cosine similarity between the hidden state representations of harmful and benign prompts across the layers of several LLMs. A lower similarity score indicates a layer can better distinguish between the two prompt types. For each model, the layer with the minimum cosine similarity is marked with a $\times$ .

Jailbreak Defense

Jailbreak defense studies how to stop adversarial prompts from eliciting harmful outputs while preserving helpful behavior.

See all 60 papers

Re-Triggering Safeguards within LLMs for Jailbreak Detection
May 11, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +2LLM Jailbreak Attacks Jailbreak Detection
1Xidian University · 2Xi’an Jiaotong University
DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization
Dec 10, 2024Xiaoqun Liu, Weiming Qi, Qiben YanAdversarial Robustness of Language Models LLM Jailbreak Attacks
Michigan State University, East Lansing, MI 48824, USA
Analysis of LLMs Against Prompt Injection and Jailbreak Attacks
Feb 24, 2026Piyush Jaiswal, Aaditya Pratap, Shreyansh Saraswati +2Prompt Injection Attacks on LLMs Language Model Safety Evaluation
NIT Trichy, Tiruchirappalli, India · NCE Chandi, Nalanda, India · Bishop Cotton Boys’ School, Bengaluru, India +2

May 11, 2026J/K move · Enter open · S save

Re-Triggering Safeguards within LLMs for Jailbreak Detection

Zheng Lin, Zhenxing Niu, Haoxuan Ji, Yuzhe Huang+1

1Xidian University · 2Xi’an Jiaotong University

LLM Jailbreak Attacks Jailbreak Detection Jailbreak Defense

Open paper AskPDF arXiv

This paper proposes a jailbreaking prompt detection method for large language models (LLMs) to defend against jailbreak attacks. Although recent LLMs are equipped with built-in safeguards, it remains possible to craft jailbreaking prompts that bypass them. We argue that such jailbreaking prompts are inherently fragile, and thus introduce an embedding disruption method to re-activate the safeguards within LLMs. Unlike previous defense methods that aim to serve as standalone solutions, our approach instead cooperates with the LLM's internal defense mechanisms by re-triggering them. Moreover, through extensive analysis, we gain a comprehensive understanding of the disruption effects and develop an efficient search algorithm to identify appropriate disruptions for effective jailbreak detection. Extensive experiments demonstrate that our approach effectively defends against state-of-the-art jailbreak attacks in white-box and black-box settings, and remains robust even against adaptive attacks.

Abstract

Jailbreak Defense

Explore similar work

Re-Triggering Safeguards within LLMs for Jailbreak Detection

DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization

Analysis of LLMs Against Prompt Injection and Jailbreak Attacks