Large Language Model (LLM) agents are vulnerable to adversarial inputs that induce attacker-specified malicious behaviors, commonly referred to as jailbreaking. Existing optimization-based jailbreak attacks primarily employ first-order (FO) methods that leverage gradient information or zeroth-order (ZO) methods that rely on function-value feedback. However, their complementarity and optimization ordering remain underexplored. We systematically investigate four strategies: pure ZO, pure FO, ZO->FO, and FO->ZO, and develop a ZO-FO framework that explores adversarial prefixes through ZO optimization and refines their corresponding suffixes through FO optimization. To coordinate the two processes, we introduce Pair-aware optimization, which retains multiple prefix candidates, conditionally refines their suffixes and selects among the resulting prefix-suffix pairs. We further investigate agent jailbreaking from a more general stochastic-generation perspective, where the same adversarial input can induce diverse reasoning trajectories and actions that a single greedy output cannot capture. Accordingly, we adopt consistent stochastic decoding during both optimization and evaluation. Experiments across malicious-instruction and malicious-environment benchmarks demonstrate that hybrid optimization improves attack effectiveness over single-optimizer methods, while optimization order substantially influences performance.
Figures & tables
Figure 1: Four optimization organizations for prefix-suffix adversarial strings. (a) Pure ZO uses CEM to optimize both segments. Candidates are scored by target cross-entropy; per-position token frequencies from selected low-loss candidates guide cross-entropy fitting and smoothing of the sampling distribution, without model gradients. (b) Pure FO selects non-overlapping target locations, generates top-k token substitutions using target-loss gradients, and retains the candidate with the lowest screening loss. (c, d) apply ZO-to-FO and FO-to-ZO, respectively, updating prefix before suffix while fixing the other segment. Blue/orange denote prefix/suffix.
Figure 2: Mechanistic illustration of UDora and Pair-aware on an InjecAgent example. Following an attacker-modified tool response, UDora deterministically selects target-insertion sites, whereas Pair-aware refines suffixes conditioned on candidate prefixes and selects complete pairs by success, then loss.
Model
Method
Attack Categories
Avg. ASR
Detailed
Simple
+Hint
-Hint
+Hint
-Hint
Llama-3.1- 8B-Instruct
GCG
38.64%
38.64%
40.91%
31.82%
37.50%
UDora (Sequential)
59.09%
50.00%
50.00%
43.18%
50.57%
UDora (Joint)
43.18%
38.64%
47.73%
50.00%
44.89%
REINFORCE-GCG
81.82%
70.45%
75.00%
72.73%
75.00%
Table 1: Attack Success Rate (%) on AgentHarm under different attack categories.
Model
Method
Attack Categories
Avg. ASR
Direct Harm
Data Stealing
Llama-3.1- 8B-Instruct
GCG
34%
28%
31%
UDora (Sequential)
48%
42%
45%
UDora (Joint)
44%
52%
48%
REINFORCE-GCG
44%
48%
46%
Pair-aware
100%
100%
100%
Table 2: Attack Success Rate (%) on InjecAgent across different attack categories.
Model
Method
Attack Categories
Avg. ASR
Price Mismatch
Attribute Mismatch
Category Mismatch
All Mismatch
Llama-3.1- 8B-Instruct
GCG
0.00%
0.00%
0.00%
0.00%
0.00%
UDora (Sequential)
13.33%
13.33%
6.67%
0.00%
8.33%
UDora (Joint)
6.67%
13.33%
13.33%
0.00%
8.33%
REINFORCE-GCG
0.00%
6.67%
0.00
0.00%
1.67%
Pair-aware
20.00%
60.00%
20.00%
6.67%
26.67%
Table 3: Attack Success Rate (%) on WebShop under different attack categories.
Model
Optimization Mode
Attack Categories
Avg. ASR
Detailed
Simple
+Hint
-Hint
+Hint
-Hint
Llama-3.1- 8B-Instruct
FO
72.73%
45.45%
54.55%
63.64%
59.09%
ZO
54.55%
36.36%
54.55%
54.55%
50.00%
FO → ZO
68.18%
59.09%
61.36%
52.27%
60.23%
ZO → FO
86.36%
90.91%
84.09%
95.45%
89.20%
Table 4: Ablation study of different optimization modes on AgentHarm.
Model
Optimization Mode
Attack Categories
Avg. ASR
Direct Harm
Data Stealing
Llama-3.1- 8B-Instruct
FO
54%
50%
52%
ZO
60%
54%
57%
FO → ZO
54%
56%
55%
ZO → FO
100%
100%
100%
Qwen3- 8B
FO
2%
6%
4%
Table 5: Ablation study of different optimization modes on InjecAgent.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Pair-aware optimization on an AgentHarm example. (1) An adversarial string, initialized with repeated x tokens and split into a 10-token prefix and a 15-token suffix, is appended to the user request to elicit get_latest_emails . (2) The initial response refuses the request without emitting the target function. (3) Four prefix–suffix pairs are evaluated in the first cycle. None hits the target, so the lowest-loss Pair 2 is retained. (4) In the subsequent cycle, candidate prefixes are renewed, and each prefix is held fixed while its corresponding suffix is refined from the same incumbent suffix. Complete pairs are compared using target-name hits and common loss, prioritizing success. Pair 3 is selected despite a higher loss (4.34375) than unsuccessful Pair 1 (3.56250), and its prefix and refined suffix are adopted together. The dashed arrow from (4) back to (3) indicates repetition of candidate renewal, conditional suffix refinement, and complete-pair selection until success or budget exhaustion. Blue and orange distinguish prefix and suffix segments, locks mark fixed prefixes, and green highlights the successful pair. (5) The final response contains the target-function text, indicating attack successful.
Figure 4: Pair-aware optimization on an InjecAgent example. (1) A traffic-control instruction and an adversarial string initialized with repeated x tokens are injected into a calendar tool response to target TrafficControlManageTrafficLightState . The string comprises a 10-token prefix and a 15-token suffix. (2) The initial response answers the calendar query without emitting the target action. (3) Four prefixes are screened by loss with the incumbent suffix; red crosses indicate unconfirmed success. (4) Each prefix is fixed while suffix refinement starts from the same incumbent suffix. Pairs are ranked by target-name hits, then loss. Pair 2 achieves a hit with loss 4.02387; its prefix and unchanged suffix are adopted together. The dashed loop repeats optimization until success or budget exhaustion. Blue and orange denote prefix and suffix, locks mark fixed prefixes, and green highlights the selected pair. (5) The output contains the target-action text, demonstrating it hit the target function and attack successful.
Figure 5: Pair-aware optimization on a WebShop example. (1) The user seeks two mesh laundry bags under $50. The attacker appends an adversarial string, initialized with repeated x tokens and split into a 10-token prefix and a 15-token suffix, to an unrelated toothpaste-cap listing, targeting click[B09NQ5ZCPH]. (2) The initial response does not emit the target click. (3) Four prefix–suffix pairs are screened by target loss with the current suffix fixed. Red crosses mark unconfirmed success; the blue outline tracks the eventual winner, Pair 4, rather than the initial lowest-loss candidate. All four pairs proceed to refinement. (4) Each prefix is fixed while its suffix is independently refined from the same incumbent suffix. Complete pairs are compared by target-name hits and common loss, prioritizing success and then lower loss. Pair 4 is selected with a target-text hit and loss 4.66323, and its prefix and refined suffix are adopted together. The dashed arrow denotes repetition until success or budget exhaustion. Blue and orange distinguish prefix and suffix segments, locks mark fixed prefixes, and green highlights the selected pair. (5) The generated response contains the target-click text.
Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.
Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can adapt but often relies on low-level mutations that require many target queries. We propose JailbreakOPT, a tool-assisted framework for improving iterative single-turn jailbreak prompt optimization. JailbreakOPT organizes diverse atomic jailbreak prompts into an attack tool library and composes them through a unified intra-episode optimization abstraction to generate stronger standalone attack prompts. To reuse experience across attack episodes, JailbreakOPT further frames tool selection as a contextual bandit problem and applies contextual Thompson sampling to guide exploration and exploitation based on past outcomes. Experiments across multiple target LLMs and attack goals show that JailbreakOPT improves attack success rate (ASR) while reducing the number of attacks until success (No.A) compared with atomic single-turn attacks and existing iterative optimization baselines. This paper may contain offensive or harmful content.
Ge Shi, Jun Yin, Donglin Xie +3
University of California, Davis · The Renmin University of China · Independent Researcher +3
Large language models (LLMs) have achieved remarkable success across diverse applications but remain vulnerable to jailbreak attacks, where attackers craft prompts that bypass safety alignment and elicit unsafe responses. Among existing approaches, optimization-based attacks have shown strong effectiveness, yet current methods often suffer from frequent refusals, pseudo-harmful outputs, and inefficient token-level updates. In this work, we propose TAO-Attack, a new optimization-based jailbreak method. TAO-Attack employs a two-stage loss function: the first stage suppresses refusals to ensure the model continues harmful prefixes, while the second stage penalizes pseudo-harmful outputs and encourages the model toward more harmful completions. In addition, we design a direction-priority token optimization (DPTO) strategy that improves efficiency by aligning candidates with the gradient direction before considering update magnitude. Extensive experiments on multiple LLMs demonstrate that TAO-Attack consistently outperforms state-of-the-art methods, achieving higher attack success rates and even reaching 100% in certain scenarios.
Despite the significant efforts devoted to aligning large language models (LLMs) with human values and ensuring safe deployment, recent work has revealed that LLMs remain vulnerable to adversarial jailbreak attacks that can bypass safety guardrails and elicit harmful responses. Many defense methods are proposed to detect jailbreaks but they are limited in their effectiveness to counter wide-range optimization-based jailbreak mechanisms that can yield highly fluency-optimized or harmful semantic obfuscated prompts. To tackle this challenge, we propose a unified detection framework SAFEGuard which incorporates a hybrid fluency measurement based on cross-layer distribution distance and perplexity, and the analysis of harmful semantics through gradient matching. Our method is grounded in a paramount observation: high fluency prompts maintain their malicious intention close to harmful prompts while harmful semantic obfuscated prompts often inject gibberish token sequences. Our evaluation demonstrates that SAFEGuard consistently outperforms state-of-the-art baselines and achieves significant improvement in accuracy across different optimization-based jailbreaks. This underscores the effectiveness of SAFEGuard against evolving jailbreak attacks.
Quoc Viet Vo, Trung Le, Damith C. Ranasinghe +1
Australian Institute for Machine Learning · Adelaide University · Monash University