Organizations: Shanghai Qi Zhi Institute · East China Normal University · The Chinese University of Hong Kong · The Hong Kong University of Science and Technology (Guangzhou) · Georgia Institute of Technology · Tsinghua University
As jailbreak attacks against large language models (LLMs) become increasingly diverse, red-teaming has come to rely on large libraries of jailbreak prompts and attack strategies to evaluate the robustness of victim models and their defense settings. However, attack effectiveness is highly condition-dependent: an attack that succeeds under one victim-defense setting may fail under another, while exhaustively evaluating the full attack library for each setting incurs substantial token and monetary costs. This motivates studying how jailbreak attacks should be prioritized under a limited attempt budget for efficient robustness evaluation. In this work, we formalize this problem as budgeted attack scheduling over a fixed attack library and investigate three complementary signals under a fixed victim-defense condition: (1) static historical attack success rate (ASR) priors; (2) semantic guidance from LLMs; and (3) online feedback from earlier queries. Building on these signals, we design four scheduling strategies and evaluate them across three victim models and three defense settings on GPT-5.2. Extensive experiments clearly characterize the conditions under which each signal is most effective. Historical ASR provides a strong initialization, but its reliability degrades under defense-induced distribution shifts. Semantic guidance consistently improves or matches static prior-based scheduling in the cold-start regime, while online feedback becomes particularly valuable when transferred priors become stale. Semantic guidance and online feedback can also partially compensate for missing historical priors. Taken together, these findings characterize budgeted red-teaming as an attack scheduling problem in which historical priors, semantic guidance, and online feedback play complementary roles.
Figures & tables
Figure 1: Workflow of budgeted attack scheduling. Given an attack library and an evaluation setting, the scheduler selects an ordered sequence of attacks under an attempt budget K . The execution module then applies these attacks sequentially until a jailbreak succeeds or the budget is exhausted, and reports budgeted attack performance, cost-related metrics, and scheduling-related analysis.
Policy
Historical ASR Prior
Semantic Guidance
Online Feedback
Scheduler Overhead
ASR Sort
✓
✗
✗
None
LLM-guided Sort
✓
✓
✗
LLM calls
Thompson Sampling
✓
✗
✓
None
LLM-guided Online
✓
✓
✓
LLM calls
Table 1: Attack scheduling policies considered in the main evaluation.
Victim Models
Scheduling Policies
Success Metrics
Token Usage
Success@1
Success@3
Success@5
Avg. ATA
Attack
Schedule
Cache Hit Rate
DeepSeek-V4-Pro
ASR Sort
92.00
98.00
99.00
1.14
1366
–
–
LLM-guided Sort
92.00
99.00
99.00
1.12
1241
11242
89.13
Thompson Sampling
86.20
97.60
99.00
1.22
813
–
–
LLM-guided Online
92.00
99.00
99.00
1.14
1446
11242
89.13
Gemma-4-26B
ASR Sort
59.00
84.00
90.00
1.94
7979
–
–
Table 2: Attack performance and token usage across scheduling policies and victim models.
Figure 2: Defense-induced shifts in query distribution and attack ordering. Top: Query difficulty distributions, where difficulty is defined as one minus the attack success rate across all attacks; higher values indicate harder queries. Bottom: Attack-level changes in effectiveness, with attacks ordered from top to bottom by their ASR under the current test condition.
Victim Models
Scheduling Policies
With Historical Prior
Without Historical Prior
Prior Gain (pp)
Success@1
Success@3
Success@5
Success@1
Success@3
Success@5
Δ @1
Δ @3
Δ @5
Avg. Δ
DeepSeek-V4-Pro
ASR Sort
92.00
98.00
99.00
15.00
72.00
83.00
+77.00
+26.00
+16.00
+39.67
LLM-guided Sort
92.00
99.00
99.00
47.00
93.00
96.00
+45.00
+6.00
+3.00
+18.00
Thompson Sampling
86.20
97.60
99.00
78.00
93.00
96.00
+8.20
+4.60
+3.00
+5.27
LLM-guided Online
92.00
99.00
99.00
47.00
93.00
97.00
+45.00
+6.00
+2.00
+17.67
Gemma-4-26B
ASR Sort
59.00
84.00
90.00
18.00
62.00
74.00
+41.00
+22.00
+16.00
+26.33
Table 4: Effect of historical priors on attack scheduling under the no-defense setting. Success@1/3/5 compares each scheduling policy with and without access to historical attack-success priors. Δ denotes the corresponding improvement in pp, and Avg. Δ reports the mean gain across the three attempt budgets.
Figure 3: Order sensitivity of online feedback across native and defended settings. Success@ K is evaluated over five query orders for Thompson Sampling and LLM-guided Online. Hollow markers show individual orders, solid markers show the mean, and vertical stems indicate the min–max range; the inset summarizes Success@5 variation across orders.
As large language models are deployed behind increasingly sophisticated safety mechanisms, evaluating their robustness under realistic adversarial conditions becomes increasingly difficult and critical. Existing jailbreaking evaluations often rely on static prompts, limited interaction horizons, or lightly protected systems, failing to reflect protected deployment settings with prompt-based safety protections. We show that the introduction of a prompt-based safety protection reduces the average attack success rate of a strong multi-turn baseline from 0.896 to 0.608 (32.1% relative), demonstrating that unprotected evaluations substantially overestimate real-world vulnerabilities. We propose Adaptive Traceback and Tactic Curation for Knowledge-based Red-teaming (ATTCK-R), a planning-driven multi-turn red teaming framework designed for protected black-box systems. ATTCK-R models jailbreaking as a long-horizon trajectory optimization problem using a structured belief state, introduces optimization-based tactic curation to abstract reusable strategies from prior successes and failures, and employs informed traceback to resume attacks from strategically meaningful states rather than restarting. Evaluated across 9 widely used LLMs configured with a single prompt-based safety protection layer, ATTCK-R consistently outperforms strong recent multi-turn jailbreaking frameworks. Ablation results confirm that long-horizon planning and tactic curation improve attack effectiveness, while adaptive traceback improves efficiency through structured recovery from conversational failure.
Large language models remain vulnerable to jailbreaks, and automated red teaming is the standard way to find jailbreaks in large language models at scale. Current methods either draw more samples at test time through search, rewriting, and tree expansion, or train a stronger attacker offline with reinforcement learning. Both share a limitation: once an attack on a specific target behavior begins, the attacker's weights are frozen. Any signal it gathers about the behavior stays in its context window and is discarded afterward. The attacker never adapts its proposal distribution mid-attack, so success depends almost entirely on the sampling budget, and under a budget affordable at scale, many behaviors remain unbroken. We propose Red-TTT, which updates the attacker's parameters during the attack on each behavior. At each round, the attacker samples a group of candidates, scores them against the victim's replies, and takes a policy-gradient step before drawing the next group, so what it discovers about the current victim is consolidated into weights rather than accumulated as context. We also adapt the training objective to red teaming, where success is judged by the single best sample rather than the average. Red-TTT requires only sampling access to the victim and integrates into existing attack pipelines with no other changes. Against the Best-of-N baseline, Red-TTT raises attack success rate from 55.9% to 72.4% on average at a budget of 120 samples, improving over the baseline in every configuration and cracking many behaviors previous method cannot. The code is available at https://github.com/SaFo-Lab/Red-TTT
Tongyan Hu, Hao Li, Xiaogeng Liu +8
Johns Hopkins University · National University of Singapore · Washington University in St. Louis +2
Red-teaming is becoming a central part of large language model (LLM) safety evaluation, yet current practice still relies heavily on expert-written prompts or fixed benchmark suites. This creates a gap between what is easy to test and what deployed models can actually do: failures may be rare, context-sensitive, and distributed across many threat categories. We study automated red-teaming as a constrained adversarial search problem and introduce a learning-driven framework that couples category-aware attack generation with hierarchical vulnerability detection. The method starts from curated safety seeds, expands them through meta-prompt-guided and evolutionary search, and scores the resulting prompt--response pairs with lexical, semantic, and behavioral detectors. Across six threat categories on GPT-OSS-20B, the framework discovers 47 validated vulnerabilities, including 21 high-severity cases and 12 novel attack patterns. Under matched query budgets, it achieves a 3.9× higher discovery rate than manual expert red-teaming while maintaining 89% detection accuracy and full category coverage. Ablations show that the gains do not come from more prompts alone: diversity constraints prevent template collapse, coverage constraints prevent category blind spots, and semantic detection recovers failures missed by lexical rules. These results suggest that red-teaming can be made more scalable and reproducible when treated as adaptive search rather than as a static checklist.
Zhang Wei, Hanxuan Chen, Peilu Hu +19
Independent Researcher · Hunan University · Shenzhen Kaihong Digital Industry Development Co., Ltd. +12