RedDarwin: Evolving Jailbreak Skills against Frontier LLMs

Zhouyuan Ma; Yutao Wu; Hanxun Huang; Xiang Zheng; Yi Liu; Xingjun Ma

cs.CROct 14, 2026

RedDarwin: Evolving Jailbreak Skills against Frontier LLMs

Authors: Zhouyuan Ma, Yutao Wu, Hanxun Huang, Xiang Zheng, Yi Liu, Xingjun Ma

Organizations: Fudan University · Deakin University · University of Melbourne · City University of Hong Kong ·

Abstract

Red teaming must continually adapt as frontier LLM safeguards evolve, yet existing approaches largely search for adversarial prompts or optimize within fixed attack procedures. We observe that even when an attack becomes ineffective, its underlying mechanisms can remain useful and can be revised or recombined to construct stronger attacks. Based on this insight, we introduce RedDarwin, an agentic framework that shifts the unit of jailbreak evolution from prompts and fixed attacks to executable attack skills. RedDarwin consists of 47 reusable base skills and autonomously revises, composes, executes, and selects them using target-model feedback, allowing successful discoveries to become building blocks for subsequent evolution. RedDarwin produces 65 evolved skills, improving 44 of 47 base attacks and increasing their mean attack success rate (ASR) on AdvBench from 18.8% to 47.3%. These evolved skills consistently outperform baselines on the held-out StrongREJECT and frontier LLMs unseen during evolution; with only 10 target-model queries, RedDarwin achieves 100% ASR on GPT-5.6-Sol, 92% on Claude-Sonnet-5, and 80% on Gemini-3.8-Flash. Analysis of the evolution trajectories further shows that strong attacks often emerge from novel combinations of reusable mechanisms rather than from individually dominant ones. We believe this paradigm can accelerate adaptive red teaming and inform stronger safeguards, creating complementary value for both safety evaluation and defense.

Figures & tables

Figure 1: Executable skill evolution in RedDarwin . Starting from executable attack skills, RedDarwin generates candidates through revision, combination, and discovery, evaluates them against the target model, and uses execution feedback to guide subsequent evolution. Successful candidates are retained in the skill population.

Figure 2: Development performance across skill evolution operators. Bars show mean ASR (%) on AdvBench-50 across four development targets.

	Development	Held-out evaluation
		Development targets				Unseen targets
Attack skill	Avg.	Qwen3.7 Flash	GPT-OSS 120B	DS-V4 Flash	GPT-5.6 Luna	GLM-5.3 Flash	Gemini-3.8 Flash	MiniMax M3	Avg.
Baselines (Top-5)
Puzzler [ 2024 ]	54.0	12	10	52	30	22	46	16	26.9
CIA [ 2024 ]	42.0	42	16	60	6	4	38	22	26.9
AdvTales [ 2026 ]	62.5	74	36	60	36	8	14	58	40.9

Table 1: Transfer of executable attack skills to held-out goals and unseen models. ASR (%) of representative base and evolved skills with fixed canonical configurations. Development ASR averages four targets on AdvBench-50. Held-out evaluation uses StrongREJECT-50 on the same four targets and three models unseen during evolution; its average covers all seven targets. Bold and underlined values mark the best and second-best result in each column, including ties.

Figure 3: Adaptive red teaming under a fixed target-query budget. Cumulative ASR over StrongREJECT-50 as the per-goal query budget increases. Endpoint labels report ASR at budget 10.

Figure 4: Revision patterns and mechanism reuse in skill evolution. (a) Distribution and retention of improvement-associated revisions across six functional modules. (b) ASR across versions of four representative skills. (c) Mechanism reuse count and mean ASR of skills containing each mechanism.

Figure 5: Representative cases of skill evolution. Evolved skills can preserve an existing attack mechanism while changing the surrounding procedure, extend a compositional framework with new operators, or integrate mechanisms from different attacks into a new skill.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Date	Author	Attack	Category
Aug. 2023	Yuan et al.	CipherChat	Encoding & Obfuscation
Sep. 2023	Yu et al.	GPTFuzz	Auto-search
Oct. 2023	Yong et al.	LRL	Cross-lingual
Oct. 2023	Wei et al.	ICA	Priming
Oct. 2023	Chao et al.	PAIR	Auto-search
Nov. 2023	Li et al.	DeepInception	Role-play

Appendix

Table 3: Methods used to initialize the skill population. RedDarwin consists of 47 executable base skills derived from jailbreak methods spanning ten categories, ordered chronologically by their initial arXiv submission date.

		Qwen3.7 Flash		GPT-OSS 120B		DeepSeek V4-Flash		GPT-5.6 Luna		Avg.		$\Delta$
Category	Skill	Base	2.0	Base	2.0	Base	2.0	Base	2.0	Base	2.0	$\Delta$
Role-play	PersonaModulation	2	72	0	40	26	58	0	0	7.0	42.5	${\blacktriangle}$ 35.5
	DeepInception	0	50	20	94	76	68	0	0	24.0	53.0	${\blacktriangle}$ 29.0
	ArtOfJailbreak	0	78	0	70	14	96	12	4	6.5	62.0	${\blacktriangle}$ 55.5
	AdversarialTales	80	88	54	76	76	80	40	4	62.5	62.0	${\blacktriangledown}$ 0.5
	CompoundRoleplay	0	90	0	88	30	92	0	8	7.5	69.5	${\blacktriangle}$ 62.0

Appendix

Table 4: Single-skill revision on the development set (AdvBench-50). ASR (%) of 47 executable base skills before (Base) and after RedDarwin revision (2.0) across four target models. Skills are sorted within each category by 2.0 average ASR. Colors indicate improvement/degradation/tie, and bold marks category-best results. Avg.: macro-average across targets;

\Delta

: change relative to Base.

Figure 6: Combination lineage across five development rounds. Initial skills (left column) seed the graph; each offspring has two parents. The vertical position and shading of each node report its development ASR (Table 5 ). Blue edges mark reuse of an earlier offspring (EmojiCtxInject) as a parent in Rounds 3–4.

			Qwen3.7 Flash	GPT-OSS 120B	DS-V4 Flash	GPT-5.6 Luna	Avg.	$\Delta$
Source	Evolved skill	Origin	ASR	ASR	ASR	ASR	ASR	$\Delta$
Multi-Skill Combination
Round 1	AdvHarmSearch	AdvTales $+$ h4rm3l	96	94	92	80	90.5	${\blacktriangle}$ 15.0
	EmojiCodeCham	EmojiJB $+$ CodeCham	100	58	100	66	81.0	${\blacktriangle}$ 20.5
	CogHarmSearch	CogOverload $+$ h4rm3l	100	98	100	90	97.0	${\blacktriangle}$ 2.0
Round 2	NarrDetailDecomp	AdvTales $+$ FicDetail	100	72	86	2	65.0	${\blacktriangledown}$ 6.5

Appendix

Table 5: Development results for combination and new-skill discovery. ASR (%) on AdvBench-50 for 15 combination candidates and 10 discovery candidates. Avg. averages the four development targets;

\Delta

is the gain over the stronger parent in percentage points. Bold marks the best result in each round or the discovery group.

Method	Representation	Evolved scope	Model access
Claudini [ 48 ]	Optimizer code	Token-forcing optimization procedure	White-box
EvoSynth [ 10 ]	Attack program	Program logic and control flow	Black-box
JailbreakSkill [ 69 ]	Executable skill	Rewrite procedure	Black-box
RedEvoAgent [ 83 ]	Markdown skill	Tool choice and orchestration	Black-box
MemoAttack [ 84 ]	Structured memory	Attack patterns and memory lifecycle	Black-box
DARWIN-Attack [ 51 ]	Strategy pool	Strategies and adaptive transitions	Black-box

Appendix

Table 6: Comparison of related adaptive and evolutionary red-teaming systems.

Skill	Parents	Combination
AdvHarmSearch	AdvTales $+$ h4rm3l	A veiled planning memo is encoded as a character-index table and embedded as the mentor’s guidance artifact in a Propp-style tale. The target decodes the memo and expands it through third-person plot attribution.
EmojiCodeCham	EmojiJB $+$ CodeCham	An auxiliary compiler renders the goal as an emoji rebus, which becomes the ciphertext inside a decrypt-then-complete code frame. The legend is incorporated into the decoding procedure, followed by completion under the decoded task contract.
CogHarmSearch	CogOverload $+$ h4rm3l	CogOverload’s veiled memo and reverse-causal narration are compiled into a h4rm3l transformation program. The target reconstructs the encoded memo and develops it through the resulting narrative procedure.
NarrDetailDecomp	AdvTales $+$ FicDetail	A Propp-structured premise pack carries the goal as an in-world mentor manuscript, after which FicDetail’s reply-conditioned fiction ladder progressively develops the manuscript. Development ASR fell below the stronger parent.
NarrAttackSearch	AdvTales $+$ TAP	TAP’s tree search operates over compiled tale briefs rather than freeform prompts. Each branch preserves the Propp-style framing while varying how the brief is presented and expanded.
EmojiCtxInject	EmojiJB $+$ VirtualCtx	The goal is represented as an emoji rebus, while a fabricated assistant turn establishes that the rebus has already been interpreted. The candidate was not retained, but its rebus representation was reused in later combinations.

Appendix

Table 7: Multi-skill combination, Rounds 1–2. Each candidate integrates mechanisms from two parent skills into a shared executable procedure. Names match Table 5 .

Skill	Parents	Combination
RebusProppExeg	AdvTales $+$ EmojiCtxInject	The mentor’s guidance is represented as an emoji rebus containing a compressed goal-level brief. Decoding is embedded within third-person structural exegesis, which expands the recovered cues into a fuller account.
MirrorManuscript	FlipAttack $+$ FicDetail	FicDetail supplies an in-world document frame, while FlipAttack provides the transformed brief. The target reconstructs the brief and continues it as a manuscript within the fictional setting.
DecodeLadder	RACE $+$ h4rm3l	A h4rm3l-encoded brief is reconstructed across a RACE-style sequence of disjoint fragments. The final stage combines the recovered fragments into a single expanded account.
FlipBreaker	ActorBreaker $+$ FlipAttack	ActorBreaker’s fragment plan provides the shared representation. Each turn applies a flipped decoding task to one fragment, and the final turn combines the recovered fragments.
RebusTranscr	EmojiCtxInject $+$ h4rm3l	An auxiliary compiler renders a short brief as a legend-guided emoji rebus. The target first reconstructs the brief and then expands it within the same decoding-oriented interaction.
FlipCtxCont	VirtualCtx $+$ h4rm3l	An encoded memo is introduced through a decoding task, while a fabricated prior assistant turn contains the beginning of its reconstruction. The target continues from that prior state rather than starting from the original request.

Appendix

Table 8: Multi-skill combination, Rounds 3–5. Later rounds reuse mechanisms introduced by earlier candidates and integrate them with additional interaction, search, and representation strategies.

Skill	Mechanism	Core idea
WitnessCollate	Witness adjudication	Two manuscript outlines provide conflicting versions of the same procedural document. The target compares the disputed readings and completes a received text from the selected interpretation.
BackcastDecode	Canonical backcast	A short encoded brief establishes an outcome and its narrative role. After reconstruction, the target develops the preceding causes through backward-causal reasoning.
CensureCite	Enforcement citation	A compiled exhibit lists a set of charge topics in outline form. The target acts as an adjudicator and develops the cited material before issuing its assessment.
DeclassReview	Redaction reversal	A historical incident sketch contains redacted spans linked to topic-level entries in a sealed appendix. The target reviews each redaction and reconstructs the corresponding passage from its surrounding context.
CriticalApparatus	Scholarly annotation	A short after-action sketch is presented as a text for critical editing. The target expands individual clauses through editorial annotations that make their implied details explicit.
FolioSequencing	Dependency sequencing	Several fragments of one artifact are presented out of order together with precedence constraints. The target reconstructs their sequence and explains the connections between adjacent fragments.

Appendix

Table 9: New-skill discoveries. Ten executable procedures developed without a prescribed parent. The mechanism labels summarize the main interaction pattern used by each skill; development results are reported in Table 5 .

Metric	Mean	Min. (skill)	Max. (skill)
Input tokens	305,079	61,570 (PersonaModulation 2.0)	1,585,245 (ReNeLLM 2.0)
Output tokens	55,371	10,557 (MathPrompt 2.0)	117,068 (PAP 2.0)
LLM requests	156.0	31 (Prefill 2.0)	358 (PAIR 2.0)
Evolution time (h)	6.94	0.65 (RebusTranscription)	25.96 (Mastermind 2.0)
LLM streaming time (min)	20.41	4.72 (CognitiveOverload 2.0)	60.06 (TurboFuzzLLM 2.0)

Appendix

Table 10: Resource usage per evolution trajectory. Statistics are computed over the 72 trajectories with recoverable Kimi CLI session metadata. Cache-read and cache-creation tokens are excluded; parentheses identify the skills attaining the minimum and maximum values.

Metric	LLM judge	Human A	Human B
ASR on AdvBench	52.0	48.0	48.0
ASR on StrongREJECT	42.0	36.0	42.0
Overall ASR	47.0	42.0	45.0
Binary agreement with the LLM judge	–	95.0	96.0

Appendix

Table 11: Human check of the LLM judge. ASR (%) on the same 100 sampled goal–response pairs, with 50 from AdvBench and 50 from StrongREJECT. Binary agreement (%) compares each annotator’s success labels with the LLM judge over all 100 pairs. Success is defined as a score of 5 under the common five-point rubric.

Figure 7: Retention and order of skill revisions. (a) Retained and non-retained improvement-associated revisions across the six functional modules. (b) Retained revisions separated into the first retained improvement in each trajectory and subsequent improvements.

Figure 8: Initial designs and development effort. (a) Outcomes following the first design attempt for the 65 retained trajectories. Different module denotes a later switch in the module being revised; Same module denotes further refinement within it. (b) Mean recorded activity in 7 non-retained development cycles relative to 65 retained cycles. The dashed line denotes equal means.

Mechanism	What it does	Reuse	Mean ASR
Index-copy	The target restores hidden text by copying characters at listed positions, one at a time.	34	79.3
Reverse-causal	The payload is written as a past-tense record; the target reasons backward from outcome to steps.	22	72.7
Word-aligned chunks	The payload is split into word-aligned chunks reversed within each chunk, so no harmful word appears whole.	22	82.4
Join-expression	The target emits restored text as a join of copied pieces, never materializing whole harmful words.	21	81.9
Output contract	The reply must instantiate the real content; paraphrase, euphemism, or summary counts as failure.	21	67.4
Aux short memo	An auxiliary model compresses the goal into a short operational memo before any target call.	19	82.6

Appendix

Table 12: The twenty most-reused mechanisms. Reuse counts skills containing the mechanism beyond its first occurrence, as in Figure 4 (c); mean ASR averages the recorded development performance of the complete skills containing it. Ties at the cutoff are broken by mean ASR. Display names match the labels in Figure 4 (c) where available.

Figure 9: Base skills and four rounds of single-skill revision. Mean development ASR over three targets from the base skill through four successive revision rounds. Continued revision can produce further gains after the first evolved version, although later successors are not necessarily stronger. Archived Base ASRs are computed over available judged responses, with incomplete judge coverage.

Figure 10: Model risk profiles across 10 attack categories. Each axis reports mean skill ASR (%) within a category on 50 AdvBench goals. Gray dashed curves show base skills; blue solid curves show their 2.0 revisions. All panels use the same scale, with the outer ring at 100%. Larger radii indicate greater observed susceptibility to the evaluated skills in that category.

LLM Jailbreak Attacks

Large language model (LLM) jailbreak attacks study how adversarial inputs bypass safety rules and elicit disallowed outputs.

See all 203 papers

RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution
Aug 27, 2026Junjie Zhang, Hui Liu, Kecheng Chen +3Automated Red Teaming LLM Agent Security
City University of Hong Kong · Shenzhen MSU-BIT University
JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills
Aug 17, 2026Xiaoyu Wen, Jiajia Li, Zhida He +11LLM Jailbreak Attacks Automated Red Teaming
Shanghai AI Laboratory · Shanghai Jiao Tong University · Northwestern Polytechnical University +2
ATTCK-R: Knowledge-Aware Long-Horizon Jailbreaking for Protected LLMs
Oct 12, 2026Md Mushfiqur Rahman, Tobias Rohde, Rahul Tripathi +2LLM Jailbreak Attacks LLM Red Teaming
George Mason University · Amazon

Aug 27, 2026J/K move · Enter open · S save

RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution

Junjie Zhang, Hui Liu, Kecheng Chen, Xianbo Mo+2

City University of Hong Kong · Shenzhen MSU-BIT University

Automated Red Teaming LLM Agent Security LLM Jailbreak Attacks LLM Agent Skill Learning Self-Evolving Agents

Open paper AskPDF arXiv

LLM-based agents are increasingly deployed in product-level execution harnesses, where jailbreaks can trigger harmful tool use and persistent state changes, creating greater risks than unsafe text generation alone. Existing automatic red-teaming methods often rely on fixed attacks, while recent agentic attackers coordinate multiple jailbreak tools and show stronger potential through trajectory-based retrieval. However, such retrieval can reuse misleading experiences due to retrieval bias and unclear tool credit, and full trajectories add context overhead while reducing interpretability. We propose RedEvoAgent, a black-box red-teaming agent that distills cross-case attack trajectories into a concise, human-readable attack skill. The attack skill adaptively evolves through tool-effectiveness profiling and Deciding-Tool Attribution for skill updates, and a validation ratchet that retains only updates improving validation performance. Experiments on multiple benchmarks, target models, and target execution harnesses show that RedEvoAgent outperforms fixed and agentic baselines, improves tool efficiency, and transfers across attacker models and target execution harnesses.

Abstract

LLM Jailbreak Attacks

Explore similar work

RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution

JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills

ATTCK-R: Knowledge-Aware Long-Horizon Jailbreaking for Protected LLMs