Automatic red teaming increasingly relies on large language models (LLMs) to iteratively diagnose failed attacks and propose improved adversarial prompts or injections. Although prior work strengthens this loop through prompt engineering and search-harness design, the mechanisms underlying its two core roles, criticism and attack proposal, has not been explored. To further investigate the core roles and therefore improve red teaming effectiveness, we disentangle two design choices: whether criticism is generated by the target session itself (self-criticism) or by a separately instantiated critic (external criticism), and whether that criticism is translated into the next attack by the target session (self-proposal) or by a separate attacker (external proposal). Through our analysis, we find that self-criticism is more effective because it is grounded in the target's live execution context, including observations, constraints, and available reasoning; removing reasoning reduces downstream attack success by 6%-17%. In contrast, self-proposal is substantially less effective because awareness of the adversarial objective induces explicit refusal and non-refusal sandbagging. These findings motivate a role-separated red-teaming approach: the target session diagnoses why an attack failed, while a separately instantiated attacker generates the next proposal. On DTAP-BENCH, spanning Telecom, Workflow, CRM, and Code domains, under both direct misuse and indirect threat models, as well as multiple agent frameworks, this configuration achieves 63.2% overall attack success rate (ASR), much higher than baselines. Our results show that effective automatic red teaming depends not only on model capability and search strategy, but also on how execution context and adversarial responsibility are distributed across model sessions.
Figures & tables
Figure 1: Self-improved red teaming based on role-sepration. The target session uses its live execution context to diagnose why an attack failed. The diagnosis is then transferred to a separate attacker that generates the next adversarial candidate. This preserves execution-grounded criticism while avoiding refusal or sandbagging that can arise when the target itself is asked to operationalize the attack.
Figure 2: Self-criticism consistently outperforms external criticism when proposal is held external. Framework- and domain-level comparisons are shown for direct misuse and indirect prompt injection. Values above paired bars show the absolute ASR improvement from self-criticism.
Figure 3: Reasoning in the execution context makes self-criticism more actionable. Proposal remains external while we compare self-criticism from the full available context with criticism generated after reasoning is removed. Values above paired bars are absolute ASR improvements.
Figure 4: Self-proposal underperforms external proposal when criticism is held as self-criticism. Framework- and domain-level comparisons are shown for direct misuse and indirect prompt injection.
Figure 5: Self-proposal substantially increases explicit refusal. The remaining ASR gap in low-refusal settings suggests an additional non-refusal failure mode consistent with sandbagging; framework- and domain-level refusal breakdowns are in Appendix C.3 .
Figure 6: Task-level refusal varies substantially across frameworks and domains. The three criticism–proposal configurations are compared at the framework and domain levels.
Figure 7: Self-criticism with external proposal achieves the highest ASR among the three compared configurations. The left panel reports direct, indirect, and combined results; the right panel reports combined ASR by fully evaluated framework.
Backend
Threat
External criticism External proposal
Self-criticism External proposal
Self-criticism Self-proposal
GPT-5.5 OpenAI SDK
Direct
56.0
63.0
59.3
Indirect
37.8
42.8
36.2
GPT-5.5 OpenClaw
Direct
57.4
64.7
62.3
Indirect
41.3
52.6
42.0
Table 1: GPT-5.5 transfer results (ASR) on the selected red-teaming subset. Self-criticism with external proposal achieves the highest ASR across both backends and both threat models, indicating that the role-separation advantage transfers to GPT-5.5.
Automated methods for red teaming LLMs are an important tool to identify LLM vulnerabilities that may not be covered in static benchmarks, allowing for more thorough probing. They can also adapt to each specific LLM to discover weaknesses unique to it. Most current automated red teaming methods are intended for tackling safety and content moderation. Thus, they make use of content safety models as evaluators and optimize for circumventing them, and as such, have not been tested with other adversarial intents not typically captured by these. We propose a pipeline for training a red teaming model that can generalize to arbitrary adversarial goals, including objectives it has not been directly trained on, and that does not depend on the existence of a pre-existing evaluator available at training time. We demonstrate that finetuning small models, such as Qwen3-8B, using this pipeline results in a substantial improvement in their ability to generate attacks for both in and out of domain adversarial goals.
Aishwarya Padmakumar, Leon Derczynski, Traian Rebedea +1
Automated red-team attacks and blue-team defenses for large language models (LLMs) are advancing quickly. However, attackers and defenders are built and tested in isolation, and the resulting scores are hard to trust. To tackle this, we present ACEA (Adversarial Co-Evolution Arena), a platform that connects a pluggable red-team adapter and a pluggable blue-team adapter to a shared target LLM and scores their attack and defense rates with an LLM judge. ACEA contributes four components. First, a pluggable, model-agnostic arena. Any red or blue project connects over a minimal HTTP protocol, which we call the ACEA Standard Adapter Protocol (ASAP). It can be written in any language, and a project that exposes nothing but the protocol is a full participant. Second, an evaluation methodology built for adversarial rounds. Seeding the target with canonical secrets gives verifiable ground truth that separates real leakage from hallucination. We also send each attack to the target even when the defense blocks it, which measures the attack's raw potency independently of whether it was stopped. Together these yield a per-round decomposition of attack strength and defense effectiveness. Third, a real-time, game-style visualization with a detailed end-of-battle report that localizes each failure. The evaluation thus becomes an actionable signal for improving a red or blue project. Fourth, an optional in-context improvement loop that turns each round's outcome into advisory hints for the next. An adapter can then adapt across rounds without keeping state, provided it reads the hints. We describe the design of ACEA and the metrics through which red and blue teams are scored head to head.
Red-teaming is becoming a central part of large language model (LLM) safety evaluation, yet current practice still relies heavily on expert-written prompts or fixed benchmark suites. This creates a gap between what is easy to test and what deployed models can actually do: failures may be rare, context-sensitive, and distributed across many threat categories. We study automated red-teaming as a constrained adversarial search problem and introduce a learning-driven framework that couples category-aware attack generation with hierarchical vulnerability detection. The method starts from curated safety seeds, expands them through meta-prompt-guided and evolutionary search, and scores the resulting prompt--response pairs with lexical, semantic, and behavioral detectors. Across six threat categories on GPT-OSS-20B, the framework discovers 47 validated vulnerabilities, including 21 high-severity cases and 12 novel attack patterns. Under matched query budgets, it achieves a 3.9× higher discovery rate than manual expert red-teaming while maintaining 89% detection accuracy and full category coverage. Ablations show that the gains do not come from more prompts alone: diversity constraints prevent template collapse, coverage constraints prevent category blind spots, and semantic detection recovers failures missed by lexical rules. These results suggest that red-teaming can be made more scalable and reproducible when treated as adaptive search rather than as a static checklist.
Zhang Wei, Hanxuan Chen, Peilu Hu +19
Independent Researcher · Hunan University · Shenzhen Kaihong Digital Industry Development Co., Ltd. +12