Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

Nivya Talokar; Ayush K Tarun; Murari Mandal; Maksym Andriushchenko; Antoine Bosselut

cs.CLFeb 18, 2026

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

Authors: Nivya Talokar, Ayush K Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut

Organizations: Independent Researcher · EPFL, Switzerland · KIIT Bhubaneswar, India · ELLIS Institute Tübingen, MPI for Intelligent Systems, Tübingen AI Center, Germany

Abstract

LLM-based agents execute real-world workflows via tools and memory. These affordances enable ill-intended adversaries to also use these agents to carry out complex misuse scenarios. Existing agent misuse benchmarks largely test single-prompt instructions, leaving a gap in measuring how agents end up helping with harmful or illegal tasks over multiple turns. We introduce STING (Sequential Testing of Illicit N-step Goal execution), an automated red-teaming framework that constructs a step-by-step illicit plan grounded in a benign persona and iteratively probes a target agent with adaptive follow-ups, using judge agents to track phase completion. We further introduce an analysis framework that models multi-turn red-teaming as a time-to-first-jailbreak random variable, enabling analysis tools like discovery curves, hazard-ratio attribution by attack language, and a new metric: Restricted Mean Jailbreak Discovery. Across AgentHarm scenarios, STING yields substantially higher illicit-task completion than single-turn prompting and chat-oriented multi-turn baselines adapted to tool-using agents. In multilingual evaluations across six non-English settings, we find that attack success and illicit-task completion do not consistently increase in lower-resource languages, diverging from common chatbot findings. Overall, STING provides a practical way to evaluate and stress-test agent misuse in realistic deployment settings, where interactions are inherently multi-turn and often multilingual.

Figures & tables

Figure 1: STING: (a) A Strategist constructs a deceptive persona and decomposes the harmful intent into executable phases. (b) The Attacker embodies the persona and attempts each phase against the Target agent. After each target response, the (c) Refusal Detector checks for refusal; if none is detected, the (d) Phase-Completion Checker assesses whether the phase objective has been met. Both evaluators provide actionable feedback to guide the Attacker’s next turn. A jailbreak is declared once all phases are successfully completed.

Target Agent	Single-	STING (Multi-turn)		X-Teaming
Model	turn	$S_{\max}=10$	$S_{\max}=5$	$S_{\max}=5$
Qwen3-Next	35.1 $\pm$ 6.1	72.7 $\pm$ 4.1	67.8 $\pm$ 4.5	27.0 $\pm$ 4.9
GPT-5.1	24.3 $\pm$ 5.3	34.1 $\pm$ 5.5	29.7 $\pm$ 5.3	5.0 $\pm$ 2.2
Gemini 3 Flash	45.9 $\pm$ 6.2	50.9 $\pm$ 5.5	47.6 $\pm$ 5.8	13.8 $\pm$ 4.0
Claude Sonnet 4.5	16.0 $\pm$ 4.8	32.3 $\pm$ 5.6	28.0 $\pm$ 5.4	2.2 $\pm$ 1.4
DeepSeek-V3.2	31.2 $\pm$ 5.5	61.8 $\pm$ 4.7	57.2 $\pm$ 4.9	15.1 $\pm$ 4.1

Table 1: AgentHarm Score (AHS) for single-turn prompting versus our multi-turn STING framework on English AgentHarm prompts. We also include X-Teaming ( Rahman et al., 2025 ) for comparison.

S_{max}

denotes the number of strategies tried. STING outperforms both single-turn prompting and X-Teaming.

Figure 2: Kaplan–Meier discovery curves (95% CI) showing the fraction of harmful behaviours for which at least one strategy succeeds (jailbreak) for a given strategy budget; RMJD summarizes each curve (higher = earlier/more jailbreak successes).

Model	Chinese	French	Ukrainian	Hindi	Urdu	Telugu
Qwen3-Next	[0.76, 1.20]	[0.95, 1.44]	[0.73, 1.16]	[0.62, 0.98]	[0.80, 1.35]	[0.44, 0.74]
GPT-5.1	[0.73, 1.38]	[0.84, 1.53]	[0.83, 1.65]	[0.98, 1.95]	[0.79, 1.47]	[0.87, 1.69]
Gemini 3 Flash	[0.84, 1.29]	[1.04, 1.64]	[1.11, 1.79]	[0.94, 1.58]	[1.18, 2.08]	[0.72, 1.37]

Table 2: Language-specific hazard-ratio 95% confidence intervals for jailbreak susceptibility, using English as the baseline. Hazard ratios below 1 indicate lower jailbreak susceptibility than English, while values above 1 indicate higher susceptibility. We highlight only intervals that do not contain 1, emphasizing languages whose jailbreak dynamics differ significantly from English.

Panel of Figure 3 — Figure 3: AgentHarm Score (%) comparison between single-turn prompting and STING across 7 languages for 3 models. Differences in misuse outcomes are less pronounced than those reported in prior chatbot-focused jailbreak studies ( Yong et al., 2023 ) .

Model	English	Chinese	French	Ukrainian	Hindi	Urdu	Telugu
Qwen3 Next	79.2 $\pm$ 4.1	75.0 $\pm$ 4.4	75.1 $\pm$ 4.4	74.0 $\pm$ 4.8	79.1 $\pm$ 3.9	76.8 $\pm$ 4.0	76.2 $\pm$ 4.4
GPT-5.1	77.8 $\pm$ 4.1	72.6 $\pm$ 4.5	69.6 $\pm$ 4.6	72.2 $\pm$ 4.5	77.5 $\pm$ 4.2	77.3 $\pm$ 4.4	77.3 $\pm$ 4.6
Gemini 3 Flash	76.0 $\pm$ 4.8	72.9 $\pm$ 5.0	73.0 $\pm$ 5.0	73.3 $\pm$ 5.0	74.7 $\pm$ 5.1	74.9 $\pm$ 4.8	73.9 $\pm$ 4.9

Table 3: Language-wise AgentHarm Score (AHS) for different models on single-turn benign AgentHarm tasks. Scores indicate comparable agentic capability across languages.

Panel of Figure 4 — Figure 4: AgentHarm Score (AHS) for Qwen3-Next and GPT-5.1 under varying reasoning settings across languages. No-thinking settings are consistently less safe. For GPT-5.1, medium reasoning is safer than high reasoning.

	No Tool Hint		With Tool Hint
Target Agent	AHS	ASR (%)	AHS (%)	ASR (%)
Qwen3-Next	72.6 $\pm$ 6.5	82.9 $\pm$ 7.4	73.1 $\pm$ 5.4	87.4 $\pm$ 6.2
GPT-5.1	31.4 $\pm$ 7.3	49.8 $\pm$ 10.2	36.7 $\pm$ 8.1	38.6 $\pm$ 9.7
Gemini 3 Flash	49.9 $\pm$ 7.3	62.6 $\pm$ 10.2	51.8 $\pm$ 7.6	56.7 $\pm$ 10.2
Claude Sonnet 4.5	30.4 $\pm$ 7.1	25.9 $\pm$ 9.1	34.5 $\pm$ 8.2	31.7 $\pm$ 9.1
DeepSeek-V3.2	59.5 $\pm$ 6.7	67.0 $\pm$ 9.7	63.9 $\pm$ 6.8	69.5 $\pm$ 10.2

Table 4: Results under prompts without versus with tool-name hints in English. AHS and ASR denote AgentHarm Score and Attack Success Rate, respectively. Tool Knowledge consistently boosts illicit-task completion in terms of AHS.

Setting	Defense	English	Chinese	French	Ukrainian	Hindi	Urdu	Telugu
Single-Turn Benign	PromptGuard	-1.5	0	0	0	0	0	0
	Safety Prompt	-5.7	-5.7	-5.9	-5.8	-5.7	-5.8	-5.8
	Omni-Moderation	0.0	0.0	0.0	-1.1	0.0	-0.9	0.0
STING (Multi-Turn) Harmful	PromptGuard	-3.8	-2.4	-0.3	-0.4	-0.8	-0.1	0
	Safety Prompt	-38.9	-38.2	-34.8	-33.3	-40.5	-38.9	-36.7
	Omni-Moderation	-12.5	-12.0	-17.2	-10.8	-7.7	-7.1	-2.4

Table 5: Relative changes (%) in AgentHarm Score (AHS) with respect to the no-defense baseline after applying PromptGuard, a Safety Prompt, and OpenAI’s Omni-Moderation API across benign and harmful settings on Qwen3-Next (

S_{max}=5

). All defenses show minimal change for benign prompts, suggesting low false-refusal rates. For harmful prompts, the Safety Prompt yields the largest AHS reductions, while the Omni-Moderation API provides moderate reductions and PromptGuard has limited effect.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Strategist	English	Ukrainian	Telugu
Gemini 3 Pro	63.2 (67.8)	55.7 (70.5)	53.5 (72.3)
Qwen3.5-27B	53.3 (55.7)	42.9 (60.9)	42.1 (59.8)

Appendix

Table 6: Phase separation quality across strategists. Values report the percentage of phase pairs with non-overlapping tool sets; values in parentheses report downstream AgentHarm Score (AHS, %). Higher phase separation indicates cleaner decomposition into distinct executable subgoals and is associated with stronger illicit-task completion.

Setting	English	Ukrainian	Telugu
Original	67.9	70.1	71.1
WE	70.7	72.4	70.1
WA+WE	67.8	70.5	72.3
WS+WA+WE	55.7	60.9	59.8

Appendix

Table 7: Effect of weakening different STING agents on AgentHarm Score (AHS, %). WE denotes weakening the evaluator agents, WA denotes weakening the attacker, and WS denotes weakening the strategist. Weakening the evaluator or attacker has little effect, while weakening the strategist produces a large AHS drop.

Language	No-Thinking		Thinking
Language	AHS (%)	ASR (%)	AHS (%)	ASR (%)
English	72.8 $\pm$ 4.1	85.2 $\pm$ 5.1	65.2 $\pm$ 5.0	87.5 $\pm$ 4.8
Ukrainian	75.1 $\pm$ 3.8	84.7 $\pm$ 5.1	64.2 $\pm$ 4.7	81.2 $\pm$ 6.0
Telugu	74.4 $\pm$ 3.4	80.2 $\pm$ 6.0	65.1 $\pm$ 4.8	74.4 $\pm$ 6.5

Appendix

Table 8: AgentHarm Score (AHS) and Attack Success Rate (ASR) for Qwen3-Next Instruct (no-thinking) vs. Thinking variants. The Thinking variant is safer in Ukrainian and Telugu, and also yields a lower AHS in English.

Language	No-Thinking		Medium		High
Language	AHS (%)	ASR (%)	AHS (%)	ASR (%)	AHS (%)	ASR (%)
English	34.1 $\pm$ 5.3	44.4 $\pm$ 7.1	30.3 $\pm$ 5.1	37.4 $\pm$ 6.8	31.5 $\pm$ 5.0	44.2 $\pm$ 7.4
Ukrainian	37.1 $\pm$ 5.0	50.5 $\pm$ 7.4	32.0 $\pm$ 5.1	42.6 $\pm$ 7.4	35.0 $\pm$ 5.5	44.7 $\pm$ 7.4
Telugu	40.5 $\pm$ 5.3	49.2 $\pm$ 7.4	32.9 $\pm$ 5.3	35.8 $\pm$ 7.1	35.4 $\pm$ 5.3	37.6 $\pm$ 7.4

Appendix

Table 9: AgentHarm Score (AHS) and Attack Success Rate (ASR) for GPT-5.1 (

S_{max}=5

) using different reasoning-efforts: none, medium, high. Highest values are bold , whereas the second highest values are underlined . The medium reasoning level is the safest.

Model	English	French	Hindi	Telugu
Qwen3-Next	67.9	68.7 (67.1)	71.2 (73.4)	69.0 (71.1)
Gemini 3 Flash	47.0	47.1 (46.3)	47.9 (48.7)	44.5 (46.6)

Appendix

Table 10: AgentHarm Score (AHS, %) under native-language strategy generation. Values in parentheses correspond to English-plan strategy generation, our default setup.

Language	Token Ratio	Similarity
Chinese	0.98	0.727
French	1.32	0.751
Ukrainian	1.85	0.734
Hindi	2.94	0.696
Urdu	2.32	0.656
Telugu	4.97	0.509

Appendix

Table 11: Token-count ratios relative to English and semantic similarity between same-phase attacks across languages.

Language	ASR (%)	AHS
French	89.20	72.01
Urdu	88.64	76.42
Chinese	85.23	75.08
English	85.23	72.57
Ukrainian	84.66	75.02
Hindi	82.95	77.64

Appendix

Table 12: Attack Success Rate (ASR) and AgentHarm Score (AHS) for (a) Qwen3-Next, (b) GPT-5.1, and (c) Gemini 3 Flash. The Combined row represents the global vulnerability (union of failures across languages), demonstrating that the aggregate risk is significantly higher than any single-language baseline.

Setting	Model	English (PG/SP)	Ukrainian (PG/SP)	Telugu (PG/SP)
Single-Turn Benign	GPT-5.1	-1.5 / -2.7	0.0 / -1.9	0.0 / -5.4
Single-Turn Benign	Gemini 3 Flash	-1.5 / -2.0	0.0 / -5.9	0.0 / -0.6
STING (Multi-Turn) Harmful	GPT-5.1	-0.3 / -13.8	-1.1 / -16.8	-1.5 / -25.7
STING (Multi-Turn) Harmful	Gemini 3 Flash	-1.2 / -41.1	-0.8 / -44.1	0.0 / -47.4

Appendix

Table 13: Relative changes (%) in AgentHarm Score (AHS) with respect to the no-defense baseline after applying PromptGuard (PG) and the Safety Prompt (SP) on GPT-5.1 and Gemini 3 Flash.

Setting	Defense	English	Chinese	French	Ukrainian	Hindi	Urdu	Telugu
Single-Turn	PromptGuard	[-3.7, 0.0]	0	0	0	0	0	0
Benign	Safety Prompt	[-9.4, -2.4]	[-9.4, -2.1]	[-9.4, -2.6]	[-9.3, -2.3]	[-9.3, -2.3]	[-9.9, -1.9]	[-9.6, -2.2]
Multi-Turn	PromptGuard	[-6.9, -1.4]	[-4.7, -0.6]	[-1.0, 0.0]	[-1.4, 0.0]	[-1.9, 0.0]	[-0.3, 0.0]	0
Harmful	Safety Prompt	[-46.3, -31.9]	[-45.0, -32.0]	[-42.1, -27.5]	[-39.8, -27.0]	[-46.87, -33.82]	[-45.6, -32.4]	[-43.14, -30.26]

Appendix

Table 14: 95% confidence intervals for relative changes (%) in AgentHarm Score (AHS) with respect to the no-defense baseline after applying PromptGuard and a Safety Prompt across settings on Qwen3-Next.

LLM Red Teaming

Large language model red teaming probes how models and agents can be induced to violate safety, security, or operational constraints.

See all 61 papers

$M^3$ : A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process
Oct 22, 2026Yuqing Huang, Jack Ji, Jithin Pradeep +2Adversarial Attacks on LLMs Multi-Turn Jailbreak Attacks
The Vanguard Group∗
REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
Aug 11, 2026Zixing Chen, Xingyuan Liu, Jie Zhu +6LLM Red Teaming LLM Agents
Fudan University · The Hong Kong University of Science and Technology · Qwen DianJin Team, Alibaba Cloud Computing +1
LoopTrap: Termination Poisoning Attacks on LLM Agents
May 7, 2026Huiyu Xu, Zhibo Wang, Wenhui Zhang +4Adversarial Attacks on LLMs LLM Red Teaming
The State Key Laboratory of Blockchain and Data Security Zhejiang University Hangzhou, Zhejiang, P. R. China · School of Cyber Science and Engineering Southeast University Nanjing, Jiangsu, P. R. China

Oct 22, 2026J/K move · Enter open · S save

$M^3$ : A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process

Yuqing Huang, Jack Ji, Jithin Pradeep, Sagarika Suresh+1

The Vanguard Group∗

Adversarial Attacks on LLMs Multi-Turn Jailbreak Attacks Automated Red Teaming Language Model Safety Evaluation

Open paper AskPDF arXiv

Modern Large Language Models (LLMs) remain vulnerable to carefully orchestrated multi-turn interactions that progressively bypass safety mechanisms. However, many existing red-teaming approaches rely on heuristics or impose strong assumptions that limit their applicability in fully black box settings. We address this gap by introducing

M^3

, an automated red-teaming framework that formalizes multi-turn jailbreaking as a finite-horizon Markov Decision Process (MDP). In

M^3

, an attacker LLM interacts with a target LLM while operating over a compact internal state produced by a state LLM. Across two benchmarks and six target models,

M^3

consistently outperforms state-of-the-art multi-turn red-teaming methods in Attack Success Rates (ASR), achieving improvements of up to 53% in ASR@5. Even with lightweight attacker,

M^3

attains gains up to 26% in ASR@1 and 34% in ASR@5, compared to baselines that use higher capacity attackers. Beyond improved effectiveness,

M^3

exhibits steeper risk escalation over conversational turns and achieves more consistent performance across risk categories (up to 4.82 improvement in Gini index) without relying on predefined risk taxonomy, providing a scalable and principled framework for multi-turn red-teaming and safety evaluation.

Abstract

LLM Red Teaming

Explore similar work

M3M^3M3: A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

LoopTrap: Termination Poisoning Attacks on LLM Agents

$M^3$ : A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process