Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures

Srinivasan Subramanian; Kazi Aminul Islam; Md. Abdullah Al Hafiz Khan

cs.CLAug 17, 2026

Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures

Authors: Srinivasan Subramanian, Kazi Aminul Islam, Md. Abdullah Al Hafiz Khan

Organizations: Department of Computer Science, Kennesaw State University, Marietta, GA, USA

Abstract

As Large Language Models (LLMs) move from conversational assistants to advanced agentic systems, guardrail failures can convert adversarial intents into harmful executions. However, most evaluation frameworks focus only on whether a user request is unsafe. This is insufficient for multi-turn failures, where adversarial intent may be distributed across turns and emerge only from accumulated context. This motivates us to go beyond detection to localize the evidence that pushes the conversation toward unsafe trajectories. We introduce MTAG-Bench, a multi-turn guardrail attribution dataset with behavioral validation and counterfactual supervision containing 4,167 conversations. We also introduce MTAG-Probe, a hierarchical attribution model for turn- and span-level evidence localization with conversation-level safety detection. MTAG-Probe improves turn-level F1 by 28.1 and 24.1 points (79.0% and 61.0% relative) over self-attribution by the model that produced the unsafe response and the LLM safety-judge attribution, respectively. Removing MTAG-Probe's top-ranked evidence within budgets of 15% and 20% of user words per conversation changes 50.6% and 60.8% of Llama Guard decisions on Geometry from unsafe to safe, respectively. In full-trajectory replay, MTAG-Probe converts 84.3% of eligible unsafe final responses to safe. These results show that MTAG-Probe can provide evidence-level explanations as a practical complement to safety detection.

Figures & tables

Figure 1: Motivation for evidence localization. Standard conversation-level safety detection determines whether an interaction is unsafe. Multi-turn attribution identifies the behaviorally relevant evidence for the unsafe trajectory.

Statistic	Train	Dev	Test	Stress	Total
Full inventory	2,852	364	364	587	4,167
Primary benchmark	1,706	364	364	–	2,434
Detection-only auxiliary	1,146	–	–	–	1,146
Hard-benign stress	–	–	–	587	587
Primary adversarial	853	182	182	–	1,217
Primary benign	853	182	182	–	1,217

Table 1: MTAG-Bench statistics.

Evidence localization
Method	Turn F1	Span F1	Top-1	False attribution $\downarrow$
MTAG-Probe	63.6	91.5	85.0	28.7
Target-model self-attribution	35.6	75.6	59.0	99.3
Safety-judge attribution	39.5	71.5	83.0	94.7
Human evidence validation
Method	Two-human hit †	Agreed recall	Prec. vs. union	Rec. vs. union

Table 2: Attribution validation across model, human, and external-guard evaluations.

Figure 2: Internal faithfulness and behavioral replay. (a) Probe decision flips after deleting the highest-ranked user-word evidence at fixed budgets. (b) Full-trajectory replay at the 15% budget, reporting the fraction of cases whose final response changes from unsafe to safe, error bars show 95% bootstrap confidence intervals.

Model	Det. AP	Turn F1	Span F1	$\boldsymbol{\Delta P}$	Flip
Full	94.5	61.9	91.4	38.2	48.4
No Cross-turn	97.5	50.6	92.8	9.4	17.0
Frozen Backbone	96.1	59.7	94.9	37.0	45.1
Single-stage	95.0	55.8	94.6	23.1	31.3
Detection-only	96.3	6.8	58.3	3.9	4.9

Table 3: MTAG-Probe ablations.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Construction of the primary corpus and detection-only auxiliary data. Solid paths contribute to the primary corpus, and dashed paths contribute detection-only auxiliary examples.

Role	Model
Response-adaptive
Adversarial generator	Qwen2.5-7B/14B-Instruct
Benign generator	Qwen2.5-7B-Instruct
Target / CF replay	Llama-3-8B-Instruct
Validator / CF judge	Mistral-7B-Instruct-v0.3
Scenario-controlled

Appendix

Table 4: Models used for dataset construction, validation and behavioral replay.

Primary benchmark
Statistic	Train	Dev	Test	Total
Avg. primary turns	17.60	17.36	17.50	17.55
Avg. primary user turns	8.80	8.68	8.75	8.77
Corpus sources
Scenario-controlled	980	212	210	1,402
Response-adaptive	726	152	154	1,032

Appendix

Table 5: Additional MTAG-Bench dataset statistics.

Tier	$N$	Both	Both (%)	One	Neither
CF strong	95	80	84.2	7	8
LLM-confirmed	86	36	41.9	10	40
CF weak	1	1	–	0	0
Overall	182	117	64.3	17	48

Appendix

Table 6: Human validation of pipeline unsafe labels.

MTAG-Probe component	Parameters
Cross-turn encoder	1,974,273
Token-localization head	164,481
Detection head	98,817
Conversation pooler	33,025
Turn-localization head	33,025
Total MTAG-Probe	2,303,621

Appendix

Table 7: Task-specific parameter count for MTAG-Probe.

Model	Det. AP	Turn AP	Span F1	$\boldsymbol{\Delta P@15}$	Flip@15	$\Delta$ Flip vs. LOTO (pp)	$\Delta$ Flip vs. SpanRand (pp)
Cross-token-gated-frozen	94.3	92.0	95.7	0.1	3.3	$-2.2$	$-0.3$
Cross-token-gated-top4	97.2	90.7	95.4	15.9	29.7	$+1.0$	$+1.7$
Cross-token-sibling-frozen	96.4	93.3	97.0	19.6	24.7	$+3.1$	$+4.2$
Cross-token-sibling-top4	97.5	94.6	97.0	4.0	9.9	$-5.1$	$-0.6$
Hierarchical-sibling-frozen	95.0	92.1	96.7	30.5	45.6	$+1.6$	$+0.4$
Hierarchical-sibling-top4	96.1	90.6	94.9	36.6	41.2	$\mathbf{+7.8}$	$\mathbf{+5.0}$

Appendix

Table 8: Development-set architecture comparison.

Budget	$\Delta$ Flip	95% CI
5%	$+15.4$ pp	$[9.3,\ 22.0]$
10%	$+20.9$ pp	$[13.2,\ 28.6]$
15%	$+31.3$ pp	$[23.6,\ 39.0]$
20%	$+35.2$ pp	$[26.9,\ 43.4]$

Appendix

Table 9: Paired development-set flip-rate difference between the hierarchical sibling and cross-token sibling top-four models.

Method	Type	Purpose
LOTO	Baseline	Masks each user turn independently and ranks turns by the resulting decrease in unsafe probability from the same conversation classifier.
Matched random	Baseline	Deletes an equal number of user words at arbitrary locations, testing whether comparable effects arise from deletion alone.
SurfaceRisk	Baseline	Ranks user evidence using lexical surface-risk cues, testing whether localization can be recovered from risky wording alone.
LLM attribution	Baseline	Asks the failed target model and an independent safety judge to directly identify evidence-bearing user turns and spans.
Last-user	Baseline	Attributes evidence to the most recent user request, testing a simple recency heuristic.
MTAG-span-random	Control	Preserves MTAG-Probe’s selected turns, per-turn deletion allocation, and contiguous-span lengths while randomizing span positions within those turns.

Appendix

Table 10: Baselines and intervention controls.

Method	Source-specific judge	Common judge
MTAG-Probe	84.3 [76.8, 90.9]	84.3 [74.5, 92.2]
Random	49.5 [39.4, 59.6]	43.1 [29.4, 56.9]
SurfaceRisk	33.5 [24.4, 42.4]	39.9 [26.1, 52.9]

Appendix

Table 11: Full-trajectory behavioral replay.

Method	5%	10%	15%	20%
MTAG-Probe	26.9	40.1	48.4	49.5
LOTO	15.4	32.4	36.8	38.5
MTAG-span-random	23.1	35.8	44.0	49.2
Matched random	3.6	8.4	11.2	15.5
SurfaceRisk	1.1	1.1	1.1	1.1
Last-user	1.1	2.7	2.2	2.7

Appendix

Table 12: Internal fixed-budget intervention results.

Method	Turn P	Turn R	Turn F1 [95% CI]	Turn AP	Span P	Span R	Span F1 [95% CI]
MTAG-Probe	56.2	73.3	63.6 [58.4, 69.2]	92.7	98.1	85.6	91.5 [88.5, 94.3]
Target-model self-attribution	25.8	57.1	35.6 [31.8, 39.0]	87.5	97.9	61.6	75.6 [69.8, 81.1]
Safety-judge attribution	30.8	55.2	39.5 [34.9, 43.8]	93.1	100.0	55.7	71.5 [65.3, 77.4]

Appendix

Table 13: Detailed turn- and span-level localization results.

Metric	Before	After	$\Delta$
Turn F1	61.9	46.9	$-$ 15.0 [ $-$ 21.5, $-$ 8.9]
Span F1	91.4	88.0	$-$ 3.4 [ $-$ 6.7, 0.0]
False attribution	26.6	27.0	+0.4 [ $-$ 5.0, 5.2]

Appendix

Table 14: Robustness to semantic user-turn paraphrase.

Cohort	$N$	Prec.	Rec.	F1	FPR	AUROC
All	364	93.5	78.6	85.4	5.5	94.0
Scenario-controlled	210	88.9	68.6	77.4	8.6	89.1
Response-adaptive	154	98.6	92.2	95.3	1.3	97.3

Appendix

Table 15: Detection performance by corpus source.

Dataset	MTAG	Llama Guard	ShieldGemma	$n$ †
Geometry	45.2	11.5	12.8	8057
CoSafe	33.1	23.5	29.5	1400
MHJ	16.8	35.2	28.9	537

Appendix

Table 16: External detection performance.

Method	Top-3	Top-5	False attribution
MTAG-Probe	95.0	97.0	28.7 [21.5, 35.6]
Target self-attribution	90.0	96.0	99.3 [98.0, 100.0]
Safety-judge attribution	95.0	97.0	94.7 [90.8, 98.0]

Appendix

Table 17: Evidence ranking and attribution specificity.

Threshold	Benign FPR	Unsafe TPR
0.5	50.0	59.3
1.0	48.9	58.8
1.5	43.4	51.1
2.0	37.4	46.2
2.5	27.5	38.5
3.0	4.9	18.1

Appendix

Table 18: SurfaceRisk lexical specificity.

Dataset	Safety guard	Budget	$N$	$n$	MTAG-Probe flip	$\Delta$ vs. Random	$\Delta$ vs. SurfaceRisk
Geometry	Llama Guard	15%	8,057	926	50.6 [44.4, 57.8]	+33.0 [26.4, 40.2]	+43.4 [35.3, 52.6]
Geometry	Llama Guard	20%	8,057	926	60.8 [53.9, 68.0]	+37.5 [30.3, 44.7]	+54.1 [45.5, 63.0]
Geometry	ShieldGemma	15%	8,057	1,029	28.7 [23.7, 35.4]	+13.5 [10.2, 17.2]	+21.7 [16.1, 28.6]
Geometry	ShieldGemma	20%	8,057	1,029	34.7 [28.5, 43.3]	+18.0 [14.7, 22.2]	+25.6 [18.6, 33.8]

Appendix

Table 19: External safety-guard transfer on the full harmful Geometry cohort.

Method	False attr.	Rate	95% CI
MTAG-Probe	248/587	42.2	[36.2, 48.1]
SurfaceRisk	218/587	37.1	–
Target self-attribution	551/587	93.9	[91.6, 95.9]
Safety-judge attribution	479/587	81.6	[77.5, 85.4]

Appendix

Table 20: Specificity on hard-benign conversations.

Guard	Budget	$n$	Flip	$n$	Flip
		1–5 user turns		$\geq$ 6 user turns
Llama Guard	15%	55	38.2	871	51.4
Llama Guard	20%	55	41.8	871	62.0
ShieldGemma	15%	435	32.2	594	26.1
ShieldGemma	20%	435	38.9	594	31.6

Appendix

Table 21: External safety-guard transfer on Geometry by trajectory length.

Method	Turn AP	Span AP	Turn Top-1
MTAG-Probe	91.5	98.8	83.5
Grad $\times$ Input	83.1	85.0	71.6
Integrated Gradients	76.0	82.5	49.5

Appendix

Table 22: Target-model gradient attribution baselines on the target-supported adversarial test cohort.

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 437 papers

Evaluating Language Model Safety Across Long Adversarial Conversations
Sep 29, 2026Parisa Salmani, Peter R. LewisLanguage Model Safety Evaluation LLM Safety
Ontario Tech University Oshawa, Canada
One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent
May 7, 2026Xinjie Shen, Rongzhe Wei, Peizhi Niu +6Language Model Safety Evaluation LLM Guardrails
Georgia Institute of Technology · University of Illinois Urbana-Champaign · UCSD +3
TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety
Oct 1, 2026Fengpeng Li, Kemou Li, Qizhou Wang +3LLM Fine-Tuning Adversarial Attacks on LLMs
PRADA Lab, King Abdullah University of Science and Technology · State Key Laboratory of Internet of Things for Smart City, University of Macau · Imperfect Information Learning Team, RIKEN Center for Advanced Intelligence Project +1

Sep 29, 2026J/K move · Enter open · S save

Evaluating Language Model Safety Across Long Adversarial Conversations

Parisa Salmani, Peter R. Lewis

Ontario Tech University Oshawa, Canada

Language Model Safety Evaluation LLM Safety Adversarial Attacks on LLMs

Open paper AskPDF arXiv

Conversational safety evaluations often test language models with a single harmful prompt, even though real-world systems interact with users through long, adaptive conversations. This study examines whether models continue to respond safely when an adversarial user persists across multiple turns. We evaluate three open-weight, instruction-tuned models on two harmful prompts across different conversation lengths and random seeds. In each setting, a second language model acts as a persistent adversarial user, while a safety classifier labels every response as safe or unsafe. Across all model-prompt combinations, first-turn safe-response rates ranged from 85% to 100%. By depth 11, they dropped to 38-61%, and by depth 101, to 15-44%. This decline appeared across models and continued well beyond the short interactions typically used in multi-turn safety evaluations. These results provide proof-of-concept evidence that strong single-turn safety does not necessarily persist during sustained adversarial interaction. They highlight the need for long-horizon evaluations and conversation-level safeguards that account for risk accumulating across turns.

Abstract

LLM Safety

Explore similar work

Evaluating Language Model Safety Across Long Adversarial Conversations

One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent

TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety