Fooling LLM Safety Guards into Judging the Wrong Text

Utkarsh Kanwat

cs.CROct 20, 2026

Fooling LLM Safety Guards into Judging the Wrong Text

Authors: Utkarsh Kanwat

Organizations: Independent Researcher

Abstract

Safety guards check content before an LLM application acts on it, and most paste that content into a plain-text prompt. Content that imitates the prompt can append a second task, and the guard answers that one instead. We call this target rebinding. Our explanation is that the guard grades the last complete task in its input, here a harmless decoy, so the harmful request goes through unchanged. In a pre-registered test, an appended forged query and decoy clear all 75 held-out AdvBench requests that Shieldstral-1.0-3B flags. The guard is not persuaded: a directive to ignore the text above clears none of them, and in a follow-up run a harmful decoy stops the attack. On our GuardRebind benchmark, each of five guards has an attack family, chosen per guard, that clears at least 93% of its flagged items. Rebinding explains this best on Shieldstral and gpt-oss-safeguard; on ShieldGemma and Llama Guard 3, generic evasion and trailing text respectively do part of the work, and Qwen3Guard falls mainly to a forgery of its whole task. Training the guard to score content no lower than its most harmful part cuts attack success from 75% to 0.8-1.6% on attack families, decoys and forged queries held out of training. All of these attacks append the forged structure after the request, and we added the held-out-decoy evaluation after a first round of test runs. Clean AUROC falls by at most 0.01, but our test of non-inferiority is inconclusive. A score-guided search still beats this trained guard on 47 of 50 items, usually by placing forged structure before the request. On about half of the items where the search wins, the same search also finds benign text, with no forged structure, that clears the guard.

Figures & tables

guard score

P(\text{yes})

harmful request alone 0.999 (flagged)

with the forged pair 0.023 (cleared)

Figure 1: Target rebinding on Shieldstral. The developer writes everything outside the shaded region, which holds the untrusted content and is interpolated verbatim. The content is a held-out AdvBench request followed by a forged <Query> / <Document> pair with a benign decoy. The guard grades the last pair, so the request is cleared and forwarded unchanged. Scores are this request’s values in the confirmatory run ( Section 3 ).

appended content	kind	bypass
nothing	control	0% [0%, 5%]
the sentence $S$ alone	control	0% [0%, 5%]
same sentence as a “Note:”	control	0% [0%, 5%]
inert filler of similar length	control	0% [0%, 5%]
forged Instruct	structural	0% [0%, 5%]
forged Instruct $+$ Query	structural	0% [0%, 5%]

Table 1: What the attacker appends, and how often it clears Shieldstral.

n=75

flagged held-out AdvBench requests,

\tau=0.5

, bypass with Wilson 95% interval. Every row with an instruction uses the same sentence

S

, a directive to disregard the section above. Only rows with the full forged query–document pair clear over

9\%

Figure 2: The guard grades the decoy. (a) Per-request scores with no attack and with the forged pair (

n=75

): every attacked score falls below

\tau

. (b) Exploratory: the same attack with three decoys (

n=39

): the bypass collapses as the decoy becomes more harmful. Bars are medians; diamonds are each decoy scored on its own.

bypass	flagged items	flag rule	attack, decoys and queries
$100\%$	75 AdvBench requests ( Section 3 )	$>0.9$	the forged pair; one fixed decoy and query
$70.9\%$	300 benchmark items, six sources ( Section 4 )	$\geq 0.5$	the angle-bracket pair; decoys from a pool
$99.2\%$	the same 300 items	$\geq 0.5$	the strongest of 52 families, chosen after the fact
$75.0\%$	297 test items ( Section 6 , base guard)	$\geq 0.5$	held-out families, decoys and queries

Table 2: The four headline Shieldstral bypass figures. Each counts only items flagged with no attack; controls use their own item sets. On Shieldstral the guard-native forgery renders as the same text as the angle-bracket pair; with its own decoy draws it clears

71.7\%

( Table 3 ).

guard	flagged	clean FNR	clean FPR	best bypass	native	instr.	harm. decoy	fam. $>50\%$
Shieldstral 3B	300	50.0	6.7	99.2 [98.2, 100]	71.7	7.0	13.6	36
Llama Guard 3 1B	374	37.7	15.8	100 [100, 100]	100	2.9	11.1	44
ShieldGemma 2B	168	72.0	2.8	100 [100, 100]	73.6	20.2	84.3	25
Qwen3Guard 0.6B	536	10.7	27.8	93.4 [91.5, 95.1]	81.7	0.7	0	8
gpt-oss-safeguard 20B	113	43.5	12.0	100 [100, 100]	71.7	6.2	9.4	16

Table 3: GuardRebind on the flagged core items,

\tau=0.5

. Policy: in the prompt (Shieldstral), in the system role (gpt-oss-safeguard), or a fixed taxonomy (others). “flagged”: harmful core items (of 600; 200 for gpt-oss-safeguard) flagged with no attack. Clean FNR, FPR: no-attack miss rate on harmful and flag rate on benign items. Rates in %. “best”: the strongest of 52 attack families (two-level bootstrap 95% interval, unadjusted for the maximum). “native”: the guard-native forgery, appended. “instr.”: the strongest instruction-only payload (a forged instruction field, a bare sentence or a “Note:”). “harm. decoy”: the strongest of four forgeries with a harmful decoy. “fam.

>50\%

”: attack families (of 52) above 50% bypass. No control exceeds 50%; the strongest (benign text appended, on Llama Guard 3) reaches 41.2%. gpt-oss-safeguard is label-only (score

=

its verdict, strict reading), user-message channel only.

	bypass $\downarrow$			detection
	fixed decoy		held-out
condition	train	held-out	decoy	AUROC	recall	FPR
none (base)	87.9	85.1	75.0	0.810	47.1	5.9
B (untrained boundary)	90.7	85.4	76.8	0.827	47.8	5.4
structure detector	0.0	24.5	23.2	0.814	47.9	5.9
inference-time segment-max	0.0	24.6	23.4	0.810	47.1	5.9

Table 4: Defense on the test split, Shieldstral,

\tau=0.5

, in %. Bypass is over benign or borderline decoys (lower is better): training and held-out families with the fixed decoy and query, and held-out families with held-out decoys and queries, the primary measure. Detection is on the clean test sample. T has two seeds (range); the other trained rows have one. Intervals for selected held-out-decoy figures are in the text.

condition	OR-Bench-hard	XSTest-safe	structured benign	benign decoy $\downarrow$	harmful decoy $\uparrow$
	over-refusal $\downarrow$			benign $+$ forged pair
none (base)	51.5	6.7	5.8	0.2	100
B (untrained boundary)	52.8	8.0	6.8	0.1	100
structure detector	51.5	6.7	58.6	87.5	100
inference-time segment-max	51.5	6.7	9.8	13.0	100
T (2 seeds)	52.0–54.3	2.7–5.3	5.0	0.7	100

Table 5: Utility and trailing-content checks for the conditions of Table 4 , in %. Over-refusal is the flag rate on benign sets (lower is better). The last two columns are benign content carrying a forged pair whose decoy is benign (flagging is a false positive, lower is better) or harmful (the content then contains harmful text, so flagging is correct, higher is better).

guard	score-guided search	structural search	all attacks	plain-text control
none (base)	50/50 [93%, 100%]	50/50 [93%, 100%]	50/50 [93%, 100%]	35/50 [56%, 81%]
T	47/50 [84%, 98%]	4/50 [3%, 19%]	49/50 [89%, 100%]	26/50 [39%, 65%]
B $+$ T	34/50 [54%, 79%]	4/50 [3%, 19%]	$\geq$ 34/50 (no GCG)	not run

Table 6: Adaptive attacks on Shieldstral, 50 flagged test items, seed 0. Each cell is the number of items cleared, with Wilson 95% interval. “All attacks” is the per-item worst case over the three attacks (GCG ran on 20 items and not on B

+

T, so that cell is a lower bound). The plain-text control is not counted in it.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

	prediction	outcome
P1	instruction-only $\leq 10\%$	confirmed (0%)
P2	forged pair $\geq 70\%$	confirmed (100%)
P3	harmful decoy lowers bypass	not a valid test as run: the panel’s “harmful decoy” scores $0.03$ alone, so it is the borderline decoy; bypass fell to 29–35%; the exploratory three-decoy comparison and M3 address the prediction
P4	not recency: prepending clears markedly less than appending	confirmed (inert filler appended last 0%; forged pair appended 100%, prepended 15%)
P5	syntax not load-bearing	confirmed (all variants high, no labels 0%)
P6	wide policy-dependence	refuted: median 99%, range 87–100%

Appendix

Table 7: Pre-registered predictions and outcomes. P: confirmatory study; M: benchmark; F, A: defense. “Refuted”: the data contradict the prediction; “not confirmed”: the test is inconclusive. Intervals are 95%.

condition	train	held-out	AUROC	recall	FPR	OR-B.	XST.	struct.	benign	harmful
	bypass $\downarrow$		detection			over-refusal $\downarrow$			benign $+$ forged pair
naive adv. training (2 seeds)	0.9–1.0	0.9–1.1	0.921–0.925	83.3–84.6	12.6–14.9	0.3–0.8	8.0–10.7	0.0	7.8–8.2	8.0–8.6
Llama Guard 3 1B, base	84.0	82.0	0.758	58.7	16.9	40.5	1.3	23.4	2.4	97.3
Llama Guard 3 1B, T	2.3	2.5	0.771	58.4	15.0	38.3	1.3	23.0	22.5	100
Qwen3Guard 0.6B, base	28.3	9.5	0.917	88.5	23.6	76.8	21.3	30.4	10.5	100
Qwen3Guard 0.6B, T	0.0	0.0	0.924	89.0	23.3	77.3	20.0	29.4	25.9	100

Appendix

Table 8: Fixed-decoy conditions on the test split,

\tau=0.5

, in %. Columns as in Tables 4 and 5 ; naive adversarial training has two seeds (range), the other rows seed 0.

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 194 papers

Sep 8, 2026

Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts

Automatic safety judges -- for example Llama Guard, or GPT-4o given a grading prompt -- produce the numbers behind reported jailbreak success rates, defense evaluations, and safety leaderboards. We ask whether these judges grade what a reply contains or how it sounds. We place a fixed piece of text before or after a model reply (a style wrapper) and leave the reply itself unchanged byte-for-byte, so any change in verdict is an error of the judge. Over 600 JailbreakBench replies (300 harmful, 300 refusals), 9 wrappers, and 8 judges, using paired tests and a measured per-judge repeat-disagreement rate (the noise floor), we find that most judges barely move but specific judges have specific, cheap blind spots. A token refusal ("I can't help with that") placed before the unchanged harmful text flips 19.9% of GPT-4o-mini's correct "unsafe" verdicts to "safe" (noise floor 0.5%; 18.2% under majority-of-three re-scoring), yet moves Claude only 0.4%. The deployed Llama Guard 4 guard model is evaded by an "educational course" preface on 12.3% of harmful replies, and the flips persist across repeats, while a second deployed guard (gpt-oss-safeguard-20b) is immune. Changing only the grading prompt to ignore tone cuts the GPT-4o-mini attack roughly tenfold on the same model, so the weakness lives in the judge, not in the content. Pure tone-only decoration that adds no new claim still flips up to 2.1% of verdicts on the vulnerable judges, several times their noise floors. On this artifact set the judges also disagree about which target model is safest, and a bootstrap shows the ranking is unstable before any wrapper is added. We release the data, wrappers, code, and per-verdict labels.

Yongxi Zhou, Wenbo Ye, Yuanzhe Liu +3

Northeastern University, Massachusetts, USA · University of Southern California, California, USA · Georgia Institute of Technology, Georgia, USA

LLM Guardrails LLM-as-a-Judge Language Model Safety Evaluation LLM Safety Benchmarks Safety Filtering Adversarial Robustness of Language Models+1

Ask PDF

Jun 12, 2026

From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails

LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents. However, we reveal that the very reasoning and task-following capabilities enabling this protection introduce a novel vulnerability: attackers can inject crafted data to trap the guardrail in extended reasoning loops, effectuating a systematic denial-of-service (DoS) attack. To systematically expose this threat, we design a beam-search optimization framework that crafts natural-language payloads to maximize guardrail reasoning length, utilizing an LLM proposer guided by a strategy bank. Based on the observation of guardrail's schema-following nature, we also provide another attack framework driven by mechanism-aware structural mutations with less computational load. The attack efficacy is systematically evaluated in two parts. First, in standalone evaluations, the attack generalizes across diverse guardrail architectures, safety templates, and agent benchmarks. Payloads optimized on a single open-source surrogate successfully transfer to eight leading model backbones (e.g., Claude, GPT, Gemini, DeepSeek, and Qwen), achieving a 13--63

\times

token amplification. Second, in end-to-end real-world agent deployments (web, desktop, code, and multi-agent systems), the attack reveals up to a 148

\times

latency amplification. We show that a single poisoned document can saturate shared guardrail infrastructures, effectively starving co-located agents and paralyzing the entire system. By uncovering this availability flaw, our work underscores the urgent need to develop cost-bounded, reasoning-robust guardrails.

Yuguang Zhou, Xunguang Wang, Pingchuan Ma +3

1The Hong Kong University of Science and Technology · 2Zhejiang University of Technology

LLM Guardrails Adversarial Attacks on LLMs Cybersecurity AI Agent Security

Ask PDF

May 24, 2026

Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection

Large language model (LLM) safety classifiers such as Llama Guard are effective at detecting overtly harmful prompts but remain vulnerable to adversarial jailbreak attacks that disguise malicious intent through role-play scenarios, fictional framing, and indirect requests. We present Reflect-Guard, a method that augments LLM-based safety classifiers with chain-of-thought self-reflection capabilities through parameter-efficient fine-tuning. Our approach distills analytical reasoning from GPT-4o-mini into structured reflection annotations, then trains Llama-Guard-3-8B via QLoRA to generate logical self-reflections before issuing safety verdicts. Using only 1000 training examples and updating just 0.5% of model parameters (~42M), Reflect-Guard achieves substantial improvements on two challenging benchmarks. On WildGuardTest, F1 score improves from 0.770 to 0.842 (+7.2 pp), with recall on adversarial prompts increasing from 0.513 to 0.921 (+40.8 pp). On JailbreakBench, the attack success rate drops from 10.3% to 1.8%, representing an 82.5% relative reduction. These gains are especially pronounced on adversarial inputs, where the explicit reasoning step enables the model to see through obfuscation techniques that defeat standard pattern-matching approaches. Our results demonstrate that teaching safety classifiers to reason about adversarial intent, rather than simply classify surface patterns, is a promising direction for robust LLM safety.

Lixing Lin, Juli You, Yue Li +4

yaleYale University, New Haven, CT, USA · indIndependent Researcher · columbiaColumbia University, New York, NY, USA +2

Language Model Safety Evaluation LLM Guardrails Safety Filtering Adversarial Robustness of Language Models Jailbreak Detection Jailbreak Defense

Ask PDF

Abstract

LLM Guardrails

Explore similar work

Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts

From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails

Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection