Layered LLM Defenses as an Ensemble: Access Tiers, Inference Cost, and the Measured Failure Correlation Between Defense Layers

Abrar Alotaibi; Muhammad Shahid Jabbar; Sadam Al-Azani; Moataz Ahmed

cs.CRAug 28, 2026

Layered LLM Defenses as an Ensemble: Access Tiers, Inference Cost, and the Measured Failure Correlation Between Defense Layers

Authors: Abrar Alotaibi, Muhammad Shahid Jabbar, Sadam Al-Azani, Moataz Ahmed

Organizations: Information and Computer Science Department, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia · Department of Computer Science, College of Computer Science and Information Technology, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia · SDAIA-KFUPM Joint Research Center for Artificial Intelligence, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

Abstract

Practitioners defend large language models (LLMs) by stacking defenses, assuming the layers compound. A stack is an ensemble, and ensembles compound only under a condition the LLM security literature recommends but never measures: the members must fail on different inputs. Two instruments make that measurable. The Adversary Access-Tier Model (AATM) grades an adversary by the access it holds, from system-only (A0) to influence over training data (A4). A cost model sorts defenses into five classes of inference-time overhead; because two classes require training weights or reading activations, they tier the defender as AATM tiers the adversary. From these we derive how a stack behaves, and the quantities a defender cares about diverge: coverage saturates within a tier, cost rises by class, false refusals accumulate as a union, and residual attack success falls multiplicatively only under independence. We measure that independence. Running one adaptive adversary against a seven-layer stack, failure correlation is positive in all fifteen measurable pairs ( $φ$ from $0.30$ to $0.75$ ), and the joint residual exceeds the multiplicative prediction by up to $0.172$ . Stratifying on behavior difficulty dissolves most of the association, so the dependence is predominantly common-cause, but it survives permutation inference, majority-vote grader labels, and externally calibrated thresholds. The same stack refuses four in five benign prompts while remaining statistically indistinguishable from its strongest single layer. The dependence is architectural rather than sampling-based: members correlate through the model they all wrap, so no wider member pool weakens it. Diversity therefore selects stack members but does not predict what an assembled stack delivers, which has to be measured end to end.

Figures & tables

Tier	Name	What the adversary holds	Representative attacks
Outside the model
A0	No model access	The surrounding system only: dependencies, retrieved content, the deployment pipeline	Indirect prompt injection [ 43 ] , retrieval poisoning, RCE in LLM frameworks
Through the model, cumulative
A1	Black-box query	Inputs and outputs through a public interface	Direct jailbreaks, direct prompt injection, transfer attacks
A2	Gray-box	$+$ logits or token probabilities	Logprob-guided adaptive jailbreaks, some extraction
A3	White-box	$+$ weights, and the ability to compute gradients	Gradient suffix attacks [ 149 ] , safety-region pruning, fine-tuning attacks

Table 1: Adversary access tiers. A1 to A3 are cumulative; A0 and A4 are not, and are marked accordingly.

Figure 1: Taxonomy of LLM blue-teaming defenses by intervention point.

Defense layer	Tier assumed	Attacks addressed	Adaptive-tested	Overhead class	Cost profile
L1: Input/output censorship	A1	Direct jailbreaks, direct injection, harmful output	Rarely; constitutional classifiers are the exception	B; C (smoothing); A (guard prompts); E (probe screening)	Per-query passes; false positives
L2: Model training & fine-tuning	Design-time	Alignment failures, some A1 jailbreaks, prompt injection (trained)	Rarely	D	High one-time train; low per-query
L3: Adversarial training	Design-time	Optimized suffix attacks within budget	Partially	D; A (tuned prefixes)	High train; possible utility loss
L4: Monitoring & response	A0–A1	Anomalies, some injection, operational faults	n/a	B (often async); E (representation monitors)	Infrastructure; detection variance
L5: Infrastructure hardening	A0, A2–A3	Extraction, inversion, RCE; A0 injection (by isolation)	n/a	n/a (system-level)	Deployment complexity; low compute

Table 2: Master mapping of the five defense layers to AATM tiers, attack coverage, adaptive evaluation, and overhead class (Section 9 : A = additive tokens, B = extra serial passes, C = multiplicative sampling, D = training-time and amortized, E = activation-derived. Classes D and E additionally require defender access to weights or activations respectively.)

Dependency	Representative defenses	Fails when
Token surface	Perplexity filtering, blacklists, retokenization, Adversarial Prompt Shield	Attack is fluent or paraphrased
Semantic classification	ShieldLM, Llama Guard, WildGuard, ShieldGemma, constitutional classifiers	Harm is context-dependent or outside the classifier’s policy
First-token distribution	SafeDecoding, standard safety fine-tuning	Adversary controls or bypasses the opening tokens [ 94 ]
Internal representations	Circuit breakers, latent adversarial training, probe monitors	Harmful computation is not separable in the monitored subspace
Input provenance	Spotlighting, StruQ, SecAlign, instruction hierarchy	Trust boundary is mislabeled, or the model overrides learned precedence [ 28 ]
Perturbation stability	SmoothLLM, erase-and-check	Attack is robust to character-level perturbation, or exceeds the certified size

Table 3: Reviewed defenses classified by the property each relies on, with the condition under which that property stops holding.

Defense layer	Primary Strengths	Key Limitations	Best Suited For
L1: Input/output censorship	Immediate protection; easy deployment; model-agnostic	False positives; latency overhead; bypass via obfuscation	User-facing apps; content moderation
L2: Model training & fine-tuning	Embedded security; improved robustness	High computational cost; requires retraining; potential utility loss	Safety-critical systems; long-term deployments
L3: Adversarial training	Strong attack resistance; adaptive learning	Computationally expensive; may not generalize to novel attacks	Research environments; high-security apps
L4: Monitoring & response	Real-time detection; adaptive responses	Detection accuracy varies; infrastructure complexity	Production environments; enterprise deployments
L5: Infrastructure hardening	System-level protection; prevents extraction attacks	Implementation complexity; hardware dependencies	Edge deployments; proprietary model protection

Table 4: Strengths, limitations, and deployment fit by intervention layer. Resource requirements are given by overhead class in Table 2 .

Metric	Definition	Range (direction)
Attack Success Rate (ASR)	Fraction of adversarial inputs that elicit the targeted unsafe behavior	0–1 (lower better)
Defense Success Rate (DSR)	Fraction of adversarial inputs the defense correctly blocks or neutralizes	0–1 (higher better)
False Positive Rate (FPR)	Fraction of benign inputs incorrectly flagged or refused	0–1 (lower better)
AUPRC	Area under the precision–recall curve for safety classification	0–1 (higher better)
F1	Harmonic mean of precision and recall for harmful-content detection	0–1 (higher better)
Utility preservation	Retained task performance under the defense	higher better

Table 5: Cross-cutting metrics used to evaluate LLM defenses. The last two are defined over a pair of defenses.

Benchmark	Ref	Year	Focus	Composition and notes
AdvBench	[ 149 ]	2023	Harmful-behavior elicitation; suffix attacks	520 harmful behaviors and strings; the standard target set for GCG-style attacks and many defense evaluations
HarmBench	[ 79 ]	2024	Standardized automated red teaming and robust refusal	Common set of harmful behaviors scored by a shared classifier; compares many attacks and defenses head-to-head
SafetyBench	[ 138 ]	2023	Multiple-choice safety knowledge	Thousands of questions across seven safety categories, in English and Chinese
OR-Bench	[ 23 ]	2025	Over-refusal measurement	Around 80,000 prompts that appear toxic but are safe to answer, with a hard subset
JailbreakBench	[ 19 ]	2024	Open jailbreak robustness with a leaderboard	100 behaviors; standardized artifacts and evaluation for both attacks and defenses
StrongREJECT	[ 105 ]	2024	Rigorous jailbreak scoring	Rubric-based autograder that checks whether a jailbreak yields genuinely useful harmful content

Table 6: Established benchmarks used to evaluate LLM defenses and safety.

Figure 2: How the two quantities that trade against each other behave under stacking. (a) Residual attack success of a two-layer stack against the failure correlation

\rho

, at three marginal attack success rates. (b) False refusal rate against stack depth, at three per-layer rates.

	fluent (AutoDAN-style)	GCG
generations / steps	30 generations	60 steps
population	24	–
elite fraction	0.25	–
mutation rate	0.15	–
suffix length	–	20 tokens
top- $k$	–	128

Table 7: Attack configuration. Both adversaries run at a fixed budget with no restarts, against the same targets and the same behavior set. Only the block matching the adversary in use is live in a given run.

Defense	Dependency row	Threshold	Benign block rate
perplexity (window 16)	token surface	4.849077	0.010
token-anomaly	token surface	0.272857	0.010
probe 16	internal repr.	28.924215	0.010
probe 8	internal repr.	110.767708	0.010
Llama Guard 3 8B	semantic	categorical	–
refusal-prime (prefix)	first-token	n/a	–

Table 8: Defense operating points. Thresholded filters are calibrated to block

1\%

of the benign prompts; the realised block rate is

0.010

for all four. Llama Guard, the refusal-priming prefix and SmoothLLM have no scalar threshold.

Prompt style	mean windowed NLL	admitted by the perplexity filter
Plain behavior	3.18	$5/5$
AutoDAN (fluent)	4.47	$5/5$
Compliance template	5.15	$0/5$
GCG suffix	13.56	$0/5$

Table 9: Fluency of four prompt styles and whether the perplexity filter admits them. Mean windowed negative log-likelihood over five probe prompts per style; the admission column counts how many of the five pass the filter at its calibrated threshold.

Figure 3: Residual attack success rate of each defense under the two adaptive adversaries. Vicuna-7B-v1.5,

n=100

Defense pair	Rows	$\hat{p}_{1}$	$\hat{p}_{2}$	$\Delta$	$\phi$ (95% CI)	$\phi/\phi_{\max}$	$q$
Same row (H1)
probe 16 $\times$ probe 8	internal repr.	0.68	0.60	0.172	$\mathbf{0.75}$ [0.62, 0.88]	0.89	$<0.001$
perplexity $\times$ token-anomaly	token surface	0.66	0.35	0.079	$\mathbf{0.35}$ [0.18, 0.50]	0.66	$<0.001$
Cross row (H2)
perplexity $\times$ probe 8	surface $\times$ internal	0.66	0.60	0.144	0.62 [0.45, 0.77]	0.71	$<0.001$
perplexity $\times$ probe 16	surface $\times$ internal	0.66	0.68	0.131	0.59 [0.42, 0.76]	0.62	$<0.001$

Table 10: Measured failure correlation, Vicuna-7B-v1.5 [ 141 ] under the fluent (AutoDAN-style) adversary,

n=100

, StrongREJECT gold.

\Delta

is the excess over the independence prediction. Llama Guard pairs are omitted (Section 11.5 ).

Figure 4: Observed joint breach rate against the independence prediction

\hat{p}_{1}\hat{p}_{2}

for all fifteen measurable pairs. Vicuna-7B-v1.5 under the fluent adversary,

n=100

Figure 5: Failure correlation

\phi

for all fifteen measurable pairs, ordered by effect size, with the behavior bootstrap and the grader-only interval shown separately.

Defense	adaptive (fluent)	static: transfer suffix	static: plain prompt
perplexity	0.66	0.00	0.07
probe 16	0.68	0.28	0.08
refusal-prime	0.60	0.11	0.03
SmoothLLM	0.54	0.04	0.06
token-anomaly	0.35	0.28	0.07

Table 11: Residual attack success under one adaptive and two static adversaries, for the five defenses measurable against all three. The adaptive column is the fluent sentence-level search; the two static columns are a single precomputed GCG suffix reused unchanged, and the unmodified behavior prompt. All gold-judged on the same

n=100

behaviors.

Pair	Rows	$\phi$	crude OR	CMH OR	$p$	$q$
probe 16 $\times$ probe 8	same	0.75	68.0	$\mathbf{158.7}$	$<0.001$	$<0.001$
perplexity $\times$ token-anomaly	same	0.35	6.0	1.39	0.907	0.943
refusal-prime $\times$ SmoothLLM	cross	0.56	12.3	4.34	0.023	0.173
perplexity $\times$ refusal-prime	cross	0.58	14.5	3.95	0.063	0.314
perplexity $\times$ probe 8	cross	0.62	19.1	3.07	0.209	0.562
probe 16 $\times$ refusal-prime	cross	0.45	7.3	0.72	0.943	0.943

Table 12: Difficulty-stratified association for the two same-row pairs and a representative selection of cross-row pairs; all fifteen are in the supplementary material.

Figure 6: Failure correlation for every measurable pair under three labeling regimes: as reported, majority of five grader judgments, and external thresholds.

Defense pair	$Q$	dis.	$DF$	$DF_{\mathrm{ind}}$	$\phi$	$\phi/\phi_{\max}$
perplexity $\times$ probe 16	0.891	0.18	0.580	0.449	0.59	0.62
probe 16 $\times$ probe 8	0.977	0.12	0.580	0.408	0.75	0.89
perplexity $\times$ probe 8	0.909	0.18	0.540	0.396	0.62	0.71
perplexity $\times$ refusal-prime	0.880	0.20	0.530	0.396	0.58	0.66
probe 16 $\times$ refusal-prime	0.769	0.26	0.510	0.408	0.45	0.54
perplexity $\times$ SmoothLLM	0.887	0.22	0.490	0.356	0.57	0.73

Table 13: Pairwise diversity for the fifteen measurable pairs, in the vocabulary of Kuncheva and Whitaker [63] , computed on the correctness indicator and ordered by

DF

. Llama Guard is excluded.

Rule	residual ASR	benign FRR
Combined from independently measured decisions
$k=1$ of 7 (veto)	0.010	0.520
$k=2$ of 7	0.190	0.300
$k=3$ of 7	0.430	0.110
$k=4$ of 7	0.590	0.080
$k=5$ of 7	0.680	0.080

Table 14: Combination rules over the seven defenses. The stack blocks an input when at least

k

members block it.

Defense Category	Primary Threat	Secondary Threat	Key Target Areas
Input/Output Censorship	User-centric	Model-centric	Input validation; response filtering; content moderation
Model Training	Model-centric	Data-centric	Architecture security; safety training; fine-tuning optimization
Adversarial Training	Model-centric	Infrastructure	Defense generation; attack resistance; model hardening
Monitoring Systems	Infrastructure	User-centric	Threat detection; incident response; system monitoring
Infrastructure Hardening	Infrastructure	Data-centric	System security; deployment protection; access control

Table 15: Defensive strategies mapped to threat dimensions.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Acronym	Expansion	Acronym	Expansion
AATM	Adversary Access-Tier Model	GCG	Greedy Coordinate Gradient
ASR	Attack Success Rate	JBB	JailbreakBench
AUPRC	Area Under the Precision–Recall Curve	LAT	Latent Adversarial Training
AUROC	Area Under the ROC Curve	LLM	Large Language Model
CMH	Cochran–Mantel–Haenszel	OR	Odds Ratio
DPO	Direct Preference Optimization	PII	Personally Identifiable Information

Appendix

Table 16: Acronyms used in this paper.

Term	Definition
Backdoor attack	Attack inserting triggers during training that cause malicious outputs when activated.
Blue teaming	Defensive security practices protecting systems from attacks.
Data poisoning	Attack manipulating training data to compromise model behavior.
Diversity	In classifier fusion, the extent to which combined components fail on different inputs; failure correlation is one of its standard pairwise measures.
Failure correlation ( $\phi$ )	Correlation between the breach indicators of two defenses on a common behavior set under a common adversary; the $\rho$ of Eq. ( 6 ).
Guardrails	Safety mechanisms constraining LLM behavior through filtering or validation.

Appendix

Table 17: Technical terms used in this paper.

LLM Security

Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.

See all 327 papers

When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
Jul 27, 2026Tong Zhang, Zexin Li, Simin Chen +1LLM Inference Efficiency Language Model Safety Evaluation
Fudan University · University of California, Riverside · Columbia University
CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation
Sep 18, 2026Jiale Luo, Eric HanJailbreak Defense LLM Jailbreak Attacks
School of Computing National University of Singapore
Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing
Jun 1, 2026Alexandre Cristovão MaioranoLLM Red Teaming AI Agent Security Benchmarks
Lumytics

Jul 27, 2026J/K move · Enter open · S save

When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs

Tong Zhang, Zexin Li, Simin Chen, Yun Peng

Fudan University · University of California, Riverside · Columbia University

LLM Inference Efficiency Language Model Safety Evaluation Jailbreak Defense Refusal Behavior in Language Models

Open paper AskPDF arXiv

Jailbreak defenses are essential for protecting large language models (LLMs), but they can also introduce secondary costs that weaken model utility. We present a systematic study of these defense trade-offs along three dimensions: performance impact, over-refusal on benign inputs, and inference cost. Rather than treating defenses as a single class, we organize them by operational strategy and examine how different strategies correlate with different side-effect profiles. Across state-of-the-art defense methods, widely used benchmark datasets, and representative open-source LLMs, we find that defenses rarely improve downstream capability, but instead vary in how they trade safety gains against usability and efficiency. In particular, rule-based defenses best preserve task performance, highly conservative self-reflective defenses often increase over-refusal, and multi-round defenses incur the largest runtime overhead. These results provide both a benchmark for evaluating defense side effects and practical guidance for selecting defenses under deployment constraints.

Abstract

LLM Security

Explore similar work

When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs

CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation

Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing