Inference-Layer Security: Defending Against Adversarial Inference and Infrastructure Abuse

Keifer Lee

cs.CRSep 29, 2026

Inference-Layer Security: Defending Against Adversarial Inference and Infrastructure Abuse

Authors: Keifer Lee

Organizations: New York Machine Learning Research Guild (NYMLR) New York, NY, USA

Abstract

A Technical Report: Operating a large language model (LLM) as a service requires more than inference infrastructure: the provider must also defend against adversarial interactions that seek to exploit the service, including jailbreaking for harmful use, sophisticated denial of service, and distillation attacks. We study this problem at the inference layer, using a hypothetical frontier lab, Five Elements Inc., as a running example. Because no public labelled dataset of adversarial LLM usage exists, we introduce a structural causal model (SCM) that generates a realistically grounded, labelled dataset of user-sessions, with coordinated multi-account campaigns, platform feedback, and three tiers of label observability. On this dataset we train a practical gradient-boosted detector that classifies each user-session as benign or malicious and, if malicious, by attack type. Against oracle labels the detector very nearly solves the binary task (AUPRC $0.993$ ), yet against the operational labels a real Trust & Safety team would hold, the same model scores an AUPRC of only $0.313$ : the detector is more accurate than the labels used to evaluate it. For attack-type attribution, a naive argmax is dominated by the $98\%$ benign prior (macro-F1 $0.295$ ), whereas a simple thresholded decision engine raises macro-F1 to $0.489$ without sacrificing accuracy. The dataset is publicly released.

Figures & tables

Stage	Attacks at this stage	Defensive levers
baked-in: no runtime lever
Training data corpus curation	data poisoning (250 docs); sock puppets; PII injection	none—corpus is frozen
Training / RLHF pre-train + tune	backdoor via fine-tune; alignment failures; RLHF gaming	none—weights are frozen
structural runtime (deploy / infra) — scope of this paper begins here
Deployment supply chain	LiteLLM/Trivy compromise; SANDCLOCK credential stealer	SBOM; image signing; reproducible builds
Serving infra gateway, isolation	cross-tenant KV leak; credential theft; LLMjacking	tenant isolation; key rotation; TLS pinning; gateway rate limits

Figure 1. The serving stack and where exploits enter at each stage: the model can be poisoned via its training corpus; the infrastructure can be compromised for jailbreaks or data leaks (e.g. leaking system prompts, or other users’ prompts); and the user level can be exploited to serve adversarial results, jailbreak, or distill the model. A table-style diagram of six serving stages, from training data to user session, listing the attacks that fire at each stage and the defensive levers available.

Panel of Figure 2 — Figure 2. The complete Five Elements Inc. serving pipeline, with its intervention (fast) and policy (slow) feedback loops. Box-and-arrow diagram of the request path from users through the service app, serving layer and response, into the transaction database and ETL pipeline, feeding a Trust and Safety team that drives a fast intervention loop back to users and a slow policy loop back to the serving layer.

Panel of Figure 3 — Figure 3. The Ladder of Abstraction: L0 raw event streams $\to$ L7 noisy multi-agent games. Build each rung before climbing. Eight stacked rungs from L0 raw event streams to L7 noisy multi-agent games, with L2 point predictors highlighted as the starting point.

Class	What it is	Routed to	What a miss actually costs
distillation	Capability extraction by competitors	Legal & Policy	a competitor ships your exfiltrated capability
data_extraction	Training-data / PII / system-prompt exfiltration	Legal & Policy	training-data, PII, or system-prompt leakage
jailbreak	Safety bypass (past-tense framing, persona injection)	Trust & Safety	a safety bypass loose in the wild
harmful_use	Extortion, malware, ransomware-as-a-service	Trust & Safety	real-world harm—extortion, malware, RaaS
agentic_misuse	Tool-chain abuse as an attack platform	Trust & Safety	your own tools turned into an attack platform
bot_farm	Coordinated fake-account infrastructure	Platform Integrity	a fake-account fleet scaling unchecked

Table 1. The attack-type taxonomy

\mathcal{C}

: what each class is, which team at Five Elements Inc. a detection is routed to, and what a miss actually costs.

Panel of Figure 4 — Figure 4. The SCM’s generative pipeline. Campaign metadata is sampled, then realized into concrete campaign details, which fan out into $N$ account realizations. Each account acts on the inference service; the service conditions both the accounts and the campaign in return. The “etc.” marks a deliberately simplified slice—the full SCM samples far more per box. Campaign metadata is sampled and realized into campaign details, which fan out into N accounts; the accounts act on an inference service, which conditions the accounts and campaign in return.

distillation	0.30	data_extraction	0.10
harmful_use	0.20	dos	0.07
jailbreak	0.15	multi_surface	0.05
credential_abuse	0.10	bot_farm	0.03

Table 2. Campaign-type prior

\boldsymbol{\theta}

Feature	Distillation (active)	Benign power user	Separation
sessions_per_day	${\sim}4$ /day	${\sim}5$ /day	overlap
n_requests_per_session	${\sim}33$ /session, heavy	${\sim}25$ /session, heavy	overlap
is_api	$95\%$ API	$25\%$ API	separable
avg_inter_request_time_sec	${\sim}3$ s, regular	${\sim}12$ s, varied	partial
std_inter_request_time_sec	${\sim}8$ s jitter, robotic	${\sim}45$ s jitter, bursty	key signal
avg_prompt_perplexity	${\sim}0.0$ , tight	${\sim}0.2$ , looser	overlap

Table 3. Configuration of an active distillation session versus a benign power user.

Panel of Figure 5 — Figure 5. Distillation campaign lifecycle. Setup and warmup emit benign cover traffic, making the account indistinguishable from a casual user; the systematic extraction pattern only emerges from probing onward. Abort can fire from any phase once $70\%$ of the fleet is burned. Phase state machine: setup, warmup, probing, active, cooldown, then success when coverage reaches 0.80; abort from any phase when 70 percent of the fleet is burned.

Influence	Captures	What it carries for distillation
Intervention state	what the platform is currently doing to the account	whether it is rate-limited; its sophistication (which governs how it evades)
Campaign context	the cross-account coordination signals	today’s slot allocation; remaining coverage, $1-\text{cov}$
Trajectory	the account’s own accumulated history	its personal saturation, $\text{covered}/\text{assigned}\in[0,1]$

Table 4. The three influences stitched into each day’s sampling context.

Panel of Figure 6 — Figure 6. A 60-day distillation campaign rollout. Top: campaign phase bands. Middle: fleet composition per tick—active, rate-limited, and burned accounts. Bottom: a single account’s session-density trace, with a rate-limit event mid-campaign and the characteristic ramp from a benign flatline to active extraction. Three aligned panels over 60 days: phase bands, a stacked bar chart of active, rate-limited and burned accounts, and a line of one account's sessions per day rising through the active phase with a dip at a rate-limit event.

M3 : no delay, no noise; M2 : + investigation delay; M1 : + delay + noise.
Channel	Binary	Attack type	Campaign type
		(per account)	(per ring)
M3 oracle	is_adversarial	attack_type	campaign_type
M2 golden	(derived)	label	campaign_label
M1 operational	(derived)	noisy_label	noisy_campaign_label

Table 5. The labelling scheme: purpose

\times

observability.

Account fate	Type	What gets labelled
Suspended / revoked	always-adv.	All sessions backfilled—intentional from day one
Suspended / revoked	not always-adv.	Only the session that triggered suspension—earlier ones were genuinely ambiguous
Active, label lands late	always-adv.	All sessions backfilled once the investigation retroactively confirms them
Active, label lands late	not always-adv.	Only sessions from label_arrival_day onward—preserving temporal honesty
Never confirmed	any	Unlabelled —the adversaries that simply slip through

Table 6. Backfilling rules by account fate and attack type.

Step	Transformation	What it does	Why
1	Clip horizon	Keep the burn-in/burn-out band ( $n_{\text{burn\_in}}{=}40$ to $n_{\text{burn\_out}}{=}320$ )	Drop the cold-start transient (head) and the censored tail whose labels have not arrived
2	Materialise label grid	Fold the noisy_* / label / attack_type channels into M1 / M2 / M3 $\times$ ( is_adversarial , attack_type , campaign_type )	Turn the M3 $\to$ M2 $\to$ M1 hierarchy into concrete train/eval columns
3	Rolling features	Trailing 7-day per-account aggregates ( $W^{(7)}_{u,t}$ , strictly $d_{s}\leq t$ )	Expose behavioural trajectory to the model without leaking the future
4	Select allowed features	Tiers 1–3 (noisy) + geo + cumulative, plus the rolling block	Restrict to production-observable signals; exclude oracle-like surface flags

Table 7. The feature-engineering pipeline at a glance.

Split	M1 (operational)	M2 (golden)	M3 (oracle)
Train	1.53%	0.74%	2.77%
Val	1.35%	0.57%	2.61%
Test	1.47%	0.70%	3.22%

Table 8. Adversarial prevalence per split and label channel.

Class	$n_{k}$	Inv. freq.	+ clamp	+ clamp + $\sqrt{\cdot}$
benign	985,000	0.11	158.7	12.6
bot_farm	4,100	27.1	27.1	5.2
distillation	3,200	34.7	34.7	5.9
jailbreak	2,600	42.7	42.7	6.5
harmful_use	1,400	79.4	79.4	8.9
dos	1,100	101.0	101.0	10.1

Table 9. Per-class weights under the three weighting stages, for an illustrative class distribution consistent with the

{\sim}1.5\%

adversarial train prevalence. Spread = max/min weight.

Channel	Prev.	AUROC	AUPRC	P@1%FPR	R@1%FPR
M1 operational	1.47%	0.842	0.313	0.360	0.379
M2 golden	0.70%	0.994	0.609	0.360	0.801
M3 oracle	3.22%	0.997	0.993	0.767	0.993

Table 10. Binary is_malicious evaluation on the test split, by label channel.

Channel	Top-1	Top-3	Macro-acc	Log-loss	Routing
M1 operational	0.988	0.997	0.194	0.067	0.989
M2 golden	0.997	1.000	0.244	0.021	0.997
M3 oracle	0.983	0.998	0.247	0.058	0.983

Table 11. Multi-class attack-type evaluation under the naive argmax policy.

Panel of Figure 7 — Figure 7. Per-class precision–recall curves for the attack-type GBDT (test split, M3 ), ordered roughly best to worst. A curve that bows toward the top-right corner is separable; note how distillation and jailbreak bow high even though their argmax F1 sits at zero. Nine precision-recall curves, one per attack-type class.

Decision rule	Acc.	Mac.-F1	Mac.-rec.	Routing	$\Delta$ Mac.-F1
argmax (baseline)	0.983	0.295	0.247	0.983	—
absolute lift	0.984	0.324	0.267	0.984	$+0.029$
relative lift	0.987	0.422	0.407	0.989	$+0.127$
benign floor $\to$ argmax	0.984	0.358	0.316	0.985	$+0.063$
benign floor $\to$ abs. lift	0.984	0.321	0.271	0.984	$+0.026$
benign floor $\to$ rel. lift	0.992	0.489	0.520	0.996	$+0.194$

Table 12. Decision engines on the test split ( M3 ); thresholds tuned on validation ( M1 ).

Good ranking, compatible with argmax
		F1 by decision engine
Class	PR-AUC	argmax	abs. lift	rel. lift	floor $\to$ argmax	floor $\to$ abs.	floor $\to$ rel.	meta
benign	1.000	0.992	0.992	0.995	0.993	0.992	1.000	0.996
dos	0.661	0.616	0.592	0.561	0.571	0.585	0.513	0.381
bot_farm	0.801	0.475	0.641	0.444	0.695	0.667	0.379	0.634
credential_abuse	0.615	0.451	0.370	0.289	0.608	0.426	0.337	0.509

Table 13. Per-class performance on the test split ( M3 ): one-vs-rest PR-AUC (ranking) and F1 under each decision engine. Grouped by whether ranking and decision agree. Best F1 per class in bold.

Panel of Figure 8 — Figure 8. Global feature importance for the attack-type GBDT: mean $|\mathrm{SHAP}|$ over a 200k-row test sample. Horizontal bar chart of mean absolute SHAP values for the fifteen most important features, led by the seven-day rolling mean request count.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Attack type	Phase	Day	Req.	API	Avg IRT	Std IRT	QSE	Safety	Cov. $\Delta$	Soph.	Label ( M2 )
benign	dormant	9	6	–	22.74	14.94	0.214	0.001	0.000	0.00	—
bot_farm	active	104	17	✓	2.15	0.67	0.069	0.005	0.000	0.12	bot_farm
dos	active	1	382	✓	0.49	0.43	0.014	0.035	0.039	0.60	dos
credential_abuse	exploitation	36	26	✓	8.37	2.92	0.747	0.079	0.000	0.65	credential_abuse
distillation	active	27	28	✓	3.84	15.53	0.069	0.002	0.000	0.40	—
jailbreak	cooldown	59	5	–	30.10	11.30	0.693	0.028	0.000	0.84	jailbreak

Appendix

Table 14. A stratified sample of raw user-session records (oracle M3 fields included). IRT = inter-request time (s); QSE = query-structure entropy; cov.

\Delta

= domain coverage delta.

LLM Security

Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.

See all 322 papers

Feb 21, 2026

MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs

Defending LLMs against adversarial jailbreak attacks remains an open challenge. Existing defenses rely on binary classifiers that fail when adversarial input falls outside the learned decision boundary, and repeated fine-tuning is computationally expensive while potentially degrading model capabilities. We propose MANATEE, an inference-time defense that uses density estimation over a benign representation manifold. MANATEE learns the score function of benign hidden states and uses diffusion to project anomalous representations toward safe regions--requiring no harmful training data and no architectural modifications. Experiments across Mistral-7B-Instruct, Llama-3.1-8B-Instruct, and Gemma-2-9B-it demonstrate that MANATEE reduce Attack Success Rate by up to 100% on certain datasets, while preserving model utility on benign inputs.

Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav +4

Jailbreak Defense LLM Safety Adversarial Robustness of Language Models Adversarial Robustness

Ask PDF

Aug 17, 2026

Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures

As Large Language Models (LLMs) move from conversational assistants to advanced agentic systems, guardrail failures can convert adversarial intents into harmful executions. However, most evaluation frameworks focus only on whether a user request is unsafe. This is insufficient for multi-turn failures, where adversarial intent may be distributed across turns and emerge only from accumulated context. This motivates us to go beyond detection to localize the evidence that pushes the conversation toward unsafe trajectories. We introduce MTAG-Bench, a multi-turn guardrail attribution dataset with behavioral validation and counterfactual supervision containing 4,167 conversations. We also introduce MTAG-Probe, a hierarchical attribution model for turn- and span-level evidence localization with conversation-level safety detection. MTAG-Probe improves turn-level F1 by 28.1 and 24.1 points (79.0% and 61.0% relative) over self-attribution by the model that produced the unsafe response and the LLM safety-judge attribution, respectively. Removing MTAG-Probe's top-ranked evidence within budgets of 15% and 20% of user words per conversation changes 50.6% and 60.8% of Llama Guard decisions on Geometry from unsafe to safe, respectively. In full-trajectory replay, MTAG-Probe converts 84.3% of eligible unsafe final responses to safe. These results show that MTAG-Probe can provide evidence-level explanations as a practical complement to safety detection.

Srinivasan Subramanian, Kazi Aminul Islam, Md. Abdullah Al Hafiz Khan

Department of Computer Science, Kennesaw State University, Marietta, GA, USA

LLM Safety LLM Safety Evaluation LLM Safety Benchmarks Faithfulness of Language Model Explanations Adversarial Attacks on LLMs

Ask PDF

Jun 1, 2026

Gate AI: LLM Security Benchmark Evaluation Methodology and Results

Published evaluations of prompt-injection and jailbreak detectors for Large Language Models often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points. We describe an evaluation harness that addresses both. The detector under evaluation is scored across 16 public benchmarks (12,111 samples) using 5-fold cross-validation. StratifiedKFold (by row) is the headline pass; a parallel StratifiedGroupKFold pass over a composite key (parent-prompt id plus MinHash + LSH near-duplicate clusters at Jaccard

\gtrsim 0.8

) runs alongside it as a leakage-premium diagnostic. A single global operating point is selected on the held-out folds (max F1 subject to FPR

\leq 1\%

) and applied uniformly to every dataset, so per-dataset results reflect one threshold rather than per-benchmark optimisation. Generalisation is examined through a battery of diagnostics (leave-one-dataset-out cross-validation, a random-label control, adversarial validation, permutation feature importance, length-bias correlation, classifier-head agreement, cross-source near-duplicate detection, threshold transferability, train-vs-OOF agreement, and a paraphrase-invariance probe), most with a quantitative pass threshold and the remainder with a stated failure mode. For every external comparison, the detector's threshold is re-tuned to the competitor's published false-positive rate so head-to-head values are evaluated at matched operating points.

Ryle Goehausen, Marcus Sousa

LLM Safety Benchmarks Language Model Safety Evaluation LLM Security Jailbreak Detection Prompt Injection Attacks on LLMs Prompt Injection Detection

Ask PDF

Abstract

LLM Security

Explore similar work

MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs

Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures

Gate AI: LLM Security Benchmark Evaluation Methodology and Results