On the Effectiveness of Kernel-Level Evidence for Agent Security

Spencer King; Zhilu Zhang; Mikhail Kuznetsov; Kay Liu; Baris Coskun; Wei Ding

cs.CRSep 24, 2026

On the Effectiveness of Kernel-Level Evidence for Agent Security

Authors: Spencer King, Zhilu Zhang, Mikhail Kuznetsov, Kay Liu, Baris Coskun, Wei Ding

Organizations: University of Georgia · Amazon Web Services

Abstract

LLM agents are deployed into infrastructure that grants them broad host authority, yet existing agent-security benchmarks and defenses operate almost exclusively at the application telemetry layer: the served tool manifest, the user prompt, and the model's messages. Some threats, however, smuggle malicious instructions and actions past the application boundary, leaving them invisible to that layer. In this work, we bridge that gap by pairing application-level agent telemetry with kernel-level syscall traces to present the first paired-evidence characterization of kernel-level versus application-layer signal for agent security. To quantify the value of the enhanced telemetry, we introduce Agent Cross-Layer Evidence (ACE), a paired-session corpus of 4,047 sessions and 17 threat models spanning six delivery-vector families and 14 of the 25 OWASP LLM and agentic threat categories, organized into 12 attack mechanics with per-mechanic characterization of where the most discriminative evidence lies. Across four distinct detector families, we find that kernel evidence is discriminative on its own and that composing it with application-layer evidence generally outperforms either single-layer view, revealing complementary signals that single-layer analyses can miss. We further demonstrate generalization to unseen attack families and transfer to an alternate agent runtime. Together, these findings establish the value of cross-layer evidence for agent security.

Figures & tables

Figure 1: Cross-View detection on ACE. Each session is configured, captured in-container with a strace kernel-trace sensor, emits three paired artifacts, and is scored across the App-View / Kernel-View / Cross-View by models from the three groups shown (ML/DL, LLM, fine-tuned SLM), which comprise the four detector families of § 3 .

Mechanic	OWASP Coverage	Fold	ACE	MSB	MCPT	ToB	AgD	ASB	RC	AgH	TE
Bash command injection	L1, L6, A2, A11	A	$\CIRCLE$	$\Circle$	$\Circle$	$\Circle$	$\Circle$	$\LEFTcircle$	$\LEFTcircle$	$\Circle$	$\LEFTcircle$
Credential direct read	L2, A2, A3 ∗	B	$\CIRCLE$	$\Circle$	$\CIRCLE$	$\LEFTcircle$	$\Circle$	$\LEFTcircle$	$\CIRCLE$	$\Circle$	$\LEFTcircle$
Silent exfil via logging	L2, A2	B	$\CIRCLE$	$\Circle$	$\Circle$	$\LEFTcircle$	$\Circle$	$\Circle$	$\Circle$	$\Circle$	$\Circle$
Resource exhaustion	L10, A4	E	$\CIRCLE$	$\Circle$	$\Circle$	$\Circle$	$\Circle$	$\Circle$	$\LEFTcircle$	$\Circle$	$\Circle$
Audit-log poisoning	A1, A8	F	$\CIRCLE$	$\Circle$	$\Circle$	$\Circle$	$\LEFTcircle$	$\LEFTcircle$	$\Circle$	$\Circle$	$\Circle$
Schema-shape tampering	L3	G	$\CIRCLE$	$\LEFTcircle$	$\LEFTcircle$	$\CIRCLE$	$\Circle$	$\Circle$	$\Circle$	$\Circle$	$\Circle$

Table 1: The 12 attack mechanics in ACE and their prior-benchmark coverage. Fold : held-out fold letter (A–G) in the OWASP-aligned OOD design. OWASP Coverage : anchoring categories from OWASP LLM Top 10 v2025 (L

x

) and Agentic AI Threats and Mitigations v1.0 (A

x

), defined by full OWASP name in Appendix E.6 ;

\ast

= partial mapping. Right nine columns rate each benchmark’s coverage of the mechanic:

\CIRCLE

= Direct (evaluable, 1:1 named category);

\LEFTcircle

= Partial (overlapping category at a different abstraction level, e.g., a benchmark naming “prompt injection” broadly without separating description-poisoning from response-embedded variants);

\Circle

= Absent. Column-header abbreviations: MSB = MCP Security Benchmark, MCPT = MCPTox, ToB = Trail of Bits, AgD = AgentDojo, ASB = Agent Security Benchmark, RC = RedCode, AgH = AgentHarm, TE = ToolEmu.

Mechanic	App	Kernel	Cross	Signal	$n_{+}$	$n_{-}$
Bash command injection	.812 $\pm$ .051	.846 $\pm$ .046	.915 $\pm$ .027	CKA	264	311
Credential direct read	.876 $\pm$ .051	.882 $\pm$ .039	.978 $\pm$ .012	CKA	227	529
Silent exfil via logging	.586 $\pm$ .045	.976 $\pm$ .011	.970 $\pm$ .010	KC	244	469
Resource exhaustion	.915 $\pm$ .044	.939 $\pm$ .027	.973 $\pm$ .016	CKA	109	309
Audit-log poisoning	.608 $\pm$ .109	.932 $\pm$ .049	.954 $\pm$ .039	CK	11	108
Schema-shape tampering	.672 $\pm$ .071	.844 $\pm$ .051	.867 $\pm$ .032	CK	85	214

Table 2: Per-mechanic ID AUROC across 10 detectors. Mean of per-detector AUROCs computed from pooled held-out ID predictions (5-fold stratified) for the 10 non-Falco detectors of Table 8 , under the canonical positive-class labeling (§ 3.1 ) with matched-benign-per-host aggregation.

\pm

is the mean patch-clustered bootstrap SE (1,000 resamples, fixed seed) averaged across detectors, not the SE of the mean. Winning view per row in bold (before rounding). Signal lists the views (A=App, K=Kernel, C=Cross) clearing the acceptable-discrimination threshold

\alpha{=}0.75

(Appendix H.2 ) in descending AUROC order;

n_{+}

n_{-}

are per-mechanic positive and matched-negative session counts before detector-specific scoring exclusions.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Threat model (delivery vector)	Mechanic	OWASP	Fold
MCP tool tampering	All 12 mechanics	(all)	A–G
File $\to$ bash injection (built-in)	Bash command injection	LLM01+06	A
Web $\to$ bash injection (built-in)	Bash command injection	LLM01+06	A
User-direct $\to$ bash injection (built-in)	Bash command injection	LLM01+06	A
Memory $\to$ bash injection (built-in)	Bash command injection	LLM01+06	A
File $\to$ credential exfil (built-in)	Credential direct read	LLM02	B

Appendix

Table 3: The 17 threat models in ACE mapped onto the 12 attack mechanics and 7 OWASP-aligned held-out folds. mcp_tool_tampering contributes fixtures to every fold; the 16 built-in-tool threat models each map to a single mechanic. Three built-in threat models captured zero malicious sessions (agent-side refusal ceiling) and are kept in every training pool as pure-negatives.

Fold	Held-out attack family	Mal.	Dorm.	Lat.	Benign	Scored	All
A	Bash command injection	264	59	225	252	575	800
B	Credential exfil (direct read + silent logging + network redirect)	508	366	72	603	1,477	1,549
C	Prompt injection (metadata + retrieval reference + preference manipulation)	195	32	18	180	407	425
D	Output directive write	162	68	20	167	397	417
E	Resource exhaustion	109	188	13	121	418	431
F	Persistence (audit-log poisoning + multi-stage persistence)	35	51	4	100	186	190

Appendix

Table 4: Session-label counts per held-out fold under the 7-fold OWASP-aligned OOD design. Malicious = malicious variant ran and the fire predicate matched on the captured evidence (

y{=}1

). Dormant = malicious variant ran but the target tool was never exercised (

y{=}0

). Latent in this table is signal-bearing Latent — malicious variant ran and the target tool was exercised but the fire predicate did not match — dropped from scoring as structurally ambiguous. Signature-free Latent (66 sessions corpus-wide, all in the fold-Z sensor-blind-spot control) is out of the A–G scoring cohort by design and does not appear in this table. Benign counts reflect matched-benign-per-host pairing. Scored = Malicious + Dormant + Benign; All = Scored + Latent. Rows appearing in multiple folds (typically Benign sessions on hosts that contributed attacks to more than one OWASP family) are counted once per fold, so the grand total (

4{,}115

) exceeds the number of unique corpus sessions covered by A–G (

3{,}626

4{,}047

Family	Detector	View	F	FL	FLD	$\Delta_{\text{FL}-\text{F}}$	$\Delta_{\text{FLD}-\text{F}}$
Classical	AdaBoost	App	0.689	0.699	0.560	$+0.010$	$-0.130$
	AdaBoost	Kernel	0.733	0.717	0.553	$-0.016$	$-0.179$
	AdaBoost	Cross	0.840	0.827	0.681	$-0.013$	$-0.159$
	XGBoost	App	0.703	0.686	0.536	$-0.018$	$-0.167$
	XGBoost	Kernel	0.750	0.730	0.580	$-0.019$	$-0.169$
	XGBoost	Cross	0.850	0.830	0.709	$-0.020$	$-0.141$

Appendix

Table 5: Mean OOD AUROC across the 7 OWASP-aligned folds (A–G) for every trained detector, under three positive-class definitions. Detectors, views, and pipelines match Table 8 : F column AUROCs recover the OOD side of Table 8 exactly (to three decimals), so absolute values are comparable across the two tables.

\Delta_{\text{FL}-\text{F}}

and

\Delta_{\text{FLD}-\text{F}}

are the deltas relative to F. AUROC changes are smaller under FL than under FLD. Per-cell clustered-bootstrap SE is on the same scale as Table 8 (typically

\pm 0.04

–

0.08

), omitted here for readability.

	p50		p90		p99
Feature	Benign	Malicious	Benign	Malicious	Benign	Malicious
execve	7	7	24	26	63	23,453
clone	30	30	56	91	420	23,493
connect	29	31	51	67	259	93,907
sendto	24	28	75	74	241	251
distinct external IPs	11	11	13	20	40	41

Appendix

Table 6: Per-feature quantile comparison, Benign (

n{=}1{,}225

unique sessions) vs. Malicious (

n{=}1{,}358

unique sessions) on ACE, with class definitions from Appendix E.3 . External-IP count excludes loopback, link-local, and multicast ranges (Appendix G.1 ). Medians are near-identical (within 4 counts on six of seven features,

{\sim}6\%

relative on openat ); separation is a tail phenomenon.

ID	Category	Cov.	Note
OWASP LLM Top 10 v2025
LLM01	Prompt Injection	✓	Metadata + retrieval-reference injection
LLM02	Sensitive Information Disclosure	✓	Credential read + silent exfil + network redirect
LLM03	Supply Chain	✓	Schema-shape tampering; tampered-MCP delivery
LLM04	Data and Model Poisoning	$\times$	Training-pipeline attack, not runtime
LLM05	Improper Output Handling	✓	Output directive write

Appendix

Table 7: OWASP coverage of ACE.

\checkmark

= represented (at least one mechanic in Table 1 instantiates an attack under the category);

\sim

= partial coverage (OS-observable slice only, see notes);

\times

= out of scope for ACE’s attack fixtures. Full definitions in the prose above.

	ID			OOD
Model	App	Kernel	Cross	App	Kernel	Cross
Rule-based runtime baseline (Falco default rule pack)
Falco	—	.565 $\pm$ .039	—	—	.572 $\pm$ .084	—
Classical
AdaBoost	.872 $\pm$ .022	.905 $\pm$ .020	.975 $\pm$ .007	.689 $\pm$ .075	.733 $\pm$ .090	.840 $\pm$ .059
XGBoost	.922 $\pm$ .016	.931 $\pm$ .015	.988 $\pm$ .004	.703 $\pm$ .076	.750 $\pm$ .081	.850 $\pm$ .059

Appendix

Table 8: AUROC detector benchmark on ACE with mean per-fold clustered bootstrap SE. Mean AUROC across folds (5 stratified ID, 7 OWASP-aligned OOD),

\pm

is the mean per-fold patch-clustered bootstrap standard error (§ 3.3 ). App / Kernel / Cross denote the evidence slice, and each detector family consumes that slice under its own representation (Appendix F ). LLM-judge rows use the OWASP-grounded prompt with 14 references (

K{=}1

), fixed for ID and excluding the test family for OOD (Appendix J.3 ). The full per-model 4-configuration LLM sweep is in Table 17 in Appendix J.1 . Qwen OOD cells use a common query cohort across the full prompt sweep (Appendix J.3 ). Winning view per (model, regime) in bold.

	ID			OOD
Model	App	Kernel	Cross	App	Kernel	Cross
Classical
AdaBoost	.818 $\pm$ .035	.867 $\pm$ .031	.963 $\pm$ .011	.571 $\pm$ .125	.626 $\pm$ .140	.773 $\pm$ .111
XGBoost	.885 $\pm$ .026	.907 $\pm$ .022	.983 $\pm$ .006	.589 $\pm$ .127	.661 $\pm$ .131	.797 $\pm$ .099
Deep tabular
TabPFN	.905 $\pm$ .022	.935 $\pm$ .017	.982 $\pm$ .007	.498 $\pm$ .131	.692 $\pm$ .126	.822 $\pm$ .103

Appendix

Table 9: Average precision (AP) detector benchmark on ACE with mean per-fold clustered bootstrap SE. Companion to Table 8 at ACE’s positive base rate (

\approx 37\%

on the ID cohort; per-fold OOD mean

\approx 35\%

). Mean AP across folds (5 stratified ID, 7 OWASP-aligned OOD),

\pm

is the mean per-fold patch-clustered bootstrap standard error under the same convention as Table 8 . Views, protocols, fold designs, and label policy are identical to Table 8 (§ 3.1 ), over the 10 non-Falco detectors; the rule-based Falco baseline is reported on AUROC only, and its native-threshold precision and recall are given directly in Appendix L.2.2 . Winning view per (model, regime) in bold.

Mechanic	$\alpha{=}0.70$	$\alpha{=}0.75$	$\alpha{=}0.80$
Bash command injection	CKA	CKA	CKA
Credential direct read	CKA	CKA	CKA
Silent exfil via logging	KC	KC	KC
Resource exhaustion	CKA	CKA	CKA
Audit-log poisoning	CK	CK	CK
Schema-shape tampering	CK	CK	CK

Appendix

Table 10: Signal-label sensitivity to the discrimination threshold

\alpha

. Labels recomputed from the mean AUROCs of Table 2 at

\alpha\in\{0.70,0.75,0.80\}

;

\alpha{=}0.75

is the “acceptable discrimination” anchor used in Table 2 . Lowering

\alpha

0.70

adds Kernel for Retrieval reference injection and Preference manipulation. Raising it to

0.80

removes both single-layer views for Metadata prompt injection and Network redirect exfil. Every mechanic retains Cross at all three thresholds; Preference manipulation is Cross-only at

0.75

and

0.80

	Full cohort			Captured-prompt subset
Model	App	Kernel	Cross	App	Kernel	Cross
Qwen3-235B	.705	.775	.829	.613	.803	.809
Qwen3-80B	.678	.725	.810	.608	.698	.791
Qwen3-32B	.672	.741	.723	.581	.766	.661
Qwen2.5-3B	.713	.901	.881	.716	.914	.935
Qwen2.5-7B	.722	.882	.920	.736	.941	.968

Appendix

Table 11: OOD performance on sessions with captured initial user prompts. Mean AUROC across seven families, comparing the full scoring cohorts with their captured-prompt subsets. Qwen judges use the OWASP-grounded, family-held-out

K{=}1

protocol. SLMs use their fold-specific LoRA adapters. App / Kernel / Cross denote the three evidence views.

	Cross minus App		Cross minus Kernel
Detector	$\Delta$	95% interval	$\Delta$	95% interval
AdaBoost	$+.150$	$[+.101,+.209]$	$+.107$	$[+.052,+.165]$
XGBoost	$+.147$	$[+.083,+.217]$	$+.101$	$[+.043,+.157]$
TabPFN	$+.241$	$[+.167,+.315]$	$+.097$	$[+.031,+.161]$
TabICL	$+.166$	$[+.112,+.228]$	$+.038$	$[-.029,+.098]$
Qwen3-235B	$+.125$	$[+.075,+.175]$	$+.054$	$[-.002,+.098]$

Appendix

Table 12: Paired OOD AUROC differences across all four detector families. Positive differences favor Cross-View. Qwen judges use OWASP-grounded, family-held-out references; SLMs use fold-specific adapters. Intervals use paired cluster resampling and are not adjusted for multiple comparisons.

	ID			OOD
Model	App	Kernel	Cross	App	Kernel	Cross
Train-from-scratch
MLP (tuned)	.864 $\pm$ .022	.781 $\pm$ .032	.897 $\pm$ .018	.721 $\pm$ .091	.629 $\pm$ .094	.664 $\pm$ .089
FT-Transformer	.883 $\pm$ .020	.796 $\pm$ .029	.900 $\pm$ .018	.694 $\pm$ .089	.630 $\pm$ .087	.670 $\pm$ .095
Pretrained tabular foundation models (in-context)
TabPFN	.933 $\pm$ .016	.952 $\pm$ .012	.986 $\pm$ .006	.633 $\pm$ .103	.777 $\pm$ .086	.874 $\pm$ .071

Appendix

Table 13: Tabular DL ablation on ACE. Mean AUROC across folds (5 stratified for ID, 7 OWASP-aligned for OOD);

\pm

is the mean per-fold patch-clustered bootstrap standard error (§ 3.3 ). MLP and FT-Transformer are train-from-scratch; TabPFN and TabICL are pretrained tabular foundation models scored in-context with no per-corpus training. Winning view per (model, regime) in bold.

Hidden layers	Dropout	Weight decay	AUROC
$\mathbf{(256,128,64,32)}$	$\mathbf{0.4}$	$\mathbf{10^{-3}}$	$\mathbf{0.9027}$
$(256,128,64,32)$	$0.4$	$10^{-4}$	$0.9027$
$(512,256,128)$	$0.4$	$10^{-3}$	$0.9014$
$(128,64)$	$0.2$	$10^{-3}$	$0.9004$
$(128,64)$	$0.2$	$10^{-4}$	$0.9003$
$(256,128,64)$	$0.4$	$10^{-3}$	$0.8997$

Appendix

Table 14: MLP grid search on ACE stratified fold_0 (Cross-View). All 16 configurations sorted by AUROC. Selected configuration in bold; ties broken by earlier appearance in the grid.

$n_{\text{blocks}}$	$d_{\text{block}}$	Dropout	AUROC
$\mathbf{2}$	$\mathbf{96}$	$\mathbf{0.1}$	$\mathbf{0.9203}$
$3$	$96$	$0.1$	$0.9185$
$2$	$96$	$0.2$	$0.9119$
$3$	$96$	$0.2$	$0.9118$
$4$	$96$	$0.2$	$0.9088$
$4$	$96$	$0.1$	$0.9069$

Appendix

Table 15: FT-Transformer grid search on ACE stratified fold_0 (Cross-View). All 12 configurations sorted by AUROC. Selected configuration in bold.

	Raw ordered event stream				Summary
Fold	LSTM-LM (Kim 2016)	BERT-MIL (windows)	Set Transformer (tokens)	Set Transformer (events)	BERT-on- summary
A	.560 $\pm$ .107	.519 $\pm$ .114	.530 $\pm$ .104	.512 $\pm$ .104	.752 $\pm$ .099
B	.545 $\pm$ .087	.523 $\pm$ .081	.574 $\pm$ .072	.532 $\pm$ .076	.853 $\pm$ .046
C	.448 $\pm$ .104	.491 $\pm$ .105	.538 $\pm$ .075	.532 $\pm$ .091	.606 $\pm$ .089
D	.680 $\pm$ .210	.472 $\pm$ .142	.768 $\pm$ .115	.652 $\pm$ .235	.978 $\pm$ .021
E	.585 $\pm$ .153	.729 $\pm$ .100	.483 $\pm$ .192	.677 $\pm$ .153	.877 $\pm$ .066

Appendix

Table 16: Sequence-DL detector ablation on ACE OWASP-aligned OOD folds. Per-fold AUROC

\pm

patch-clustered bootstrap SE (1,000 resamples). All cells use the paper’s canonical AUROC scoring cohort (§ 3.1 ): signal-bearing latent sessions dropped from both training and scoring, matching Table 8 . The four raw-stream detectors consume the ordered syscall event stream, and are ordered left to right by mean OOD AUROC (which coincides with grouping the two sequence-architecture detectors ahead of the two set-architecture ones). BERT-on-summary consumes the R1 JSON rendering from Appendix J.2 . Mean row reports the arithmetic mean of per-fold point estimates with the fold-to-fold standard deviation.

	ID			OOD
Model	App	Kernel	Cross	App	Kernel	Cross
Baseline template, $K{=}0$ (zero-shot)
Qwen3-235B	.607 $\pm$ .032	.581 $\pm$ .042	.541 $\pm$ .026	.590 $\pm$ .061	.589 $\pm$ .085	.520 $\pm$ .041
Qwen3-80B	.608 $\pm$ .035	.516 $\pm$ .019	.540 $\pm$ .031	.594 $\pm$ .077	.513 $\pm$ .018	.512 $\pm$ .067
Qwen3-32B	.605 $\pm$ .040	.593 $\pm$ .024	.534 $\pm$ .035	.573 $\pm$ .073	.564 $\pm$ .040	.502 $\pm$ .061
OWASP-grounded template, $K{=}0$ (zero-shot)

Appendix

Table 17: Full LLM-judge sweep on ACE with mean per-fold clustered bootstrap SE. Mean AUROC across folds (5 stratified for ID, 7 OWASP-aligned for OOD);

\pm

is the mean per-fold patch-clustered bootstrap standard error (§ 3.3 ). Every LLM judge is expanded into all four (prompt template

\times

shot count) combinations: Baseline = task instruction + evidence + response format; OWASP-grounded = baseline plus OWASP-informed attack descriptions and observable signature examples.

K{=}0

= zero-shot;

K{=}1

= 14 labeled references, fixed across ID folds and selected from each OOD training fold with the test family excluded. OOD cells use a common query cohort across all models, templates, views, and shot counts. Reference sessions are excluded from scoring; see Appendix J.3 for the full protocol statement. Winning view per (model, regime) within each configuration block in bold.

	R1	R2	R3	R4
Model	(bounded summary)	(dedup verbatim)	(family tokenized)	(chronological verbatim)
Qwen3-235B	0.731	0.541	0.671	0.599
Qwen3-80B	0.598	0.628	0.575	0.563
Qwen3-32B	0.679	0.606	0.508	0.587
mean	0.669	0.592	0.585	0.583

Appendix

Table 18: Kernel-View representation ablation on ACE. Pooled ACE-full AUROC under OWASP-grounded

K{=}0

on the paper’s canonical AUROC cohort (§ 3.1 ;

n{=}3{,}651

after dropping

396

signal-bearing latent sessions). Winner per row in bold. R1 is the paper’s canonical Kernel-View rendering; R2/R3/R4 are three alternative renderings that consume the same underlying strace event stream. Oversize and parse-fail rows are excluded from AUROC.

Model	App	Kernel	Cross
Baseline template
Qwen3-235B	.695 $\pm$ .065	.780 $\pm$ .068	.831 $\pm$ .049
Qwen3-80B	.688 $\pm$ .070	.812 $\pm$ .053	.771 $\pm$ .053
Qwen3-32B	.660 $\pm$ .079	.743 $\pm$ .068	.693 $\pm$ .069
OWASP-grounded template
Qwen3-235B	.730 $\pm$ .069	.772 $\pm$ .061	.842 $\pm$ .045

Appendix

Table 19: Detection with reference examples covering all attack families. Mean AUROC across seven family cohorts on the common query set used for the reference-protocol comparison. Each judge receives 14 fixed references and undergoes no parameter updates.

\pm

is mean per-fold patch-clustered bootstrap SE (1,000 resamples, seed 0). Winning view per row in bold.

Support set	Macro AUROC	Pooled AUROC
Zero-shot ( $K{=}0$ )	.634	.630 $\pm$ .026
Canonical ( $K{=}1$ )	.842	.831 $\pm$ .022
Random seed 1	.845	.847 $\pm$ .020
Random seed 2	.842	.833 $\pm$ .021
Random seed 3	.837	.847 $\pm$ .020

Appendix

Table 20: Exemplar-selection robustness for Qwen3-80B OWASP-grounded Cross-View. All configurations score the same 3,579-session cohort, excluding the support union and common-score omissions. Macro AUROC is the unweighted mean over the seven OWASP-aligned folds A–G. Pooled AUROC also retains eligible negative-only H/Z pool controls. Its

\pm

values are patch-clustered bootstrap SEs from 1,000 resamples with seed 0, not uncertainty across reference draws. These matched-cohort results have a different denominator from the LLM prompt comparison in Table 17 and the reference-protocol comparison in Table 19 .

Training	Evaluation	AUROC [95% interval]
Original	Original	.922 $[.887,.958]$
Original	Redacted	.620 $[.551,.690]$
Redacted	Redacted	.759 $[.686,.836]$

Appendix

Table 21: Llama-3.1-8B Kernel-View under identifier redaction. Mean AUROC across seven held-out families, with pointwise 95% cluster-bootstrap intervals for each condition.

Detector	View	Native (recall @ FPR)	@1%	@5%	@10%
Cisco mcp-scanner (YARA)	Manifest	0.159 @ 16.3% FPR	—	—	—
XGBoost	Cross	0.922 @ 3.8% FPR	0.862	0.936	0.970
TabPFN	Cross	0.917 @ 4.0% FPR	0.856	0.923	0.978
Qwen3-80B $K{=}1$	Cross	0.732 @ 7.3% FPR	0.007	0.007	0.733
Llama-3.1-8B LoRA	Kernel	0.981 @ 2.6% FPR	0.822	0.993	1.000

Appendix

Table 22: Scanner vs. ACE detectors on the tampered-MCP subset (

n{=}1{,}871

; signal-bearing latent dropped per § 3.1 ). Recall on 731 confirmed-effect malicious sessions and FPR on 1,140 non-fired sessions (after dropping 22 signal-bearing latent sessions per Appendix E.3 ). The Native column reports each detector at its native malicious-verdict threshold (see § L.1.1 ). The @1% , @5% , and @10% columns report recall under those false-positive ceilings using strict score cutoffs on this evaluation cohort. Ties can place achieved FPR below the named ceiling. Cisco mcp-scanner produces a binary rule-fires flag with no threshold knob and is reported at its native operating point only.

	ACE catches	ACE misses
Scanner catches	108	8
Scanner misses	566	49

Appendix

Table 23: Scanner vs. XGBoost Cross-View (Classical). Joint verdicts on the 731 confirmed-effect malicious tampered-MCP sessions. ACE catches 566 of the 615 scanner-misses (

92.0\%

complementary coverage);

49

731

(

6.7\%

) sessions evade both defenses.

	ACE catches	ACE misses
Scanner catches	106	10
Scanner misses	564	51

Appendix

Table 24: Scanner vs. TabPFN Cross-View (Deep tabular). Joint verdicts on the 731 confirmed-effect malicious tampered-MCP sessions. ACE catches 564 of the 615 scanner-misses (

91.7\%

complementary coverage);

51

731

(

7.0\%

) sessions evade both defenses.

	ACE catches	ACE misses
Scanner catches	75	41
Scanner misses	460	155

Appendix

Table 25: Scanner vs. Qwen3-80B

K{=}1

(LLM, Cross-View). Joint verdicts on the 731 confirmed-effect malicious tampered-MCP sessions. Qwen catches 460 of the 615 scanner misses (

74.8\%

complementary coverage). The remaining 155 of 731 attacks (

21.2\%

) evade both.

	ACE catches	ACE misses
Scanner catches	115	1
Scanner misses	602	13

Appendix

Table 26: Scanner vs. Llama-3.1-8B LoRA (SLM, Kernel-View). Joint verdicts on the 731 confirmed-effect malicious tampered-MCP sessions. ACE catches 602 of the 615 scanner-misses (

97.9\%

complementary coverage);

13

731

(

1.8\%

) sessions evade both defenses.

Detector	A	B	D	E	F	avg
XGBoost Cross-View	.941	.718	.995	.806	.750	.842
TabPFN Cross-View	.487	.418	1.000	.872	.535	.662
Qwen3-80B $K{=}1$ Cross-View	1.000	.993	.995	1.000	1.000	.998
Llama-3.1-8B LoRA Kernel	1.000	.605	.879	.961	.801	.849

Appendix

Table 27: Per-fold cross-agent generalization AUROC on ACE-XA. Four detector families score 577 unique sessions from 592 captures, excluding 15 latent sessions. The 208 pure-benign sessions are reused across folds, yielding 1,409 session/fold predictions per detector. Columns are the five OWASP folds ACE-XA covers and their unweighted mean. Winners are bolded before rounding. ACE-XA replaces both the agent LLM and scaffold.

Model	Responses	Input (M)	Output (M)	Cost
Qwen3-235B-A22B-2507	70,542	536.8	4.02	$122
Qwen3-Next-80B-A3B	70,542	536.8	4.56	$86
Qwen3-32B	70,302	522.5	3.86	$81
Total	211,386	1,596.2	12.43	$288

Appendix

Table 28: Estimated LLM-judge sweep cost by model. Response and token counts are provider-reported; dollar figures use the September 2026 model and region rates in § N.2 .

Explore similar work

Apr 25, 2026cs.CR

A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework

Agentic AI systems introduce a security surface that is qualitatively different from that of stateless LLMs. They persist memory, invoke external tools, coordinate with peer agents, and operate across sessions, allowing attacks to emerge not only at the prompt interface but also through architectural state, delegated authority, and long-horizon interactions. Existing security taxonomies, however, primarily organize threats by attack type, such as prompt injection or jailbreaking, and therefore obscure where in the agentic stack a threat arises and over what timescale it manifests. We propose the Layered Attack Surface Model (\lasm), a structural taxonomy for agentic AI security. \lasm decomposes the agentic stack into seven layers -- Foundation, Cognitive, Memory, Tool Execution, Multi-Agent Coordination, Ecosystem, and Governance -- and augments them with a four-class temporality axis covering instantaneous, session-persistent, cross-session cumulative, and sub-session-stack threats. We use this 7

\times

4 framework to analyze 116 papers from 2021--2026. The resulting map reveals that the upper layers of the agentic stack remain sharply under-explored, especially for long-horizon and stack-propagating threats; multiple documented attack regions have no corresponding defenses; and current benchmarks provide no coverage for cross-session or sub-session-stack failure modes. We further derive a cross-layer defense taxonomy, defense recipes for canonical attack classes, and a dependency DAG that separates near-term engineering gaps from fundamental research challenges. We release the per-paper coding, robustness scripts, and a reference Agent Bill of Materials schema to support reproducible analysis.

Kexin Chu

School of Computing, University of Connecticut

Threat Agentic Taxonomy Survey

Ask PDF

Jun 9, 2026cs.CR

Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation

Large language model (LLM) agents are rapidly moving from conversational interfaces to software components that plan, invoke tools, maintain memory, and act on external environments. This transition changes the nature of security risk. In agentic settings, failures are no longer limited to unsafe text generation. Untrusted content may redirect control flow, misuse tool privileges, corrupt persistent state, leak sensitive information, or trigger harmful external actions. At the same time, research on LLM agent security is expanding quickly but remains fragmented across attack families, defense layers, application domains, and evaluation settings. This paper synthesizes 247 papers through a lifecycle-based, systems-oriented framework that models agent security around the interaction of information flow, delegated authority, and persistent state. We organize the literature around four questions: how LLM agent security should be modeled, which threat surfaces and attack families dominate, what defenses have been proposed and with what tradeoffs, and how security claims are evaluated. We find that prompt injection and tool-mediated control-flow hijacking still dominate the field, while persistent state corruption and multi-agent propagation are becoming central emerging concerns. We further find that current defenses provide useful building blocks but remain weakly compositional, and that existing benchmarks still underrepresent long-horizon, stateful, and deployment-sensitive risks. We argue that secure LLM agents require explicit trust boundaries, principled privilege control, provenance-aware state management, and evaluation practices aligned with realistic operational settings.

Yuchen Ling, Shengcheng Yu, Zhenyu Chen +1

State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Technical University of Munich, Heilbronn, Germany

Large Language Model Safety Large Language Model Agents

Ask PDF

Jun 26, 2026cs.CR

LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity

Large language model (LLM) agents are rapidly being integrated into real-world systems. Their autonomy and tool-use capabilities generate substantial value while simultaneously expanding the security attack surface. This survey provides a comprehensive overview of the opportunities and challenges of LLM agents in security, focusing on two core areas: (1) threats to LLM agents themselves and corresponding mitigation strategies (LLM agents self-security), and (2) the role of LLM agents in empowering the cybersecurity lifecycle across offense and defense (LLM agents empowered cybersecurity). We first examine the internal and external attack surfaces of agents, propose a taxonomy organized by threat sources, and analyze associated mitigations and evaluation frameworks. We then investigate how agent capabilities are applied in cybersecurity practice and present, to our knowledge, the first agent-empowerment framework aligned with the full cyber offense-defense lifecycle. By systematically surveying these two areas, we are the first to highlight a positive feedback synergy between LLM agents self-security and empowered cybersecurity, offering new insights for the advancement of both. We further identify current limitations and outline promising directions for future research. The insights provided aim to catalyze the coordinated development of LLM agents self-security and agent empowered cybersecurity, paving the way for more capable and robust agent applications.

Yiwei Xu, Yong Zhuang, Xuanming Liu +6

School of Cyber Science and Engineering, Wuhan University, Wuhan, China. · Department of Computer Science and Engineering, University at Buffalo, Buffalo, USA.

Cybersecurity Large Language Model Agents Systematic Study

Ask PDF