CSTS: A Canonical Security Telemetry Substrate for AI-Native Cyber Detection

Abdul Rahman

cs.CRMar 24, 2026

CSTS: A Canonical Security Telemetry Substrate for AI-Native Cyber Detection

Authors: Abdul Rahman

Abstract

Cybersecurity data remains fragmented across vendors, formats, schemas, and deployment environments, forcing AI and analytics programs to spend disproportionate effort on ingestion, normalization, and brittle source-specific engineering. This paper introduces the Canonical Security Telemetry Substrate (CSTS), a canonical, AI-ready telemetry foundation designed to harmonize heterogeneous cyber data into a common representation over persistent entities, typed relations, events, temporal state, and provenance. CSTS is intended to move cybersecurity analytics beyond ad hoc record normalization toward a reusable substrate that supports anomaly detection, graph learning, forecasting, behavior-based modeling, and agentic cyber AI. We formalize the core design principles of CSTS, define its representational components, and explain how it preserves source-specific nuance through explicit mappings and extensible metadata while still enabling portable downstream inference. We further position CSTS as a cloud-agnostic and deployment-agnostic substrate suitable for on-prem, hybrid, and multi-cloud environments. The result is a unifying telemetry model that reduces the blue-collar burden of cyber data engineering and creates a clearer path to scalable, interoperable, and model-agnostic cyber AI.

Figures & tables

Task	Setting	Method	[email protected]	Prec.	Rec.	AUROC	Best F1
LM	EnvA $\rightarrow$ EnvA	Baseline	0.894	0.840	0.955	0.955	0.894
LM	EnvA $\rightarrow$ EnvB	Baseline	0.135	0.079	0.457	0.660	0.182
LM	EnvA $\rightarrow$ EnvA	CSTS	0.378	0.236	0.955	0.772	0.382
LM	EnvA $\rightarrow$ EnvB	CSTS	0.508	0.941	0.348	0.991	0.821

TABLE I : Lateral movement transfer performance comparing event-centric and CSTS-based representations.

Task	Level	Method	[email protected]	AUROC	Best F1
LM	P0	Baseline	0.135	0.660	0.182
LM	P1	Baseline	0.000	0.000	0.000
LM	P2	Baseline	0.000	0.000	0.000
LM	P3	Baseline	0.000	0.000	0.000
LM	P0	CSTS	0.316	0.690	0.356
LM	P1	CSTS	0.233	0.664	0.337

TABLE II : Robustness under targeted schema perturbation (LM task, EnvA

\rightarrow

EnvB).

Task	Setting	Method	[email protected]	Prec.	Rec.	AUROC	Best F1	Best Thr.	Train+	Train-	Test+	Test-	AUROC CI Low	AUROC CI High	F1 CI Low	F1 CI High
LM	EnvA $\rightarrow$ EnvA	Baseline	0.000	0.000	0.000	0.316	0.000	0.05	4	87	1	38	0.189	0.500	0.000	0.000
LM	EnvA $\rightarrow$ EnvB	Baseline	0.000	0.000	0.000	0.640	0.041	0.05	5	125	1	125	0.500	0.726	0.000	0.000
LM	EnvA $\rightarrow$ EnvA	CSTS	0.000	0.000	0.000	0.203	0.000	0.05	7	53	3	23	0.000	0.500	0.000	0.000
LM	EnvA $\rightarrow$ EnvB	CSTS	0.000	0.000	0.000	0.218	0.063	0.10	10	76	3	58	0.000	0.600	0.000	0.000

TABLE III : External Sysmon LM transfer results on splunk/attack_data (real-log smoke test). Results use deterministic motif-based proxy labels and include split class counts and bootstrap confidence intervals; they are reported as robustness/smoke-test evidence rather than definitive external generalization evidence. Because proxy-positive windows are highly imbalanced, fixed-threshold metrics ([email protected]) are unstable; we therefore emphasize AUROC, best-F1 under a fixed threshold sweep, and bootstrap intervals for uncertainty.

Task	Level	Method	[email protected]	AUROC	Best F1	Best Thr.	Train+	Train-	Test+	Test-
LM	P0	Baseline	0.000	0.640	0.041	0.05	5	125	1	125
LM	P1	Baseline	0.000	0.000	0.000	0.50	0	0	0	0
LM	P2	Baseline	0.000	0.000	0.000	0.50	0	0	0	0
LM	P3	Baseline	0.000	0.000	0.000	0.50	0	0	0	0
LM	P0	CSTS	0.000	0.218	0.063	0.10	10	76	3	58
LM	P1	CSTS	0.000	0.218	0.063	0.10	10	76	3	58

TABLE IV : Schema-perturbation robustness on splunk/attack_data external Sysmon logs (EnvB perturbations P0–P3). The event-centric baseline relies on raw column names and fails under targeted schema changes, while the CSTS pipeline retains signal via alias recovery and canonicalization.

Task	Setting	Method	[email protected]	Prec.	Rec.	AUROC	Best F1	Best Thr.	Train+	Train-	Test+	Test-	AUROC CI Low	AUROC CI High	F1 CI Low	F1 CI High
LM	EnvA $\rightarrow$ EnvA	Baseline	0.000	0.000	0.000	0.000	0.000	0.50	0	0	0	0	0.000	0.000	0.000	0.000
LM	EnvA $\rightarrow$ EnvB	Baseline	0.057	0.029	1.000	0.864	0.093	0.85	1	14	2	143	0.500	0.993	0.000	0.136
LM	EnvA $\rightarrow$ EnvA	CSTS	0.000	0.000	0.000	0.500	0.000	0.05	1	3	0	3	0.500	0.500	0.000	0.000
LM	EnvA $\rightarrow$ EnvB	CSTS	0.154	0.085	0.800	0.738	0.178	0.95	1	6	5	103	0.362	0.926	0.038	0.296

TABLE V : External Sysmon LM transfer results on OTRF Security-Datasets (real-log smoke test). Deterministic ordered motif-based proxy labels are sparse in the selected corpora/splits; we therefore report bootstrap confidence intervals and split class counts and do not treat these results as definitive external generalization evidence.

Task	Level	Method	[email protected]	AUROC	Best F1	Best Thr.	Train+	Train-	Test+	Test-
LM	P0	Baseline	0.057	0.864	0.093	0.85	1	14	2	143
LM	P1	Baseline	0.000	0.000	0.000	0.50	0	0	0	0
LM	P2	Baseline	0.000	0.000	0.000	0.50	0	0	0	0
LM	P3	Baseline	0.000	0.000	0.000	0.50	0	0	0	0
LM	P0	CSTS	0.154	0.738	0.178	0.95	1	6	5	103
LM	P1	CSTS	0.154	0.738	0.178	0.95	1	6	5	103

TABLE VI : Schema-perturbation robustness on OTRF external Sysmon logs (EnvB perturbations P0–P3). Targeted raw-schema perturbations break baseline parser assumptions, while CSTS remains operable through alias-aware parsing and canonical graph construction.

Quantity	CADETS (train)	TRACE (test)
Window minutes ( $\Delta$ )	30
Quantile ( $q$ )	0.40
Number of windows ( $n$ )	181	16
Windows above train threshold	74	0
Train-only threshold ( $\tau$ )	3.972
Score median (p50)	3.812	3.678

TABLE VII : DARPA TC E3 producer-divergence viability artifact (CADETS

\rightarrow

TRACE), computed under leakage-safe train-only thresholding. Although channel/token coverage is high in both producers, TRACE window scores fall entirely below the CADETS-derived threshold, yielding

0

test positives and motivating treatment as a portability boundary case study rather than a transfer benchmark on this minimal slice.

Task	Setting	Method	[email protected]	Prec.	Rec.	AUROC	Best F1
ZDT	EnvA $\rightarrow$ EnvA	Baseline	0.217	0.131	0.640	0.728	0.315
ZDT	EnvA $\rightarrow$ EnvB	Baseline	0.170	0.105	0.447	0.593	0.205
ZDT	EnvA $\rightarrow$ EnvA	CSTS	0.145	0.081	0.719	0.520	0.151
ZDT	EnvA $\rightarrow$ EnvB	CSTS	0.118	0.064	0.765	0.459	0.125

TABLE VIII : Zero-day threat detection (ZDT) transfer performance comparing event-centric and CSTS representations.

Cybersecurity

Cybersecurity develops methods to protect digital and cyber-physical systems from attacks, vulnerabilities, data loss, and misuse.

See all 354 papers

CTIFoundry: An Agent-Native Corpus Scaffold for Cyber Threat Intelligence
Aug 19, 2026Yutong Cheng, Changze Li, Qian Cui +4Cyber Threat Intelligence Agentic Retrieval
Virginia Tech · Amazon · Northwestern University
Cyber Threat Intelligence for Artificial Intelligence Systems
Mar 5, 2026Natalia Krawczyk, Mateusz Szczepkowski, Adrian Brodzik +1Cyber Threat Intelligence Cybersecurity
Orange Innovation Poland
CyTReX: Explainable AI-Based Cybersecurity Threat Reasoning Framework for DER Networks
Oct 3, 2026Damilola Popoola, Souradeep Bhattacharya, Manimaran GovindarasuCybersecurity Explainable Artificial Intelligence
Department of Electrical and Computer Engineering Iowa State University, Ames, IA 50011

Aug 19, 2026J/K move · Enter open · S save

CTIFoundry: An Agent-Native Corpus Scaffold for Cyber Threat Intelligence

Yutong Cheng, Changze Li, Qian Cui, Wei Ding+3

Virginia Tech · Amazon · Northwestern University

Cyber Threat Intelligence Agentic Retrieval LLMs for Cybersecurity

Open paper AskPDF arXiv

Cyber threat intelligence (CTI) is increasingly consumed not by human analysts but by LLM agents that compose multi-step investigations at query time. The harness side of this shift has matured rapidly, but the corpus side has not: threat reports and vulnerability databases are still packaged for retrieval-augmented generation, as opaque chunks behind an embedding index. We argue that this substrate, not model capability, is the bottleneck on agentic CTI investigation, and present CTIFoundry, an agent-native corpus scaffold. At build time, CTIFoundry materializes the latent structure of a CTI corpus: a deterministic ontology graph over four authoritative knowledge bases (CVE, CWE, CAPEC, ATT&CK) whose official cross-references become typed, traversable edges; a span-grounded report layer whose canonical, alias-resolved cross-vendor entities index provenance-carrying chunks; and hybrid dense+lexical retrieval surfaces. At query time this structure is exposed through seven typed tools and three procedural skills mounted on a stock, widely-used open-source agent harness. On the public CTIConnect benchmark, swapping only the action surface lifts the identically-harnessed agent from 0.610 to 0.829 overall F1 with gpt-5.4 and from 0.470 to 0.745 with claude-haiku-4-5: a small model on CTIFoundry surpasses a flagship model on the flat substrate. The scaffolded agent is simultaneously more accurate and more efficient: on both Claude models it answers with roughly half the tool calls per question. The ablation distills design principles for matching corpus scaffolding to data modality, in CTI and beyond. Build-time validation guarantees zero fabricated identifiers by construction, and the scaffold sustains 1,168 investigations end-to-end at about 2.6 cents each.

Abstract

Cybersecurity

Explore similar work

CTIFoundry: An Agent-Native Corpus Scaffold for Cyber Threat Intelligence

Cyber Threat Intelligence for Artificial Intelligence Systems

CyTReX: Explainable AI-Based Cybersecurity Threat Reasoning Framework for DER Networks