From Threat Intelligence to Firewall Rules: Semantic Relations in Hybrid AI Agent and Expert System Architectures

Chiara Bonfanti; Davide Colaiacomo; Luca Cagliero; Cataldo Basile

cs.AIMar 4, 2026

From Threat Intelligence to Firewall Rules: Semantic Relations in Hybrid AI Agent and Expert System Architectures

Authors: Chiara Bonfanti, Davide Colaiacomo, Luca Cagliero, Cataldo Basile

Organizations: Department of Control and Computer Engineering, Politecnico di Torino, Italy

Abstract

Web security demands rapid response capabilities to evolving cyber threats. Agentic Artificial Intelligence (AI) promises automation, but the need for trustworthy security responses is of the utmost importance. This work investigates the role of semantic relations in extracting information for sensitive operational tasks, such as configuring security controls for mitigating threats. To this end, it proposes to leverage hypernym-hyponym textual relations to extract relevant information from Cyber Threat Intelligence (CTI) reports. By leveraging a neuro-symbolic approach, the multi-agent system automatically generates CLIPS code for an expert system creating firewall rules to block malicious network traffic. Experimental results show the superior performance of the hypernym-hyponym retrieval strategy compared to various baselines and the higher effectiveness of the agentic approach in mitigating threats.

Figures & tables

Figure 1: The architecture of our agentic solution.

Task	Method	Task-based Eval.			Semantic Metrics
Task	Method	F1 w	Acc. w	Top-k Acc.	BERT Score F1	ROUGE L
A-1	Word2Vec	0.070	0.990	0.250	–	–
A-1	GloVe	0.118	0.991	0.382	–	–
A-1	SecureBERT	0.043	0.990	0.210	0.831	–
A-2	CoT	0.308	0.932	0.935	0.862	0.402
A-2	Ours	0.329	0.934	0.968	0.858	0.444

Table 1: Task A results comparing our hypernym-based prompting approach. Best-performer per metrics is written in boldface.

Dimension	Krippendorff’s $\alpha$	Cohen’s $\kappa$			Spearman’s $\rho$
Dimension	Krippendorff’s $\alpha$	Unweighted	Linear	Quadratic	Spearman’s $\rho$
Technical Correctness	$+0.5768$	$+0.5371$	$+0.5625$	$+0.5817$	$+0.6942$
Fidelity to CTI	$+0.5215$	$+0.3445$	$+0.4595$	$+0.5599$	$+0.5515$
Scope Calibration	$+0.5030$	$+0.5827$	$+0.6540$	$+0.7251$	$+0.7143$

Table 2: Task B Inter-rater reliability metrics by dimension.

Method	F1	ACC	TOP-10	BERTScore	ROUGE-L F1
Our Method (3-Stage 50%)	0.329	0.934	0.968	0.858	0.444
Few-Shot + Hyponyms	0.311	0.935	0.952	0.860	0.411
CoT + Hypernyms	0.308	0.933	0.984	0.865	0.407
CoT Baseline	0.308	0.932	0.935	0.862	0.402
Few-Shot + Hypernyms	0.287	0.929	0.984	0.856	0.400
CoT + Hyponyms	0.294	0.931	0.984	0.871	0.391

Table 3: Task A-2 prompting focused analysis.

Autonomous Cyber Defense

Autonomous cyber defense uses AI agents to detect, investigate, and contain attacks while balancing operational safety, latency, and human oversight.

See all 72 papers

CTIFoundry: An Agent-Native Corpus Scaffold for Cyber Threat Intelligence
Aug 19, 2026Yutong Cheng, Changze Li, Qian Cui +4Cyber Threat Intelligence Agentic Retrieval
Virginia Tech · Amazon · Northwestern University
FALCON: Transforming Cyber Threat Intelligence into Deployable IDS Rules with Self-Reflection
Aug 26, 2025Shaswata Mitra, Subash Neupane, Martin Duclos +5Cyber Threat Intelligence Network Intrusion Detection
The University of Alabama · Meharry Medical College · The University of Texas at El Paso
AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions
Jun 7, 2026Chenglin YangAI Agent Reliability LLM-as-a-Judge
Independent Researcher

Aug 19, 2026J/K move · Enter open · S save

CTIFoundry: An Agent-Native Corpus Scaffold for Cyber Threat Intelligence

Yutong Cheng, Changze Li, Qian Cui, Wei Ding+3

Virginia Tech · Amazon · Northwestern University

Cyber Threat Intelligence Agentic Retrieval LLMs for Cybersecurity

Open paper AskPDF arXiv

Cyber threat intelligence (CTI) is increasingly consumed not by human analysts but by LLM agents that compose multi-step investigations at query time. The harness side of this shift has matured rapidly, but the corpus side has not: threat reports and vulnerability databases are still packaged for retrieval-augmented generation, as opaque chunks behind an embedding index. We argue that this substrate, not model capability, is the bottleneck on agentic CTI investigation, and present CTIFoundry, an agent-native corpus scaffold. At build time, CTIFoundry materializes the latent structure of a CTI corpus: a deterministic ontology graph over four authoritative knowledge bases (CVE, CWE, CAPEC, ATT&CK) whose official cross-references become typed, traversable edges; a span-grounded report layer whose canonical, alias-resolved cross-vendor entities index provenance-carrying chunks; and hybrid dense+lexical retrieval surfaces. At query time this structure is exposed through seven typed tools and three procedural skills mounted on a stock, widely-used open-source agent harness. On the public CTIConnect benchmark, swapping only the action surface lifts the identically-harnessed agent from 0.610 to 0.829 overall F1 with gpt-5.4 and from 0.470 to 0.745 with claude-haiku-4-5: a small model on CTIFoundry surpasses a flagship model on the flat substrate. The scaffolded agent is simultaneously more accurate and more efficient: on both Claude models it answers with roughly half the tool calls per question. The ablation distills design principles for matching corpus scaffolding to data modality, in CTI and beyond. Build-time validation guarantees zero fabricated identifiers by construction, and the scaffold sustains 1,168 investigations end-to-end at about 2.6 cents each.

Abstract

Autonomous Cyber Defense

Explore similar work

CTIFoundry: An Agent-Native Corpus Scaffold for Cyber Threat Intelligence

FALCON: Transforming Cyber Threat Intelligence into Deployable IDS Rules with Self-Reflection

AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions