From Requirements to Attack Trees: Grounded LLM Agents for Design-Time Security Review

Akash Iyer; Taha Demirkan; Keerthi Koneru; Aaryan Siddharthan; Sheethal Kumar; Ramesh Radhakrishnan

cs.CROct 2, 2026

From Requirements to Attack Trees: Grounded LLM Agents for Design-Time Security Review

Authors: Akash Iyer, Taha Demirkan, Keerthi Koneru, Aaryan Siddharthan, Sheethal Kumar, Ramesh Radhakrishnan

Organizations: Center for Advanced AI, Accenture

Abstract

Design-level security weaknesses can arise from requirements, trust assumptions, missing controls, and data flows before implementation begins. Existing security practices often identify these issues after code is written. We present a multi-agent LLM framework for design-time security analysis from product requirement documents and architecture diagrams. The proposed framework parses architecture diagrams into graph representations, generates misuse and failure cases, constructs attack trees, checks governance and compliance gaps, recommends mitigations, assigns enterprise security-domain tags, and produces a candidate revised architecture recommendation for expert review. The framework does not retrieve from Common Weakness Enumeration (CWE) databases at inference time. Instead, it analyzes system behavior, trust boundaries, component interactions, and data-flow assumptions. Misuse cases act as intermediate representations that link findings to system components and attack paths, while a validation and refinement loop filters unsupported findings and improves grounding, traceability, and actionability. We evaluate the framework on a Microsoft reference-labeled threat-modeling example, labeled synthetic PRD--architecture pairs, and two open-ended systems: Berty and Gas Town. The reference-labeled case supports threat-recovery and actionability analysis, while the open-ended cases evaluate validity, noise, traceability, actionability, redundancy, and attack-tree quality. Results show that architecture-informed, misuse-driven reasoning improves review quality compared with single-shot and ablation baselines. Keywords: LLM Multi-Agent Systems, Design-Time Security, Threat Modeling, Vulnerability Discovery, Architecture Diagrams, Security Analysis, Misuse Case Derivation, Attack Trees, Iterative Reasoning, Security Governance.

Figures & tables

Approach	Multi-Artifact Input	Misuse Modeling	Attack Trees	Iterative Reasoning	No CVE Retrieval	Domain Routing	Redesign Recommendation
Manual Threat Modeling	$\triangle$	$\triangle$	✓	✗	✓	$\triangle$	✗
Tool-Based Threat Modeling	$\triangle$	✗	$\triangle$	✗	$\triangle$	✗	✗
Security Design Flaw Detection	$\triangle$	✗	✗	✗	✓	✗	✗
Architecture / DFD Extraction	$\triangle$	✗	✗	✗	✓	✗	✗
LLM Single-Prompt	$\triangle$	✗	✗	✗	$\triangle$	✗	✗
LLM-Assisted Threat Modeling	$\triangle$	$\triangle$	$\triangle$	✗	$\triangle$	✗	✗

Table 1: Comparison of Prior Approaches and Proposed Framework. ✓: Full support.

\triangle

: Partial support — capability exists but requires structured inputs, expert facilitation, or manual effort not automated by the approach (e.g., Manual Threat Modeling supports Multi-Artifact Input partially because it can consume heterogeneous artifacts but requires human interpretation rather than automated parsing). ✗: No support.

Figure 1: Overview of the proposed multi-agent pipeline for iterative SDLC security analysis from PRDs and architecture artifacts.

Configuration	Description
Single-shot LLM	One prompt generates misuse cases, attack trees, and mitigations from PRD and architecture.
Single-shot STRIDE	One prompt generates a STRIDE-structured threat table from PRD and architecture.
No-MC	Removes the misuse-case agent and generates attack trees directly from architecture context.
PRD-only	Uses only the PRD, evaluating early-stage security analysis without architecture grounding.
Arch-Informed	Full pipeline using PRD, architecture diagram, misuse cases, attack trees, and iterative refinement.

Table 2: Evaluation configurations.

Metric	Definition
Coverage (Raw)	Fraction of ground-truth threats identified by the system: $\frac{\|T_{gen}\cap T_{gt}\|}{\|T_{gt}\|}$ .
Grounded Coverage	Fraction of ground-truth threats correctly identified and grounded in the architecture: $\frac{\|T_{mapped}\cap T_{gt}\|}{\|T_{gt}\|}$ .
Grounded Precision	Fraction of grounded generated threats that are correct: $\frac{\|T_{mapped}\cap T_{gt}\|}{\|T_{mapped}\|}$ .
Novelty	Fraction of validated threats not present in the ground truth but judged plausible: $\frac{\|T_{valid}\setminus T_{gt}\|}{\|T_{valid}\|}$ .
Validity	Fraction of generated threats judged valid after evaluation: $\frac{\|T_{valid}\|}{\|T_{gen}\|}$ .
Noise	Fraction of generated threats judged invalid or unsupported: $1-\text{Validity}$ .

Table 3: Evaluation metrics.

Domain	Config.	#MC	#AT	Raw Cov.	G. Cov.	G. Prec.	Valid.	Noise	Trace.	Action.	Novel.	Redun.	ATQ
EHS	Arch-Informed	29.7	9.0	0.97	0.97	0.84	0.97	0.03	1.00	0.91	0.63	0.13	4.40
EHS	No-MC	0.0	15.3	0.67	0.67	0.26	0.99	0.01	N/A	0.20	0.73	0.13	4.30
EHS	PRD-only	30.3	11.7	0.88	0.85	0.72	0.95	0.05	0.72	0.88	0.72	0.06	4.13
EHS	Single-shot LLM	20.7	4.3	0.44	0.40	0.24	0.88	0.12	0.00	0.06	0.66	0.23	3.45
EHS	Single-shot STRIDE	23.3	4.0	0.58	0.42	0.36	0.87	0.13	0.00	0.31	0.56	0.17	3.30
OLB	Arch-Informed	11.3	13.0	0.85	0.85	0.81	0.99	0.01	0.80	0.90	0.25	0.10	4.30

Table 4: Controlled synthetic results averaged over three runs. Synthetic designs contain intentionally injected weaknesses and manually reviewed ground truth. For synthetic No-MC, coverage is computed over threats extracted from generated attack-tree leaves; traceability is N/A because no misuse-case artifacts are generated. #MC: Misuse Cases Generated; #AT: Attack Trees Generated.

Figure 2: Core metrics on the Microsoft reference-labeled threat-modeling example. No-MC is excluded because threat-scoped metrics are not applicable when the misuse-case agent is removed.

Figure 3: Attack-tree quality on the Microsoft reference-labeled threat-modeling example. No-MC is included because it still produces downstream attack-tree artifacts.

Metric	Arch-Informed	No-MC	PRD-only	Single-shot LLM	Single-shot STRIDE
# Misuse cases generated	13.3	0.0	4.7	5.7	17.3
# Attack trees generated	9.0	6.7	2.3	3.0	6.0
Coverage (Raw)	0.83	N/A	0.17	0.17	0.33
Grounded Coverage	0.83	N/A	0.08	0.17	0.11
Grounded Precision	0.63	N/A	0.10	0.30	0.23
Validity	1.00	N/A	0.55	0.50	0.67

Table 5: Results on the Microsoft reference-labeled threat-modeling example. Values are averaged over three trials per configuration. Coverage, grounded coverage, grounded precision, and novelty are computed against the 12-threat reference set. No-MC produces only downstream attack-tree artifacts, so threat-scoped metrics are reported as N/A.

Domain	Misuse/Failure	Attack Trees	Total
IAM	2	3	5
Network Security	4	4	8
Application Security	4	4	8
Data Security	5	6	11
Cloud Security	4	4	8
DevSecOps	1	2	3

Table 6: Domain routing for the Microsoft Arch-Informed run. Counts indicate artifacts tagged for each enterprise security domain.

Design change	Diagram evidence	Linked finding or weakness	Security effect
Service-to-service authentication	Mutual TLS is annotated on WebSocket flows among edge modules M1, M2, and M3.	Missing independent authentication; unauthorized module-to-module access.	Strengthens identity and trust between internal edge modules.
Storage upload access control	RBAC is annotated on the feedback-frame upload path from M2 to Azure Storage.	Unauthorized access to feedback uploads; weak access policy for cloud storage.	Restricts storage writes to authorized modules and identities.
Input and request validation	Data validation is annotated before the OCR/text-to-speech request to Azure Cognitive Services.	Input manipulation; missing validation before external service processing.	Reduces malformed or adversarial data sent to downstream services.
Encrypted data transmission	Encrypted response and telemetry paths are annotated in the revised design.	Telemetry data loss; MITM or disclosure risk on service communication paths.	Improves confidentiality and integrity of data in transit.
Browser-side content security	A Content Security Policy note is added for the browser-facing component.	Frontend data-handling risk; inadequate output filtering or browser-side injection exposure.	Reduces client-side injection and unsafe content-handling risk.

Table 7: Candidate redesign changes for the Microsoft reference-labeled threat-modeling example: Arch-Informed run.

System	Config.	#MC	#AT	Valid.	Noise	Trace.	Action.	Redun.	ATQ
Berty	Arch-Informed	11.70	6.70	0.92	0.08	0.90	0.84	0.14	4.00
Berty	No-MC	0.00	4.70	N/A	N/A	N/A	N/A	N/A	1.30
Berty	PRD-only	6.30	5.00	0.49	0.51	0.40	0.85	0.32	2.70
Berty	Single-shot LLM	5.70	3.00	0.54	0.46	0.59	0.85	0.00	1.70
Berty	Single-shot STRIDE	19.30	4.70	0.99	0.01	0.50	0.98	0.42	3.30
Gas Town	Arch-Informed	6.30	4.70	0.97	0.03	1.00	0.96	0.04	4.30

Table 8: Open-ended real-world results averaged over three runs. Coverage and precision are omitted because no complete ground-truth threat set exists. N/A indicates that threat-scoped metrics are not applicable for No-MC. #MC: Misuse Cases Generated; #AT: Attack Trees Generated.

Figure 4: Core review-quality metrics on Berty. No-MC is excluded because threat-scoped metrics are not applicable when the misuse-case agent is removed.

Figure 5: Attack-tree quality on Berty. No-MC is included because it still produces downstream attack-tree artifacts.

Figure 6: Core review-quality metrics on Gas Town. No-MC is excluded because threat-scoped metrics are not applicable when the misuse-case agent is removed.

Figure 7: Attack-tree quality on Gas Town. The Arch-Informed pipeline achieves the highest score, while No-MC remains lower despite producing attack trees.

Figure 8: Synthetic Electronic Health System (EHS) architecture.

Figure 9: Candidate revised Electronic Health System (EHS) architecture.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Synthetic Online Banking System (OLB) architecture.

Panel of Figure 11 — Figure 11: Original and candidate revised architecture for the Microsoft reference-labeled threat-modeling case. The revised architecture adds design-time controls derived from Arch-Informed findings and mitigation recommendations, including mutual TLS, RBAC-protected storage upload, data validation, encrypted communication paths, telemetry protection, and browser-side content-security controls.

Type	Artifact	Domain tags
MC	No Rate Limiting on IoTEdge Modules	NET, GRC
MC	Missing Independent Authentication	IAM, NET, APP, GRC
MC	Lack of Fine-Grained Authorization	IAM, DATA, GRC
MC	Inadequate Output Filtering $(M3\rightarrow\mathrm{Cognitive})$	NET, DATA
MC	Missing Input Validation in Modules	APP, GRC
MC	Unauthorized Access to Cognitive Transfers	NET, DATA, CLOUD, GRC

Appendix

Table 9: Detailed domain routing for the Microsoft Arch-Informed run.

Stage	Calls	Input Tok.	Output Tok.	Total Tok.	Share
Misuse / failure-case generation	8	26,833	5,655	32,488	25.8%
Attack-tree planner	3	5,392	515	5,907	4.7%
Attack-tree workers	17	27,491	6,914	34,405	27.4%
Confidence scoring / reasoning	8	23,317	6,907	30,224	24.0%
Attack-tree validation	3	13,772	1,242	15,014	11.9%
Governance review	1	2,625	901	3,526	2.8%

Appendix

Table 10: Representative token usage for the Microsoft Arch-Informed run.

Cybersecurity

Cybersecurity develops methods to protect digital and cyber-physical systems from attacks, vulnerabilities, data loss, and misuse.

See all 355 papers

Mar 8, 2026

Where Do LLM-based Systems Break? A System-Level Security Framework for Risk Assessment and Treatment

Large Language Models (LLMs) are increasingly integrated into safety-critical workflows, yet existing security analyses remain fragmented and often isolate model behavior from the broader system context. This work introduces a goal-driven risk assessment framework for LLM-powered systems that combines system modeling with Attack-Defense Trees (ADTrees) and Common Vulnerability Scoring System (CVSS)-based exploitability scoring to support structured, comparable analysis. We demonstrate the framework through a healthcare case study, modeling multi-step attack paths targeting intervention in medical procedures, leakage of electronic health record (EHR) data, and disruption of service availability. Our analysis indicates that threats spanning (i) conventional cyber, (ii) adversarial ML, and (iii) conversational attacks that manipulate prompts or context often consolidate into a small number of dominant paths and shared system choke points, enabling targeted defenses to yield meaningful reductions in path exploitability. By systematically comparing defense portfolios, we align these risks with established vulnerability management practices and provide a domain-agnostic workflow applicable to other LLM-enabled critical systems.

Neha Nagaraja, Hayretdin Bahsi

School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, USA · Department of Software Science, Tallinn University of Technology, Tallinn, Estonia

LLM Security Cybersecurity

Ask PDF

Aug 28, 2026

LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment

Software and systems security workflows are typically procedural: analysts inspect heterogeneous artifacts, form hypotheses, invoke tools, interpret outputs, and revise plans. Large language model (LLM)-based agents, which can plan, use tools, retain state, and revise actions across multi-step workflows, are being rapidly adopted to automate this work. Given the consequences of delegating security decisions to autonomous systems, understanding how such agents are built, used, and assessed is crucial. Yet to this date, there remains a lack of systematic understanding of what has been done and how far we are in this field: the term "agent" is applied inconsistently, applications differ sharply in risk, and assessment protocols are often incomparable. To gain a comprehensive and coherent view of this area hence inform relevant future research, this paper provides a systematic literature review of the (1) technical approaches, including agent architecture, perception, memory, reasoning and planning, action space, orchestration, and self-improvement, (2) applications, with respect to the security tasks served, and (3) assessment, including the datasets, outcome and trajectory metrics, safety measures, and baselines considered, over the peer-reviewed literature spanning the emergence of this area (2023--2026). Our synthesis reveals a field that has built agents able to act but not yet agents whose authority is bounded or whose behavior is auditable. In addition to knowledge systematization, we also extend our insights into the limitations of and challenges faced by current approach, application, and assessment designs, which shed light on potentially promising future research directions.

Jingjing Nie, Jiawei Guo, Krishna Meda +1

University at Buffalo, SUNY, Buffalo, New York, USA

LLM Agent Security LLMs for Cybersecurity LLM Agent Evaluation

Ask PDF

Oct 2, 2026

Beyond Predefined Sinks: Security-Aware Dependency Analysis for LLM Agents

Large language model (LLM)-based agents increasingly connect model-generated decisions to security-sensitive software capabilities such as command execution, filesystem access, network communication, browser control, and external tools. Existing analyses often use predefined sensitive operations as anchors, but operation identity alone is insufficient to determine security implications. We present AgentSecGraph, a security-aware static analysis framework that constructs a candidate-centered Security-Aware Agent Dependency Graph (Security-ADG) for each security-sensitive operation. It augments operation identity with agent relevance, source and dependency evidence, trust-boundary context, guard evidence, and external-effect semantics. We further introduce AgentSecBench, a corpus of 67 real-world LLM-agent repositories spanning 11 ecosystems and 37,542 source files. The current analyzer identifies 23,866 static security-sensitive operation candidates across 65 repositories and emits one Security-ADG artifact per candidate. Corpus-wide analysis recovers source-to-operation dependency evidence for 9,821 candidates (41.15%) and potential guard evidence for 3,075 (12.88%), completing in 50.8 minutes. Using a separate reproduction-backed evaluation layer, we establish 22 security-sensitive behaviors across 13 repositories: one confirmed vulnerability, one pending disclosure candidate, and 20 guarded behaviors. In nine held-out cases, Security-ADG preserves 91.1% of the reference context and all five observed guards, compared with 20.0% for a sink-only view and 40.0% for a simplified ADG. These results show that security-aware dependency and contextual evidence enable distinctions that cannot be recovered from sensitive-operation identity alone.

Hang Cui

LLM Agent Security AI Agent Auditing AI Agent Security Benchmarks

Ask PDF

Abstract

Cybersecurity

Explore similar work

Where Do LLM-based Systems Break? A System-Level Security Framework for Risk Assessment and Treatment

LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment

Beyond Predefined Sinks: Security-Aware Dependency Analysis for LLM Agents