Hierarchical Agentic Incident Response with Digital-Twin-Validated Attack Inference
Authors: Yiran Gao, Juntao Chen, Tao Li
Organizations: Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, 99977 · Department of Computer and Information Sciences, Fordham University, New York, NY, 10023, USA
Network incident response remains slow and labor-intensive as the defender must infer multi-stage attacks from partial observations and translate recovery decisions into reliable system commands. Decision-theoretic planners provide principled optimization but typically rely on abstract states and predefined actions, while large language model (LLM) agents can reason over operational context but may hallucinate attacks and responses. Toward automating response planning, we present a hierarchical agentic response framework that integrates LLM-based attack inference, rollout planning, and digital-twin validation. A fine-tuned LLM infers the attack progression and affected hosts from security alerts and system measurements. An emulated network digital twin replays the inferred attack and returns discrepancies between predicted and observed effects to calibrate the inference. A separately fine-tuned planning agent uses the rollout planning method to prioritize affected components at the tactical layer. At the operational layer, the planning agent proposes high-level recovery actions, and an execution agent translates selected actions into recovery and verification commands that are validated in the digital twin. We evaluate the framework on a 33-component enterprise-network testbed under three multi-stage attack scenarios. The results show that our framework outperforms frontier-LLM baselines in recovery success rate by 18--31%.
Figures & tables
Fig. 1: Overview of the hierarchical agentic incident-response framework.
Incident response is currently managed by security operators using predefined playbooks, resulting in slow, labor-intensive security decision-making processes. Consequently, there is a growing need for automated incident response planning. Decision-theoretic approaches based on control, optimization, and reinforcement learning have been proposed to automate such planning tasks with well-grounded approaches, yet most of which, while guaranteeing strong performance, are limited to abstract models and cannot be directly applied to operational systems. A promising approach to mitigate this limitation is to use the security knowledge embedded in large language models (LLMs) to develop agentic response systems. However, current agentic approaches rely on repeated invocations of the LLM to generate a response plan, which is unreliable and limits the planning horizon due to hallucination. In this paper, we develop a principled LLM-based planning method by combining decision-theoretic planning with LLM-generated response commands. The proposed agentic incident response approach uses a rollout planner to compute a high-level response strategy that allocates security resources (the tactical scale), which is then translated into executable commands by a lightweight LLM agent (the operational scale). Within this architecture, we use a digital twin that supports tactical planning through simulation and operational execution through emulation. Across three attack scenarios, our agentic approach reduces recovery execution time by 15.1% on average and increases the recovery rate by 33.6% over frontier LLM baselines.
Yiran Gao, Tao Li, Kim Hammar
City University of Hong Kong, Hong Kong SAR · Imperial College London, United Kingdom
Incident response planning is critical for restoring compromised software systems. Existing AIOps methods automate fault diagnosis and remediation and adapt response using feedback, but lack a unified decision basis across planning and execution. This limits their ability to maintain incident state and align actions with evolving recovery objectives during long-horizon, multi-stage security incident response. We present STAIR, an end-to-end agentic response framework that organizes planning and execution around a shared incident state. STAIR maintains incident evidence, response actions, and execution feedback in Graph-as-State (GAS), using the latest state to determine the response stage and recovery objectives for stage-specialized planning. The framework retrieves effect-validated historical experience based on the current state and stage to guide action selection. An Execution Harness executes actions, returns feedback to update GAS, and validates action effects for future experience reuse. We construct STAIR-Bench, an executable incident response benchmark comprising 44 Docker-based cyber-range cases. On its held-out test cases, STAIR achieves an overall defense score of 0.95, improving over the strongest baseline by approximately 10.47%. In the independent CAGE4 environment with ongoing attacks and benign activity, STAIR reduces cumulative penalties by approximately 28.8% relative to the strongest baseline.
Hanlin Jiang, Jionghao Huang, Shaofei Li +6
Key Laboratory of High Confidence Software Technologies, Peking University, Ministry of Education · Southeast University · Huawei Technologies Co., Ltd.
Large language models (LLMs) promise to accelerate incident response in production systems, yet single-agent approaches generate vague, unusable recommendations. We present MyAntFarm.ai, a reproducible containerized framework demonstrating that multi-agent orchestration fundamentally transforms LLM-based incident response quality. Through 348 controlled trials comparing single-agent copilot versus multi-agent systems on identical incident scenarios, we find that multi-agent orchestration achieves 100% actionable recommendation rate versus 1.7% for single-agent approaches, an 80 times improvement in action specificity and 140 times improvement in solution correctness. Critically, multi-agent systems exhibit zero quality variance across all trials, enabling production SLA commitments impossible with inconsistent single-agent outputs. Both architectures achieve similar comprehension latency (approx.40s), establishing that the architectural value lies in deterministic quality, not speed. We introduce Decision Quality (DQ), a novel metric capturing validity, specificity, and correctness properties essential for operational deployment that existing LLM metrics do not address. These findings reframe multi-agent orchestration from a performance optimization to a production-readiness requirement for LLM-based incident response. All code, Docker configurations, and trial data are publicly available for reproduction.