cs.CRAug 15, 2026
SaveSysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system
Organizations: School of Computer Science, Peking University · Southeast University · School of Computing, National University of Singapore
Abstract
The rapid advancement of large language models (LLMs) has created a growing asymmetry in cybersecurity, where attack accelerates toward autonomous execution while defense remains predominantly human-intensive. Despite substantial prior work across cyber ranges, AI-driven attack, and AI-driven defense, this asymmetry persists. We trace it to a deeper root cause, that evolution itself has stalled on both sides at three layers. To overcome this, we propose co-evolution as the integrating insight, where attack and defense AI agents autonomously and safely drive each other's evolution through adversarial confrontation. Based on this insight, we present \sysevolve, comprising three co-designed components, \sysfield, \sysspear, and \sysarmor. \sysfield constructs realistic multi-host ranges. \sysspear generates efficient, safe attack schemes. \sysarmor performs real-time, interpretable defense. Together they form a self-driven adversarial loop restoring evolution at all three layers. In evaluation, \sysfield achieves zero-loss collection at 2.1% overhead and orchestrates 257 CVEs into 1,148 ranges, \sysspear improves attack success by over 25% over baseline LLMs, and \sysarmor achieves 10--1000 greater precision than prior systems and detects real APT attacks in production at Huawei and Sangfor. Our evaluation also reveals three findings about LLM agent capabilities. First, multi-step composition and larger topologies expose agent capability gaps hidden by single-step evaluations. Second, the bottleneck lies after initial access in post-compromise state utilization. Third, LLM agents are susceptible to environmental interference. When decoy endpoints are deployed in the range, agent timeouts triple and downstream completion disappears despite the success rates of initial accesses are unchanged.
Figures & tables
| Research line | Representative evaluations | Typical task starting point | Primary endpoint and residual blind spot |
|---|---|---|---|
| Knowledge and executable CTFs | CyberSecEval; Cybench; InterCode-CTF [ 3 , 55 , 52 ] | Question, challenge statement, attachment, or interactive service | Correct item or captured flag; success need not demonstrate operation across independent enterprise hosts. |
| Vulnerability discovery and reproduction | CyberGym; SEC-Bench Pro; VulnLMP [ 50 , 28 , 40 ] | Vulnerability description and code, historical engine source, or hardened research target | Differentially or cross-version verified PoC, crash, or reproducible evidence; full exploitation and lateral movement are not always required. |
| Exploit construction | ExploitBench; ExploitGym [ 29 , 49 ] | Known vulnerability materials or an input that already triggers the flaw | Milestone signals or independently confirmed unauthorized code execution; evaluation is typically centered on one vulnerable target. |
| Remote and long-horizon cyber operations | CVE-Bench; AgentCyberRange; OpenAI Cyber Range; UK AISI [ 58 , 31 , 38 , 13 ] | Remote application, exposed service, or high-level network objective | Externally checked target state, final scenario objective, or deepest reached step; defensive observability is generally not a co-equal outcome. |
Table 1: Representative cyber-agent benchmark families. The endpoint states what a successful result establishes and what it does not establish by itself.
| Evaluation | Tool Interaction | Multi-Host Attack Chain | Multi-Stage Attack Chain |
|---|---|---|---|
| CyberSecEval Series [ 3 ] | Partial | — | Partial |
| Cybench / InterCode-CTF [ 55 , 52 ] | Full | — | Partial |
| CyberGym [ 50 ] | Full | — | Partial |
| ExploitBench [ 29 ] | Full | — | Full |
| ExploitGym [ 49 ] | Full | — | Full |
| CVE-Bench [ 58 ] | Full | — | Full |
Table 2: Task coverage: tool interaction and attack-chain continuity.
| Evaluation | Environment Qualification | Isolation of System Factors | External Outcome Verification |
|---|---|---|---|
| CyberSecEval Series [ 3 ] | Partial | Partial | Partial |
| Cybench [ 55 ] | Full | Full | Full |
| InterCode-CTF [ 52 ] | Partial | Partial | Full |
| CyberGym [ 50 ] | Full | Partial | Full |
| ExploitBench [ 29 ] | Full | Full | Full |
| ExploitGym [ 49 ] | Full | Partial | Full |
Table 3: Evaluation methodology: environment, factor isolation, and verification.
Figure 1: Overview of SysEvolve . Three components form a self-driven adversarial co-evolution loop. SysField provides the range substrate, SysSpear generates attack schemes, and SysArmor detects and responds. Attack outcomes feed defense improvement, and detection results feed attack refinement.
Figure 2: Overview of SysField . SysField integrates three tightly coupled planes: safe and efficient log collection, CRADLE-guided cyber-range construction from reusable vulnerable images, and context-aware weakness orchestration with multi-layer verification. Together, they provide execution evidence, reproducible range deployment, and process-grounded evaluation of multi-stage cyber attacks.
| System challenge | Core solution | Design outcome |
|---|---|---|
| Complete evidence competes with finite host resources and can be disrupted by event floods | Isolated NoDrop collection and DPUaudit offloaded collection | A common, gap-detectable event stream whose failure behavior is explicit under both software-only and hardware-assisted deployment |
| Complex topologies contain heterogeneous and conflicting deployment constraints and may fail during long-running materialization | CRADLE high-level description, constraint compilation, desired-state execution, and checkpointed recovery | A portable topology specification plus a reproducible deployment state and recovery history |
| Independent vulnerabilities do not automatically form executable attack chains, while result-only scoring rewards shortcuts | Context-aware weakness composition, environment/component isolation, and multi-layer multi-source verification | A contained range with an execution-confirmed attack path and process-grounded evaluation evidence |
Table 4: The three core design solutions in SysField .
Figure 3: The two complementary loss-resilient collection paths used by SysField . NoDrop charges full-buffer draining to the producing thread to prevent cross-producer eviction [ 23 ] , while DPUaudit employs a DPU-assisted pull architecture with device-side buffer rotation, record reconstruction, and deduplication [ 24 ] . The mechanism diagrams are adapted from the corresponding source papers.
Figure 4: CRADLE is compiled into backend-specific IaC, then provisions the same logical range on heterogeneous testbeds.
Figure 5: Overview of context-aware weakness orchestration in SysField . Reproducible weaknesses are first normalized into capability-oriented Atoms. Dependency-compatible Atoms are then composed into enterprise-range templates, and the resulting candidate ranges are validated through end-to-end execution with a private verifier.
Figure 6: Overview of SysSpear . Three co-designed subsystems interact around the Strategy Board for knowledge alignment and evolution. With the guidance by expert knowledge from SecFlow, the attack scheme generator produces attack scheme candidates. The Static Check component then verifies correctness and safety of them, yielding a verified attack scheme as the output of SysSpear .
Figure 7: Strategy-Board-mediated attack-scheme generation in SysSpear . The Coordinator slices the current Board into bounded Assignments; Context Projection delivers only each Assignment’s relevant subgraph; isolated Solver Runs return structured evidence to the same versioned Board.
Figure 8: An attack instance generated by SecFlow using expert knowledge skills. The dependency graph guides the agent through a multi-stage campaign. Each skill’s preconditions are matched against Board evidence, and newly produced postconditions unlock subsequent skills.
Figure 9: Static-analysis-verified attack scheme checking in SysSpear . Orax checks dependency satisfiability via SMT solving. Eunomia and SymGx check software safety via fine-grained symbolic execution and SGX-specific analysis.
Figure 10: Overview of SysArmor .
Figure 11: Rule compilation pipeline with four phases
Figure 13: Overview of the NodLink pipeline. Normal provenance events are ingested from the operation stream, cached in memory, and progressively expanded from suspicious terminal nodes into a hopset for comprehensive attack detection.
Figure 14: Overview of the KnowHow pipeline. Suspicious provenance subgraphs produced by NodLink are matched against CTI-derived knowledge, organized into candidate attack lifecycles, and refined into precise and interpretable alerts.
Figure 15: Overview of STAIR. STAIR maintains an evolving incident state with Graph-as-State (GAS), performs stage-aware response planning with historical experience, and incorporates execution feedback to form a closed-loop automated response process.
| Comparison | Shared setup | Changed factor and interpretation |
|---|---|---|
| L2 model comparison | Same 50 cases, OpenAI-compatible SDK and runner, L2 context, tools, 300-turn budget, 3,600-s agent timeout, serial execution, verifier, and SysArmor configuration | Kimi K3 versus DeepSeek-V4-Pro; same-protocol descriptive model comparison, with one trajectory per model–case pair |
| Same-backbone attack system | Same 50 cases, DeepSeek-V4-Pro, public L2 scenario, 300-turn budget, 3,600-s timeout, serial execution, private verifier, and observe-only SysArmor | Single-agent runner versus SysSpear assessment/session adapter; configuration-level comparison because the runner and execution policy change together |
| L1 interference | Same manifest, DeepSeek-V4-Pro, L1 context, 300 turns, 3,600-s timeout, seed 1, and temperature 0 | none : parallelism 8; high : parallelism 4, 43 decoys, and one omitted topology hint, in fixed order; comparison of recorded configurations, not a pure decoy effect |
| Collector completeness | Same stress-oriented event-generation and accounting protocol on the reported hardware/virtualization settings | SysField versus Sysdig and LTTng; event-loss comparison only |
| Collector overhead | Same reported workload family within the overhead experiment | SysField versus HARDLOG and OMNILOG; overhead comparison only |
| Composition and scale | Same model within each reported pair | CyberGym single-step versus SysField multi-step, and 7-node versus 50-node; source reports omit trial-level uncertainty and some controls, so deltas remain protocol-specific |
Table 5: Baseline setup and comparison boundaries for the SysField evaluation.
| Metric | DPUAudit-backed SysField | Reported baselines |
|---|---|---|
| Event-loss rate | 0.0% | Sysdig: 98.5%; LTTng: 60.0% |
| Additional system overhead | 2.1% | HARDLOG: 23.2%; OMNILOG: 33.0% |
Table 6: Reported completeness and overhead of the DPUAudit-backed SysField collector. Completeness and overhead use different baseline sets.
| Model | CyberGym single-step | SysField multi-step | Change |
|---|---|---|---|
| Kimi K3 | 73.8% | 55.7% | pp |
| GLM-5.2 | 62.8% | 41.2% | pp |
| DeepSeek-V4-Pro | 57.8% | 36.2% | pp |
| GPT-5.6 Luna | 44.3% | 23.5% | pp |
Table 7: Reported attack success on single-step CyberGym tasks and SysField multi-step ranges.
| Model | 7-node range | 50-node range | Change |
|---|---|---|---|
| Kimi K3 | 60.8% | 50.2% | pp |
| GLM-5.2 | 46.1% | 36.0% | pp |
| DeepSeek-V4-Pro | 40.9% | 30.6% | pp |
| GPT-5.6 Luna | 29.0% | 19.2% | pp |
Table 8: Reported attack success as SysField range size increases from 7 to 50 nodes.
Figure 16: Stage-level completion on the same 50 Stratified-50 L2 cases. Bars report independently verified target flags. The model gap widens after initial access.
| Attack configuration | Backbone | Target 1 | Target 2 | Target 3 |
|---|---|---|---|---|
| Single-agent | Kimi K3 | 22/50 (44%) | 18/50 (36%) | 16/50 (32%) |
| Single-agent | DeepSeek-V4-Pro | 19/50 (38%) | 10/50 (20%) | 6/50 (12%) |
| SysSpear | DeepSeek-V4-Pro | 34/50 (68%) | 15/50 (30%) | 7/50 (14%) |
Table 9: Stage progression under the recorded L2 configurations. SysSpear uses DeepSeek-V4-Pro as its solver backbone; the comparison holds the backbone fixed but changes the attack-system architecture and recorded execution policy.
Figure 17: Same-backbone progression profile on the 50 Stratified-50 L2 cases. The gap between the DeepSeek-V4-Pro single-agent arm and SysSpear narrows from 30 percentage points at Target 1 to 10 at Target 2 and 2 at Target 3. Because the comparison changes the attack-system runner and execution policy, it should be interpreted as a configuration-level comparison rather than a component-level causal ablation.
| Metric | none | high |
|---|---|---|
| Target 1 / Target 2 / Target 3 | 2 / 2 / 2 | 2 / 0 / 0 |
| All-three completion | 2/50 | 0/50 |
| Objective achieved | 1/50 | 0/50 |
| Timeout | 6/50 (12%) | 19/50 (38%) |
| Mean agent time | 1,417.6 s (23.6 min) | 2,428.5 s (40.5 min) |
| Median agent time | 1,091.7 s (18.2 min) | 2,536.5 s (42.3 min) |
Table 10: DeepSeek-V4-Pro L1 results under the current none and high configurations.
Figure 18: Attack completion and defensive observability on 50 L2 cases. “Any new signal” records whether at least one new attack-window signal appears; “strict expected hit” requires the complete expected rule set for the case.
| Expected behavior | Kimi K3 | DeepSeek-V4-Pro | SysSpear |
|---|---|---|---|
| Execution-oriented tool opens a network connection | 12/40 (30.0%) | 23/40 (57.5%) | 20/50 (40.0%) |
| Network client is used inside a workload | 9/40 (22.5%) | 18/40 (45.0%) | 19/50 (38.0%) |
| Shell or interpreter executes inside a workload | 6/40 (15.0%) | 16/40 (40.0%) | 14/50 (28.0%) |
Table 11: Missing expected behaviors. Direct-run columns use 40 paired cases; the SysSpear 300-turn column uses all 50 formal rerun cases.
| System | Model configuration | L2 (9 cases) | L3 (4 cases) | Overall (13 cases) |
| SysSpear | DeepSeek-V4-Pro + GLM | 7/9 (77.8%) | 1/4 (25.0%) | 8/13 (61.5%) |
| GLM-5.1 | 7/9 (77.8%) | 1/4 (25.0%) | 8/13 (61.5%) | |
| Qwen 3.6 | 5/9 (55.6%) | 0/4 (0.0%) | 5/13 (38.5%) | |
| DeepSeek-V4-Pro-Flash | 8/9 (88.9%) | 1/4 (25.0%) | 9/13 (69.2%) | |
| DeepSeek-V4-Pro-Flash + skills | 7/9 (77.8%) | 3/4 (75.0%) | 10/13 (76.9%) | |
| Claude Code | GLM-5.1 | 6/9 (66.7%) | 1/4 (25.0%) | 7/13 (53.8%) |
Table 12: XBow 13-case comparison by difficulty for five SysSpear configurations and two Claude Code baselines. Level-2 and Level-3 contain 9 and 4 cases, respectively.
| Rule Name | Alert Count |
|---|---|
| command_injection_indicators | 7600 |
| crontab_modification | 7525 |
| permission_modification | 6670 |
| suspicious_binary_execution | 4865 |
| clear_log_activities | 4454 |
| dev_shm_execution | 3677 |
Table 13: Rule-level alert counts produced by the rule-based detection engine during the one-week deployment.
| Attack Stage | ttp | Malicious Command Line | ||||
|---|---|---|---|---|---|---|
| Initial Compromise | T1570 | /tmp/.>, /var/tmp/.>, /tmp/sshd, /root/sshd | ||||
| Establish Foothold | T1105 | /usr/sbin/sshd -D, sshd [accepted], sshd nobody [priv] | ||||
| Move Laterally | T1021.004 | sshd root [priv], sshd operator [priv] | ||||
| Escalate Privilege | T1222 | (cd ‘/var/tmp’ && [ -f ‘yum.log’ ] && (chattr -i ‘yum.log’; chmod +x ‘yum.log’ && ./‘yum.log’)) | ||||
| T1036 | (cd ‘/var/tmp’ && [ -f ‘apt.log’ ] && (chattr -i ‘apt.log’; chmod +x ‘apt.log’ && ./‘apt.log’)) | |||||
| T1022.002 | chmod +x yum.log, chmod +x .bash_logout, /usr/bin/ydvlmeowey su 12812 |
Table 14: Detection results produced by the NodLink + KnowHow inference pipeline against the real-world apt campaign. NodLink constructs the suspicious provenance subgraph, and KnowHow maps the observed behavior to MITRE ATT&CK techniques and attack stages.
| Job Configuration | Throughput (eps) |
|---|---|
| Job 01 + Job 02 | 12547 |
| Job 01 + Job 03 | 12208 |
| Job 01 + Job 02 + Job 03 | 9003 |
Table 15: Throughput of SysArmor under different Flink job configurations. “eps” denotes events per second.
Figure 19: Memory consumption of the SysArmor server during the evaluation period.
More in this topic
Cybersecurity
Cybersecurity develops methods to protect digital and cyber-physical systems from attacks, vulnerabilities, data loss, and misuse.
See all 355 papers