Recent advancements in Large Language Models (LLMs) have sparked interest in their application to Static Application Security Testing (SAST), primarily due to their superior contextual reasoning capabilities compared to traditional symbolic or rule-based methods. However, existing LLM-based approaches typically attempt to replace human experts directly without integrating effectively with existing SAST tools. This lack of integration results in ineffectiveness, including high rates of false positives, hallucinations, limited reasoning depth, and excessive token usage, making them impractical for industrial deployment. To overcome these limitations, we present a paradigm shift that reorchestrates the SAST workflow from current LLM-assisted structure to a new LLM-centered workflow. We introduce Argus (Agentic and Retrieval-Augmented Guarding System), the first multi-agent framework designed specifically for vulnerability detection. Argus incorporates three key novelties: comprehensive supply chain analysis, collaborative multi-agent workflows, and the integration of state-of-the-art techniques such as Retrieval-Augmented Generation (RAG) and ReAct to minimize hallucinations and enhance reasoning. Extensive empirical evaluation demonstrates that Argus significantly outperforms existing methods by detecting a higher volume of true vulnerabilities while simultaneously reducing false positives and operational costs. Notably, Argus has identified several critical zero-day vulnerabilities with CVE assignments.
Figures & tables
Figure 1: A toy Python example of vulnerability detection. The function subprocess.call( ⋅ , shell=True) acts as the sink , which may receive injected malicious input from the source ip and cause it to be executed. Data flow of this vulnerability is represented by the connecting purple arrows.
Figure 2: The overall workflow of the Argus framework. Given a code repository, Argus first parses it with its third-party dependencies and retrieves potential issues or vulnerabilities associated with them. For suspected vulnerabilities, Argus generates proofs of concept (PoCs) as well as corresponding repairs. It then integrates with CodeQL for sink identification and employs our Re 3 (Retrieval, Recursion, and Review) method, which combines forward and backward scanning to achieve comprehensive data flow extraction. Finally, the framework produces a detailed report containing all detected vulnerabilities, along with their data flows and verification details, for human review. All information shown in this figure is derived from actual execution results of Argus.
Figure 3: A practical example of the structured information summerized by our RAG agent. It summarizes all necessary information for the locating of potential sinks given a target dependency (e.g., org.apache.poi:poi-ooxml ).
Figure 4: Example of PoC generation and vulnerability verification. The POC agent will think about the possibility of writing proof-of-concept example code which can successfully exploit the codebase. In this way, we can ensure the correctness of sinks and start our data flow analysis in the next step.
Repository
Stars
Commits
Forks
LoC
Issues
PublicCMS
2.1k
3,479
828
335k
1
JeecgBoot
44.9k
2,137
15.7k
642k
19
Ruoyi
7.9k
1,942
2.2k
124k
16
JSPWiki
111
9,742
102
161k
12
DataGear
1.6k
4,574
367
190k
1
Yudao-Cloud
18.4k
2,614
4.6k
230k
1
Table 1: Details of the evaluation codebases. This table presents the GitHub stars, forks, commits, Lines of Code (LoC), and the number of open issues for each repository.
Repository
CodeQL
IRIS
Argus (Claude-Sonnet-3.5)
Argus (Claude-Sonnet-4.5)
# Sinks
# Vuln.
# Sinks
# Vuln.
# Sinks
# Vuln.
# Sinks
# Vuln.
PublicCMS
-
0
5840
0
8
6
8
6
JeecgBoot
-
0
8350
0
9
3
9
3
Ruoyi
-
0
980
0
3
3
3
3
JSPWiki
-
0
1828
1
7
5
7
5
DataGear
-
0
237
0
11
3
11
3
Table 2: End-to-End Vulnerability Detection Evaluation. Argus significantly outperforms CodeQL in traditional vulnerability detection. IRIS fails in realistic SAST scenarios due to the lack of required CWE information.
Repository
Argus
IRIS
Tokens
Vuln.
Tokens
Vuln.
PublicCMS
194,538
6
35,500
0
JeecgBoot
254,392
3
42,400
0
Ruoyi
32,836
3
22,300
0
JSPWiki
67,605
5
24,638
1
DataGear
9,301
3
20,300
0
Table 3: LLM API token consumption and vuln. discovery comparison.
# Vuln./# Sinks
Bare Argus
Extra with RAG
PublicCMS
5/7
1/1
JeecgBoot
3/9
0/0
Ruoyi
3/3
0/0
JSPWiki
5/7
0/0
DataGear
3/10
3/1
Yudao-Cloud
0/0
0/0
Table 4: Vulnerabilities resulting from original sink detection and extra sinks detected by RAG agents.
Figure 5: Examples of vulnerable data flows in DataGear detected by Argus.
Figure 6: Token consumption and sink discovery performance trade-off among different Argus backbones and codebases.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Screenshot of one of our realistic attack examples injected via our zero-day vulnerability discovered with Argus. The repository is PublicCMS.
Figure 8: An example prompt of our vulnerability review agent in Argus.
Figure 9: The final vulnerability analysis report exported by Argus.
The advent of agentic vulnerability detection is already becoming a watershed moment for software security. Audits conducted entirely by autonomous LLM agents are uncovering critical vulnerabilities in fundamental software underpinning digital society. Many of these vulnerabilities remained masked for years, surfacing only now with AI agents. Yet the reasoning behind these discoveries remains alarmingly opaque and unvalidated. What assumptions did the agent make about a function's inputs when it deemed that function to be secure? Failures in reasoning and incorrect assumptions can lead to missed vulnerabilities and reduce trust in agentic analysis. We propose a security-specification-first paradigm that (1) exposes the agent's tacit assumptions explicitly as security specifications and (2) continuously refines those specifications via runtime falsification. We realize our approach in Code-Augur, a novel harness for agentic vulnerability detection. Given a codebase, Code-Augur analyzes each component of the system for vulnerable code. When it deems a component to be secure, it commits the local invariants behind that judgment as in-source assertions. In parallel, Code-Augur leverages a guided fuzzer to attempt to falsify those assumptions. When the fuzzer triggers an assertion, this either reveals a genuine vulnerability or a flawed specification to refine. In both cases, this process grounds the agent's understanding, aligning its view of code intent with how the code actually behaves. On real-world subjects, Code-Augur effectively leverages security specifications to detect more vulnerabilities than other state-of-the-art agents. Additionally, Code-Augur found 22 new vulnerabilities in key open-source projects. Compared to curated specialized models like Claude Mythos, Code-Augur offers effective agentic vulnerability detection built on widely available LLMs like Sonnet and DeepSeek.
Security bugs often slip past static analysis not because they are impossible to detect, but because detection rules have not yet been written for the code in front of us. New frameworks, APIs, and coding idioms continually create such gaps, leaving developers to discover failures and manually encode new rules. We present SESAA, a framework in which an Large Language Model (LLM) agent starts from no rules and iteratively evolves a set of readable and auditable joern queries. The agent receives feedback only from a second LLM critic, which inspects flagged and unflagged code to identify likely false alarms and suspected misses, without access to ground-truth labels. The agent then uses this feedback to revise its queries. On the OWASP Benchmark for Python, SESAA reaches a mean F1 of 0.742 across 8 independent runs, well above the default configurations of five widely used tools and comparable to the strongest of them once it is tuned with access to ground-truth labels that SESAA never sees. In a pilot on real-world Python projects, queries induced from code available before a vulnerability was introduced detect 10 of 19 subsequent CVEs, including 9 detected by rules written for the same repository. The resulting detector is a set of executable, human-readable queries that engineers can inspect, modify, and run independently. These results suggest that LLM agents can turn static-analysis rule maintenance from a manually authored process into an iterative process of automated discovery and repair.
Automated vulnerability detection is a fundamental task in software security, yet existing learning-based methods still struggle to capture the structural dependencies, domain-specific vulnerability knowledge, and complex program semantics required for accurate detection. Recent Large Language Models (LLMs) have shown strong code understanding ability, but directly prompting them with raw source code often leads to missed vulnerabilities or false alarms, especially when vulnerable and benign functions differ only in subtle semantic details. To address this, we propose VulTriage, a triple-path context augmentation framework for LLM-based vulnerability detection. VulTriage enhances the LLM input through three complementary paths: a Control Path that extracts and verbalizes AST, CFG, and DFG information to expose control and data dependencies; a Knowledge Path that retrieves relevant CWE-derived vulnerability patterns and examples through hybrid dense--sparse retrieval; and a Semantic Path that summarizes the functional behavior of the code before the final judgment. These contexts are integrated into a unified instruction to guide the LLM toward more reliable vulnerability reasoning. Experiments on the PrimeVul pair test set show that VulTriage achieves state-of-the-art performance, outperforming existing deep learning and LLM-based baselines on key pair-wise and classification metrics. Further ablation studies verify the effectiveness of each path, and additional experiments on the Kotlin dataset demonstrate the generalization ability of VulTriage under low-resource and class-imbalanced settings. Our code is available at https://github.com/vinsontang1/VulTriage
Wenxin Tang, Xiang Zhang, Junliang Liu +11
Tsinghua University · Henan University · Dalian Maritime University +8