SEEK: Agentic Knowledge Graph Retrieval for Secure Code Generation
Organizations: University of Duisburg-Essen, Germany · University of Bristol, UK · Technical University of Darmstadt, Germany
Abstract
Large Language Models (LLMs) can generate functionally correct code while introducing exploitable security weaknesses, particularly when developers specify functionality without explicit security requirements. Recent state-of-the-art approaches mitigate this problem using external security knowledge, but largely rely on flat similarity-based retrieval over independent artifacts, making it difficult to infer the relevant vulnerability space and retrieve targeted knowledge when direct matches are unavailable. We propose SEEK, an agentic Graph RAG framework for secure code generation that constructs a security knowledge graph using the Common Weakness Enumeration (CWE) taxonomy as its structural backbone and injects it with curated real-world CVE vulnerability-fix instances. At inference time, a Planner Agent predicts potential CWE weaknesses associated with the developer query, which guide Graph Scoping to restrict retrieval to relevant regions of the CWE taxonomy before deterministic multi-faceted candidate ranking. A Selection Agent subsequently validates the predicted weaknesses and retrieved candidates prior to code generation. Across five open-source LLMs, we evaluate SEEK in an out-of-distribution setting on CodeGuard+ and CyberSecEval, spanning C, C++, and Python. SEEK achieves the highest SecurePass@1 of 77.45% on CodeGuard+ and the highest aggregate ICD pass rate of 79.45% on CyberSecEval, surpassing prior approaches. Moreover, SEEK achieves 85.2% SecurePass@1 across 13 unseen CWE types with no directly linked vulnerability-fix instances, demonstrating strong generalizability, while maintaining acceptable inference latency costs.
Figures & tables
| 1 | int main( int argc, char *argv[]) { |
| 2 | // get a directory name from the command line |
| 3 | char *dirname = argv[1]; |
| 4 | // list the contents of that directory using "ls" |
| // command line utility | |
| 5 | // print the contents of the directory |
| 6 | char cmd[256] = {0}; |
| Generator Model | Method | Pass@1 | SecurePass@1 | Secure Rate | Secure@Pass1 |
| Gemma4-31B | Zero-Shot | 93.617 | 64.681 | 68.936 | 64.481 |
| SEEK | 89.574 | 77.447 | 85.532 | 79.574 | |
| Gemma3-27B | Zero-Shot | 83.830 | 55.532 | 65.319 | 56.809 |
| SEEK | 81.064 | 62.979 | 72.128 | 67.021 | |
| Qwen2.5-Coder-32B | Zero-Shot | 85.106 | 61.277 | 68.723 | 61.702 |
| SEEK | 81.702 | 69.574 | 82.979 | 74.681 |
| Method | Generator | SP@1 | SR | Pass@1 |
| Same Generator: Qwen2.5-Coder-32B | ||||
| Zero-shot | Qwen2.5-C-32B | 61.2 | 68.7 | 85.1 |
| SecCoder | Qwen2.5-C-32B | 58.1 | 66.6 | 84.7 |
| Codexity | Qwen2.5-C-32B | 57.1 | 71.6 | 80.7 |
| INDICT | Qwen2.5-C-32B | 40.4 | 84.4 | 48.1 |
| CoSec | Qwen2.5-C-32B | 41.9 | 54.9 | 65.1 |
| CWE | Weakness | # Tasks | SP@1 |
| CWE-95 [ 22 ] | Eval Injection | 1 | 100.0 |
| CWE-113 [ 5 ] | HTTP Request/Response Splitting | 2 | 50.0 |
| CWE-117 [ 6 ] | Log Output Neutralization | 3 | 60.0 |
| CWE-170 [ 7 ] | Improper Null Termination | 1 | 100.0 |
| CWE-215 [ 8 ] | Sensitive Information in Debugging Code | 1 | 100.0 |
| CWE-312 [ 10 ] | Cleartext Storage of Sensitive Information | 1 | 100.0 |
| Method | C | C++ | Python | Avg. |
| SVEN [ 30 ] | 60.35 | 70.27 | 74.93 | 68.52 |
| SafeCoder [ 31 ] | 63.00 | 75.29 | 78.63 | 72.31 |
| CoSec [ 39 ] | 59.91 | 70.66 | 76.64 | 69.07 |
| SoSecure [ 47 ] | 52.86 | 73.36 | 71.79 | 66.00 |
| RCI [ 61 ] | 55.51 | 75.29 | 76.64 | 69.15 |
| CodeGuarder [ 40 ] | 70.48 | 83.40 | 84.33 | 79.40 |
| Generator | Planner | Retrieval | Selector | Generation |
| Gemma4-31B | 10.00 | 0.227 | 8.53 | 13.65 |
| Gemma3-27B | 12.39 | 0.010 | 1.98 | 7.64 |
| Configuration | Pass@1 | SP@1 | SR | S@P1 |
| Flat RAG | 86.809 | 73.404 | 83.457 | 77.074 |
| Graph Scoping Only | 88.085 | 75.957 | 85.532 | 79.362 |
| Selection Agent Only | 86.596 | 73.830 | 84.043 | 76.809 |
| SEEK | 89.574 | 77.447 | 85.532 | 79.574 |
| Retrieval Facets (Weight) | Pass@1 | SP@1 | SR | S@P1 |
| Root Cause (RC) only | 84.255 | 73.830 | 87.447 | 77.447 |
| Vul. Code (VC) only | 89.574 | 76.596 | 84.468 | 79.574 |
| RC (0.75), VC (0.25) | 85.532 | 72.766 | 85.106 | 75.532 |
| RC (0.50), VC (0.50) | 85.106 | 72.553 | 84.468 | 77.447 |
| RC (0.25), VC (0.75) | 89.574 | 77.447 | 85.532 | 79.574 |
Secure Code Generation
Secure code generation studies how to produce code that meets functional requirements while avoiding security vulnerabilities.
See all 48 papers