A Hybrid Approach for Automated Security Pattern Recovery
Organizations: University College Dublin, Dublin, Ireland
Abstract
Security-by-design emphasises integrating security into software design and development. Security pattern catalogues operationalise this principle by prescribing reusable architectural solutions to recurring security problems. However, recovering their implementations from source code remains challenging because security patterns are abstract, span multiple components, and admit many valid implementations. We present SecPatRec, a hybrid approach for automated security pattern recovery from source code repositories. We treat a security pattern as a reference architecture and frame its recovery as a deductive task: determining which prescribed roles are present, grounding them in code entities, and recovering their interaction sequences. SecPatRec pairs a ReAct-style LLM agent equipped with semantic and lexical code search for role detection and grounding with static code property graph analysis for interaction recovery. We curate a human-validated ground-truth dataset of 100 pattern instances covering five security patterns across 93 open-source Java and Python repositories, and introduce a three-level evaluation framework measuring recovery correctness against this ground truth. We evaluate SecPatRec using four LLMs, including proprietary and open-source models. Averaged over all models and pattern instances, SecPatRec achieves role-detection accuracy of 0.84, role-grounding similarity of 0.84, and an interaction-recovery score of 0.66. Compared to a one-shot end-to-end LLM baseline, SecPatRec achieves higher aggregate recovery scores and lower estimated LLM API costs per run, while processing repositories that exceed the baseline's configured input limit. The recovered role mappings and statically derived interaction sequences provide structural and behavioural evidence that can support inspection of security-pattern implementations.
Figures & tables
| Symbol | Meaning |
|---|---|
| Pattern specification | |
| pattern specification | |
| , | prescribed roles; their actions, in order |
| Codebase | |
| , | codebase, and its named code symbols extracted from union of all ASTs |
| Recovered Architecture | |
| #Repos | KLOC | #Methods | #Classes | #Tokens | #Files | CPG (MB) | |
|---|---|---|---|---|---|---|---|
| Java | 44 | 3,628.2 | 179,975 | 37,478 | 32.12M | 32,628 | 228.3 |
| Python | 49 | 2,555.4 | 92,162 | 17,307 | 20.89M | 10,622 | 590.2 |
| Total | 93 | 6,183.6 | 272,137 | 54,785 | 53.01M | 43,250 | 818.5 |
| Python | Java | |||||||
|---|---|---|---|---|---|---|---|---|
| Pattern | Pres. | Ext. | Miss. | Int. | Pres. | Ext. | Miss. | Int. |
| P1 | 32 | 17 | 21 | 31 | 33 | 14 | 23 | 39 |
| P2 | 31 | 9 | 0 | 24 | 28 | 12 | 0 | 27 |
| P3 | 33 | 0 | 7 | 19 | 30 | 6 | 4 | 28 |
| P4 | 42 | 0 | 18 | 19 | 31 | 11 | 18 | 25 |
| P5 | 40 | 0 | 0 | 34 | 34 | 6 | 0 | 34 |
Appendix figures & tables35 assets
Supplementary material from the paper’s appendix.
Appendix
| Edge | Joern step | Connects | Label |
|---|---|---|---|
| Control flow | cfgFirst , _cfgOut | a node to the node of the same method that runs next | — |
| Call | call.callee | a call to the method it runs | call |
| Reference | identifier._refOut | a name to the variable, parameter or field it refers to | data |
| Method reference | methodRef.referencedMethod | a function passed as a value (a callback, a lambda, a nested function) to its method | route-registration if passed to a call, else nested-definition |
| Added edge | Leads to | Built from | Label |
|---|---|---|---|
| Import binding | the repository method that the calling file’s import names | Joern’s import nodes | call |
| Receiver class (Python) | the method on the class the code assigns to the object a call is made on | Joern’s assignment nodes | call |
| Field access | the field that a read or write refers to | Joern’s field-access node and the object’s class | data |
| Override dispatch (Java) | the repository’s implementations of a library interface method | Joern’s class hierarchy, method names and parameter counts | dispatch |
| Framework route (Java) | the repository code that Spring Security passes a login request to | a rule written for that framework | dispatch |
| Injection provider (Java) | the @Bean method that builds the object a call is made on | Joern’s annotation nodes and return types | dependency-injection |
| #Roles | |||||
|---|---|---|---|---|---|
| Pattern | #Repos | Present | External | Missing | #Interactions |
| P1 Password-based authentication | 10 | 33 | 14 | 23 | 39 |
| P2 Verifiable-token authentication | 10 | 28 | 12 | 0 | 27 |
| P3 Obscure-token access control | 10 | 30 | 6 | 4 | 28 |
| P4 Session-based access control | 10 | 31 | 11 | 18 | 25 |
| P5 Rate limiting | 10 | 34 | 6 | 0 | 34 |
| #Roles | |||||
|---|---|---|---|---|---|
| Pattern | #Repos | Present | External | Missing | #Interactions |
| P1 Password-based authentication | 10 | 32 | 17 | 21 | 31 |
| P2 Verifiable-token authentication | 10 | 31 | 9 | 0 | 24 |
| P3 Obscure-token access control | 10 | 33 | 0 | 7 | 19 |
| P4 Session-based access control | 10 | 42 | 0 | 18 | 19 |
| P5 Rate limiting | 10 | 40 | 0 | 0 | 34 |
| LLM | Highest MLS | MLS |
|---|---|---|
| GPT-6.1 | 54 | 27 |
| DeepSeek | 44 | 20 |
| GLM | 37 | 17 |
| Muse-Spark | 43 | 16 |
| RDA | MLS | IRC | ||||
| LLM | Base | Ours | Base | Ours | Base | Ours |
| All 100 instances | ||||||
| GPT-6.1 | 0.74 | 0.79 | 0.70 | 0.83 | 0.51 | 0.72 |
| DeepSeek | 0.67 | 0.86 | 0.65 | 0.86 | 0.51 | 0.61 |
| GLM | 0.70 | 0.86 | 0.69 | 0.84 | 0.51 | 0.66 |
| Muse-Spark | 0.71 | 0.84 | 0.69 | 0.83 | 0.51 | 0.64 |
| Accuracy | New input (%) | Output (%) | Steps | Memory (MB) | |||||||
| LLM | RDA | MLS | IRC | Py | Java | Py | Java | Py | Java | Py | Java |
| No keyword search | |||||||||||
| GPT-6.1 | -0.009 | -0.015 | -0.001 | 0 | -2 | -8 | -6 | -0.6 | -0.5 | 0 | 0 |
| DeepSeek | -0.002 | -0.002 | +0.088 | +16 | +11 | -11 | +4 | -0.6 | +0.8 | 0 | 0 |
| GLM | +0.008 | +0.001 | -0.032 | +40 | +12 | -6 | -3 | -0.7 | -0.5 | 0 | 0 |
| Muse-Spark | +0.023 | +0.009 | +0.032 | -33 | -30 | -7 | -5 | -0.4 | -1.0 | 0 | 0 |
| Artifact | Link | Contents | Used in |
|---|---|---|---|
| Dataset | https://huggingface.co/buckets/AnonymousSpottedToad/securityPatternsRecovery | The 100 groundtruth architectures; the 93 codebases and their CPGs; the outputs of every experiment; the five specifications ; the Claude Code session that drafted the candidate mappings | Sections IV-B and V ; Appendices A-A and D |
| SecPatRec | https://github.com/anonymousspottedtoad/secpatrec | The role detection and mapping agent, the one-shot baseline, the two ablations and the secpat command that runs them | Sections III-B and V |
| Interaction recovery | https://github.com/anonymousspottedtoad/secpat_interaction_recovery | The traversal of the Joern CPG: walk.sc , with the rules per language in java.sc and python.sc | Section III-B2 ; Appendix C |
| Evaluation metrics | https://github.com/anonymousspottedtoad/secpat_evaluation_metrics | RDA, MLS and IRC | Section IV-C ; Appendix A-D |
| Annotation interface | https://github.com/anonymousspottedtoad/secpat_annotation_portal | The web interface the annotators used to verify the candidate architectures | Appendix D-B |
Software Security
Software security studies how to find, prevent, validate, and repair vulnerabilities in code, systems, dependencies, and software supply chains.
See all 99 papers