A First Look at System One Decision Models for Malware Detection
Organizations: Department of Engineering, Niccolò Cusano University, Rome, Italy
Abstract
Malware detection needs scores calibrated at a fixed false positive rate (FPR) and decisions an analyst can read: feature-based detectors give the first, large language models (LLMs) on decompiled code the second, through generated text of uncalibrated confidence. System One decision models answer a typed question about a text with a probability over its declared answers in one forward pass. We present the first evaluation of these models for Windows Portable Executable malware detection, analyze how they decide, and propose S1Mal, an ensemble on the typed answers and the embedding of CLM-8B. In an isolated virtual machine, Ghidra and capa decompose each binary, and the host renders the records into a canonical textual state, a digest of headers, imports, MITRE ATT&CK capabilities and strings, optionally with decompiled functions, without class shortcuts. Three readouts score each model on the state: A0, the probability of one verdict question; A1, 32 atomic behavioral questions weighed by gradient-boosted trees; A2, heads on the model's state embedding. S1Mal feeds gradient-boosted trees with the atomic answers and the A2 score of CLM-8B. We evaluate CLM-8B, ten open decision models and two LLMs read through logits on BODMAS temporal splits, SOREL-20M out-of-distribution malware and 14,534 matched benign files. On BODMAS, A0 wastes the static evidence: its area under the receiver operating characteristic curve (AUROC) spans 0.4025-0.8590 across 13 models. A1 recovers it, raising every model's AUROC by +54.61% on average; A2 lifts CLM-8B by +0.2749 to 0.9943, above a combiner on digest facts (0.9901), and reaches 0.9853-0.9948 on nine open models. S1Mal reaches 0.9946 and detects 93.47% of the malware at a realized FPR of 1.45%, and 59.15% at 0.1% FPR on the test ROC curve, against 46.59% with A2. S1Mal gives analysts calibrated decisions, each with the atomic answers behind it.
Figures & tables
| Work | Evidence | Output read | Calibration | Fixed FPR | Time split |
|---|---|---|---|---|---|
| Chawla and Prasad [ 6 ] | Ghidra pseudo-C | JSON verdict | newer test set | ||
| PE2Prompt [ 37 ] | EMBER features | forced label | n.r. | n.r. | |
| MARS [ 8 ] | evidence bundle | JSON verdict, rule-scored claims | |||
| Kurlandski et al. [ 17 ] | bytes, assembly, pseudo-C | trained language model | n.r. | n.r. | ✓ |
| AsmRAG [ 15 ] | Ghidra assembly | nearest-neighbor vote | n.r. | per function | ✓ |
| SecJev [ 7 ] | traffic, logs, telemetry | typed probability | ✓ | n.r. | transfer |
| Block | Content | Cap |
|---|---|---|
| B1 File | kind, size class, overlay, resources, TLS callbacks, signature | 60 |
| B2 Sections | count, entry section, roles or quoted names, size, ratio and entropy classes, write and execute flags | 180 |
| B3 Packing | Detect It Easy findings, packed flags of detector and heuristic | 40 |
| B4 Imports | DLL count, ordinal class, sensitive APIs by category, system DLLs | 350 |
| B5 Capabilities | capa rules with ATT&CK and MBC identifiers, rarest first | 300 |
| B6 Strings | counts per category and up to 40 quoted strings | 300 |
| Model | Size | Architecture | Window (tok.) |
|---|---|---|---|
| Kev-0.8B [ 29 ] | 0.8B | Qwen3.5, LoRA, pointer head | 8,192 |
| decider-0.8b [ 26 ] | 0.8B | Qwen3.5, fully fine-tune | 8,192 |
| decider-2b | 2B | Qwen3.5, full fine-tune, RL, merged LoRA | 8,192 |
| decider-4b (Q4_K_M) | 4B | Qwen3.5, full fine-tune, merged LoRA | 8,192 |
| Strands Decider [ 36 ] | 2B | Qwen3.5, LoRA, pointer head | 4,096 |
| Von [ 30 ] | 395M | ModernBERT-large encoder | 8,192 |
| Exp. | Question | Primary endpoint |
|---|---|---|
| E1 | Do System One models detect malware, and what do their readouts add over the digest facts and over LLMs read through logits? | TPR at on TEST, with the realized FPR on |
| E1b | Do heads on the embedding of every open model add over its atomic answers? | paired AUROC difference of A2 and A1 per model |
| E2 | Which parts of the state carry the decision? | paired AUROC difference per level of the state |
| E3 | Which questions and which combiner serve the decision? | paired AUROC difference per question design |
| E4 | Are the scores calibrated, and how much does abstention buy? | paired ECE difference before and after recalibration |
| E5 | How do inference time, memory and energy on a laptop GPU compare with the workstation, and do the answers agree? | equivalence of two serving paths of CLM-8B |
| Arm | Input | AUROC [95% CI] | pAUC 1% | TPR at | FPR | ECE |
|---|---|---|---|---|---|---|
| DET | S0 facts | 0.9901 [0.985, 0.994] | 0.8284 | 88.24% | 1.91% | 0.0139 |
| Qwen3-Embedding probe | S2 | 0.9956 [0.993, 0.997] | 0.8678 | 93.62% | 1.42% | 0.0066 |
| Qwen3-8B logits, A1 | S2 | 0.9493 [0.929, 0.964] | 0.6013 | 36.30% | 0.95% | 0.0235 |
| Foundation-Sec-8B logits, A1 | S2 | 0.9733 [0.962, 0.981] | 0.6595 | 64.19% | 1.58% | 0.0184 |
| decider-4b (GGUF Q4_K_M), A1 | S0 | 0.9834 [0.975, 0.989] | 0.7530 | 75.44% | 1.29% | 0.0167 |
| CLM-8B A2-ctrl probe | S2 | 0.9955 [0.993, 0.997] | 0.8974 | 92.94% | 1.11% | 0.0104 |
| ECE | Band at 1% risk | |||||
|---|---|---|---|---|---|---|
| Arm | raw | recal. | Map | Brier | Coverage | Risk |
| CLM-8B A0 | 0.3691 | 0.0479 | isotonic | 0.2060 | 0 (infeasible) | - |
| Qwen3-8B logits, A0 | 0.3025 | 0.0164 | isotonic | 0.1833 | 0 (infeasible) | - |
| Foundation-Sec-8B logits, A0 | 0.1542 | 0.0316 | Platt | 0.1537 | 16.01% | 3.10% |
| decider-2b A0 | 0.1620 | 0.0642 | isotonic | 0.2141 | 4.60% | 3.63% |
| Kev-0.8B A0 | 0.1084 | 0.0719 | isotonic | 0.2194 | 0 (infeasible) | - |
| Machine | Weights | AUROC | TPR at | FPR | Latency [s] | Files/s | Energy [mWh] |
|---|---|---|---|---|---|---|---|
| Workstation | bf16 | 0.9922 | 90.33% | 1.94% | 0.114 | 8.34 | 9.26 |
| Workstation | Q5_K_M | 0.9859 | 60.40% | 0.43% | 0.112 | 8.54 | 8.77 |
| Laptop | Q6_K | 0.9859 | 50.55% | 0.43% | 0.378 | 2.53 | 8.55 |
| Laptop | Q4_K_M | 0.9788 | 53.28% | 0.86% | 0.349 | 2.75 | 7.82 |
| Laptop | Q5_K_M | 0.9827 | 52.19% | 0.22% | 0.359 | 2.68 | 8.08 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Malicious (28). Command execution; persistence through Run keys, services, scheduled tasks and the startup folder; process injection; process hollowing; privilege escalation; anti-debugging; anti-VM and sandbox checks; obfuscation and packing; disabling security tools; credential access; keylogging; screen and clipboard capture; discovery; lateral movement; command and control over HTTP, DNS and raw sockets; download and execute; dropping files; exfiltration; ransomware encryption; shadow-copy deletion; wiping; cryptocurrency mining; self-propagation. |
| Benign indicators (4). Graphical application; installer or updater; runtime library; low-risk operations only. |
| AUROC [95% CI] | ||||||
|---|---|---|---|---|---|---|
| Model | State | A0 | A1 | A2 | S1Mal | Files |
| CLM-8B | S2 | 0.7194 [0.643, 0.796] | 0.9763 [0.965, 0.985] | 0.9943 [0.991, 0.997] | 0.9946 [0.991, 0.997] | 10,829 |
| Qwen3-8B (logits) | S2 | 0.7388 [0.694, 0.779] | 0.9493 [0.929, 0.964] | - | - | 10,829 |
| Foundation-Sec-8B (logits) | S2 | 0.8590 [0.799, 0.906] | 0.9733 [0.962, 0.981] | - | - | 10,829 |
| decider-4b (Q4_K_M) | S0 | 0.6667 [0.595, 0.740] | 0.9834 [0.975, 0.989] | 0.9936 [0.990, 0.996] | - | 10,829 |
| decider-2b | S0 | 0.6682 [0.574, 0.756] | 0.9824 [0.975, 0.988] | 0.9938 [0.989, 0.997] | - | 10,829 |
Malware Detection
Malware detection uses static and behavioral evidence to identify malicious software and protect devices, networks, and software ecosystems.
See all 60 papers