When triaging unknown binaries, security analysts must decide which samples to clear as benign, flag as malicious, or escalate for deeper analysis. Large Language Models (LLMs) could support these decisions by interpreting decompiled code, but a binary can contain hundreds or thousands of functions, making exhaustive LLM analysis impractical. We study how current LLMs behave when static triage is formulated as budgeted evidence acquisition: which functions they inspect and how they trade off benign clearance, malware detection, and escalation. To enable a controlled comparison, we implement TRIAGE-CPG, a fixed pipeline over a code property graph (CPG) of a decompiled binary. An LLM-free heuristic selects seed functions, an agent expands the resulting evidence dossier under a fixed budget, and a reporting stage produces a benign, suspicious, or malicious verdict together with a behavioral report. We evaluate eight closed-source and open-weight models on 1,000 Windows PE binaries, using a 600-sample test split and the same zero-shot protocol for every model. We assess the ternary verdicts under both a strict binary reading and triage balanced accuracy (tBA), which credits malware that is flagged or escalated and benign samples only when cleared. After analyzing about 40 of a mean 1,000 functions per binary, the highest-scoring model reaches a strict balanced accuracy of 0.805 and a tBA of 0.855, flagging 65% of malware, escalating another 12%, and clearing 94% of benign samples. The models implement different triage policies: high-specificity models produce small, malware-rich escalation queues, whereas recall-oriented models escalate or flag much more broadly. Both closed-source and open-weight models appear across this spectrum. Our results identify no universal deployment configuration: model choice itself determines the clear/flag/escalate trade-off under limited static evidence.
Figures & tables
Figure 1. TRIAGE-CPG pipeline. The CPG and PE metadata of a decompiled Windows PE binary provide the static evidence. EPS selects seed functions to initialize a dossier; an exploration agent may expand it under a fixed function-read budget. A reporting agent reads the frozen dossier and returns a ternary verdict and behavioral report; suspicious samples are escalated. Diagram of the TRIAGE-CPG workflow. A Windows PE binary is transformed by Ghidra and a code property graph into static evidence containing CPG functions, calls, APIs, and PE metadata. This evidence feeds a dashed TRIAGE-CPG box. Seed analysis initializes an evidence dossier; evidence exploration queries the CPG and reads from and writes to the dossier; verdict reporting reads the frozen dossier. The system produces a benign, suspicious, or malicious verdict and a report. Suspicious samples are sent to an escalation queue for deeper analyst analysis.
Figure 2. Recovered functions per binary by label (log scale). Boxes show the interquartile range and median, diamonds the mean, and dots individual binaries. Counts reach 49,519 (median 304; mean 1,208 ), with benign binaries denser than malicious ones. Two box-and-jitter plots of valid functions per binary on a logarithmic vertical axis, from one to more than ten thousand. The benign distribution, shown in blue, lies higher and is more spread out than the malicious distribution, shown in orange. The benign median is 364 recovered functions and the malicious median is 207. Both distributions are strongly right-skewed, with individual binaries reaching 49,519 functions.
Input
Bal. Acc.
Recall
Spec.
FP/FN
Metadata only (strict)
0.505
0.010
1.000
0/99
Metadata only (triage)
0.770
1.000
0.540
46/0
Code only
0.670
0.900
0.440
56/10
Code + metadata
0.795
0.890
0.700
30/11
Table 1. Development-split evidence ablation (Gemini 3.1 Flash-Lite). Metadata-only rows use both verdict readings; code rows use fixed seeding without exploration under the strict reading.
Figure 3. Development-split seed-selector performance (strict verdict, exploration off). Diagonals mark constant balanced accuracy, so selectors on a common line perform equally well. The random baseline is the mean of five runs. Scatter plot of recall against specificity for seven seed-function selectors, overlaid with diagonal reference lines of constant balanced accuracy from 0.65 to 0.85. EPS has recall 0.92 at specificity 0.63, and largest code and EPS without the code term sit just below it at recall 0.89 to 0.90 and specificity 0.62 to 0.63. Out-degree has recall 0.89 at the higher specificity of 0.68. These four selectors all fall between the 0.75 and 0.80 reference lines. API count has the same recall as EPS, 0.92, but a much lower specificity of 0.51, placing it near the 0.70 line. The random mean lies at recall 0.76 and specificity 0.69, also near the 0.70 line, and in-degree is lowest at recall 0.70 and specificity 0.57.
Cap
Avg. analyzed
Bal. Acc.
Recall
Spec.
FP/FN
50
35.5
0.820
0.870
0.770
23/13
75
44.8
0.790
0.890
0.690
31/11
100
52.0
0.790
0.910
0.670
33/9
Table 2. Development-split seed-cap sensitivity under the full pipeline and strict verdict; all other configuration values are fixed.
Figure 4. Verdict composition on the locked test set, one row per model sorted by tBA, over (a) the 300 true-malware and (b) the 300 true-benign binaries. Both panels stack the correct verdict first ( malicious leftmost in (a), benign in (b)). Each reading is a cut point: strict recall is the malicious segment of (a), and triage recall adds the adjacent suspicious one. The right block reports triage and strict balanced accuracy (tBA, BA) and the coverage and precision of the suspicious queue. Open-weight models are marked † . Short names: Pro/Flash/FL = Gemini 3.1 Pro/3.5 Flash/3.1 Flash-Lite, DSv4 = DeepSeek V4 Pro, G31/G26 = Gemma 4 31B/26B. Two panels of horizontal stacked bars, one row per model, ordered by triage balanced accuracy from Gemini 3.1 Pro at the top to Gemma 4 26B at the bottom. Panel (a) covers the 300 true-malware binaries and stacks the malicious, suspicious, and benign shares from left to right. Panel (b) covers the 300 true-benign binaries and stacks the benign, suspicious, and malicious shares from left to right. In both panels the leftmost segment is the correct verdict, so bars that are long on the left indicate better performance. Gemini 3.1 Pro has a short correct segment in panel (a) but the longest in panel (b), while Gemma 4 26B shows the opposite pattern. DeepSeek V4 Pro has by far the widest suspicious segment in both panels.
Figure 5. Strict and triage operating points on the locked test set. Each model is a segment from its strict verdict (small filled dot) to its triage reading (numbered open circle). Models are numbered 1–8 by tBA as in Figure 4 , closed-source blue and open-weight orange. Scatter plot of recall against specificity for the eight models. Each model is drawn as a line segment from its strict operating point, a small filled dot, to its triage operating point, a numbered open circle, colored blue for closed-source and orange for open-weight models. A key numbers the models one to eight by triage balanced accuracy. Reinterpreting suspicious verdicts as escalation moves every model up and to the left, raising recall and lowering specificity, by amounts that vary widely across models.
#
System
BA
tBA
Queue
Prec.
Lost
Pro alone
.805
.855
44
.841
–
1
Pro → Flash
.808
.860
35
.943
2
2
Pro → GLM 5.2
.815
.857
29
.931
2
3
Pro → G31
.832
.855
18
.889
1
4
Pro → G26
.828
.855
22
.864
0
5
Pro → DSv4
.808
.853
35
.886
2
Table 3. Post-hoc cascades on locked test outputs: top overall and open-weight pairs by tBA, with single-model baselines. Lost is malware cleared by the second model; abbreviations follow Figure 4 .
Model
Reads
Stop
Scratch.
Callees
> 150
Non-adj.
Rep.
Cited
Pro
5.4
87%
56%
21%
47%
46%
1.8
7.7
Flash
9.2
45%
39%
22%
43%
54%
5.9
4.1
FL
6.2
86%
33%
25%
43%
45%
0.0
7.2
DSv4
7.1
31%
45%
14%
35%
46%
3.6
4.1
GLM 5.2
3.6
90%
45%
11%
43%
42%
1.4
5.6
G31
8.1
75%
30%
21%
49%
47%
0.3
4.8
Table 4. Exploration and reporting behavior. Reads = mean exploration analyses; Stop = explicit done ; Scratch./Callees = target-source shares; > 150 = outside the EPS top 150; Non-adj. = not adjacent to a seed; Rep. = mean reporting inspections; Cited = explored-to-seed citation-rate ratio. Abbreviations follow Figure 4 .
Figure 6. Agreement between report characterizations and external VirusTotal-derived threat-intelligence evidence on the malware half of the locked test set, one group of bars per model, split into closed-source and open-weight groups. Grouped bar chart of agreement percentages on malware samples, one group of four bars per model, with the three closed-source models on the left and the five open-weight models on the right, separated by a dashed divider. Within each group the bars are malware class, capability, primary role, and family name. Capability agreement is highest for every model, from roughly 75 to 96 percent. Primary-role agreement is around 66 to 83 percent, followed by malware-class agreement, around 56 to 76 percent. Family-name agreement is near zero for every model, roughly 1 to 6 percent, leaving a wide gap below the other three bars.
Malware
Benign
Model
TP
Susp.
FN
TN
Susp.
FP
Gemini 3.1 Pro
12%
70%
54%
33%
57%
17%
Gemini 3.5 Flash
10%
57%
50%
34%
56%
12%
Gemini 3.1 Flash-Lite
27%
50%
22%
20%
39%
51%
DeepSeek V4 Pro
14%
54%
30%
18%
37%
48%
GLM 5.2
12%
68%
46%
34%
32%
23%
Table 5. Wrapper share within each verdict cell. Detect-It-Easy identifies packers, protectors, or installers; the baseline is 29% for malware and 33% for benign samples. Per-cell counts vary, so the mean row is the robust reference.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Binary size
n
Seeds per binary
10
25
50
75
100
150
Small ( < 100)
63
0.819
0.795
0.732
0.766
0.787
0.787
Mid (100–750)
68
0.560
0.821
0.815
0.778
0.741
0.692
Large ( > 750)
69
0.639
0.723
0.712
0.686
0.730
0.670
All
200
0.690
0.810
0.775
0.755
0.750
0.720
Appendix
Table 6. Development-split balanced accuracy by binary size and fixed seed count (exploration off). Bold marks each size class’s best count.
Table 7. Development-split exploration analyses under the strict verdict. (a) Net benign/malware binaries corrected minus hurt by enabling exploration across nine seed/reporting configurations; bold marks the deployed configuration. (b) Agent-guided and random targeting under full and action-matched budgets, reporting realized function reads and balanced-accuracy ranges. (c) Seed-selector performance under the full pipeline.
Malware analysts often inspect compiled binaries through decompiled pseudo-C, when source code is unavailable. Recent work suggests that large language models (LLMs) can assist this process by classifying decompiled code as benign or malicious, but existing pipelines typically rely on a single decompiler view. We argue that this assumption is fragile: decompilers are lossy heuristic tools, and different decompilers can expose different artefacts of the same binary. We curate a benchmark of benign utilities and malicious programs spanning a range of threat behaviors. Each sample is compiled and decompiled with both Ghidra and RetDec, yielding matched pseudo-C views. Across a range of LLMs from major model families, we find that providing both decompiler views improves malicious-class F1, mainly by increasing recall on malicious samples. Agreement analyses further show that Ghidra and RetDec make partially different errors, supporting the view that decompiler outputs provide complementary evidence. Our results suggest that multi-decompiler prompting is a simple, training-free way to improve LLM-based malware triage in practical settings.
Large language models can triage malware through direct verdicts or behavioral claims scored by an external policy. We present MARS, a malware triage framework, and compare direct classification with single-pass claim scoring using the same evidence collector and identical static evidence bundles for each model. The evaluation covers 1,195 PE and ELF binaries grouped into 1,001 near-duplicate clusters and six language models, with deterministic rules providing a baseline. Direct classification is more accurate for all six models. On samples with usable outputs from both paths, its accuracy advantage ranges from 3.7 to 20.9 percentage points, with all 95% cluster-bootstrap confidence intervals for the differences above zero. It also achieves higher malicious alert recall in ten of twelve platform and model combinations. Claim mediation provides no consistent reduction in performance variation across models. Separate subset studies find more consistent alert decisions for direct classification and a larger recall loss for the claim path when predefined indicator fields are removed. In a family identification probe, claims yield higher accuracy than verdict labels but lower accuracy than evidence text. Retained claims expose the inputs to verdict computation and permit policy revision without another model call. We reproduce archived verdicts exactly and apply a revised policy to the same records, including outputs from two additional models withdrawn by their provider. Under the evaluated claim taxonomy and additive policy, these results favor direct classification when only a verdict is required, while demonstrating that retained claims support explicit policy inspection and revision.
Hyeongjun Choi
78ResearchLab, 8, Yangpyeong-ro 25-gil, Yeongdeungpo-gu, Seoul, 07207, Republic of Korea
Large language models are being integrated into malware triage workflows as reasoning components that summarize static evidence and produce analyst-facing verdicts. This paper shows that the same reasoning capability introduces a new attack surface. We present ALIBI, a semantic cover story attack against frontier LLM-based malware analyzers. ALIBI adds a small, non-executed read-only section to a compiled binary, containing a coherent but false security product narrative, without altering imports or executable behavior. Instead of issuing direct instructions to the model, it reframes suspicious evidence as expected behavior of a benign endpoint security tool. On a frozen PE set of 50 malicious samples, the payload flips 30 of the 35 baseline-malicious samples to benign on Gemini 2.5 Pro, while GPT-5.5 Pro and Claude Opus 4.7 produce substantial severity downgrades with significant confidence reductions even when verdict labels are preserved. The attack transfers to ELF binaries, where Gemini flips 16 of 40. A verification-guided defense prompt roughly halves the benign verdicts, but 42.9 percent of malicious samples still reach benign. LLM malware analyzers therefore require provenance checks that separate verified facts from attacker-controlled claims, not narrative trust.