Malware family classification remains a challenging task in automated malware analysis, particularly in real-world settings characterized by obfuscation, packing, and rapidly evolving threats. Existing machine learning and deep learning approaches typically depend on labeled datasets, handcrafted features, supervised training, or dynamic analysis, which limits their scalability and effectiveness in open-world scenarios. This paper presents a zero-label malware family classification framework based on a weighted hierarchical ensemble of pretrained large language models (LLMs). Rather than relying on feature-level learning or model retraining, the proposed approach aggregates decision-level predictions from multiple LLMs with complementary reasoning strengths. Model outputs are weighted using empirically derived macro-F1 scores and organized hierarchically, first resolving coarse-grained malicious behavior before assigning fine-grained malware families. This structure enhances robustness, reduces individual model instability, and aligns with analyst-style reasoning.
Figures & tables
Figure 1: A sample from SBAN dataset.
Family
Description
Trojan
Generic malware performing malicious actions without self-propagation
Worm
Self-replicating malware that spreads autonomously
Virus
Malware attaching to host executables for propagation
Ransomware
Malware that encrypts or locks data for extortion
Backdoor / Remote Access Trojan
Malware enabling persistent remote control
Dropper
Malware whose primary role is payload installation
Table 1: Malware families used for classification
Figure 2: Zero-shot malware family classification prompt used to elicit decision-level predictions from pretrained large language models.
Figure 3: Proposed zero-shot LLM ensemble pipeline for malware family classification. A shared classification prompt is applied to multiple LLMs, followed by normalization, weighted hierarchical ensembling, and gold-based evaluation.
Families
Behavior Group
Trojan, Backdoor/RAT, Spyware/Infostealer, Bot
TROJAN_LIKE
Dropper, Downloader
INSTALLER
Worm, Virus
SELF_REPLICATING
Ransomware
RANSOMWARE
Packed / Obfuscated Malware
OBFUSCATED
Table 2: Mapping of malware families to behavior groups.
Model
Accuracy
Macro-P
Macro-R
Macro-F1
Qwen
0.695
0.492
0.325
0.380
CodeLLaMA
0.420
0.278
0.238
0.159
GPT-4.1
0.710
0.416
0.422
0.403
GPT-5.1
0.675
0.476
0.437
0.427
Table 3: Performance of Individual LLMs on Malware Family Classification
Method
Accuracy
Macro-P
Macro-R
Macro-F1
Uniform Voting (Row 1)
0.740
0.546
0.464
0.481
Weighted Voting (Row 2)
0.735
0.527
0.464
0.469
Weighted + Hierarchy (Initial)
0.725
0.507
0.462
0.460
Weighted + Hierarchy (Final)
0.750
0.530
0.479
0.481
Table 4: Ablation Study of Ensemble Strategies
Figure 4: Accuracy comparison of individual LLMs and ensemble strategies on the 200-sample gold standard. The final weighted hierarchical ensemble achieves the highest overall accuracy.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Size
Labels
Modalities
Strengths / Limitations
EMBER (2018) ( Anderson and Roth, 2018b )
1.1M
Binary
PE metadata
Widely used benchmark for static ML detection; lacks family labels and semantic context.
SOREL-20M (2020) ( Harang and Rudd, 2020 )
20M
Weak tags
Metadata
Large-scale dataset enabling detection research; lacks curated family structure or multimodal signals.
BODMAS (2021) ( Yang and others, 2021 )
134K
581 fam.
Struct. feats.
Supports temporal drift and family-level evaluation; limited modality diversity and no text/code semantics.
MalNet (2022) ( Freitas et al., 2022 )
5.1M
696 fam.
API graphs
Graph-based behavioral dataset for GNN research; requires graph extraction and lacks semantic textual information.
MalwareBazaar
350K+
Mixed
Binaries
Community repository for real malware; labels noisy and no benign baseline.
Maldict (2023) ( Joyce and others, 2023 )
5.5M
Behavior
Dyn. traces
Multi-attribute behavior dataset; no multimodal code–text alignment.
Appendix
Table 5: Overview of benchmark datasets used in malware classification research.
Malware Family
Number of Samples
Trojan
82
Backdoor / Remote Access Trojan
44
Spyware / Infostealer
42
Downloader
11
Worm
6
Bot / Botnet Client
5
Appendix
Table 6: Malware Family Distribution in the Gold Set
Prompt
Model
Acc.
Macro-P
Macro-R
Macro-F1
P1
Qwen
0.878
0.584
0.441
0.483
CodeLLaMA
0.718
0.468
0.347
0.271
GPT-4.1
0.142
0.033
0.071
0.045
GPT-5.1
0.122
0.138
0.097
0.052
FinalLabel
0.284
0.163
0.299
0.207
P2
Qwen
0.852
0.181
0.222
0.197
Appendix
Table 7: Prompt sensitivity results (P1–P5) on the labeled evaluation set. Metrics are reported per model and for the aggregated output ( FinalLabel ).
Figure 5: Prompt sensitivity (ensemble output): Accuracy and Macro-F1 of FinalLabel across prompts P1–P5.
Figure 6: Prompt sensitivity: Macro-F1 across prompts P1–P5 for each LLM and the aggregated output ( FinalLabel ).
Department of Computer Applications, Cochin University of Science and Technology, India · Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy