Machine Learning Transferability for Malware Detection

César Vieira; João Vitorino; Eva Maia; Isabel Praça

cs.CRMar 27, 2026

Machine Learning Transferability for Malware Detection

Authors: César Vieira, João Vitorino, Eva Maia, Isabel Praça

Organizations: GECAD, ISEP, Polytechnic of Porto, Rua Dr. António Bernardino de Almeida, 4249-015 Porto, Portugal

Abstract

Malware continues to be a predominant operational risk for organizations, especially when obfuscation techniques are used to evade detection. Despite the ongoing efforts in the development of Machine Learning (ML) detection approaches, there is still a lack of feature compatibility in public datasets. This limits generalization when facing distribution shifts, as well as transferability to different datasets. This study evaluates the suitability of different data preprocessing approaches for the detection of Portable Executable (PE) files with ML models. The preprocessing pipeline unifies EMBERv2 (2,381-dim) features datasets, trains paired models under two training setups: EMBER + BODMAS and EMBER + BODMAS + ERMDS. Regarding model evaluation, both EMBER + BODMAS and EMBER + BODMAS + ERMDS models are tested against TRITIUM, INFERNO and SOREL-20M. ERMDS is also used for testing for the EMBER + BODMAS setup.

Figures & tables

Year	Paper	Data	Approach	Metrics
2001	Schultz et al. [ 22 ]	Proprietary PE	Imports/strings/byte n-grams $\to$ NB, boosted trees	TPR 0.97; FPR 0.04; Acc 97.1%
2004	Kolter & Maloof [ 12 ]	Proprietary PE	Byte n-grams $\to$ Boosted J48	AUC 0.9958
2015	Wang et al. [ 24 ]	BIG 2015	Ensemble (RF & GBM) on byte/ASM features	0.0028 Log Loss
2015	Saxe & Berlin [ 21 ]	Proprietary PE	CNN on 2D byte–entropy images	$\uparrow$ vs. engineered n-grams
2018	Raff et al. [ 16 ]	Windows PE corpora	1D CNN on raw bytes	Acc 0.940 & AUC 0.981
2018	Anderson & Roth [ 2 ]	EMBER 2017/2018	LGBM	AUC $\approx$ 0.999; 92.9% TPR @0.1% FPR

Table 1: Overview of static malware detection approaches.

Dataset	Size (1 / 0 / -1)	Notes
EMBER -2018	1.1 M (400k / 400k / 300k)	Files scanned in or before 2018
SOREL-20M	$\approx$ 20 M (10M / 10M)	1 Jan 2017 - 10 Apr 2019; temporal splits.
BODMAS	$\approx$ 134 K (57,293 / 77,142)	29 Aug 2019–30 Sept 2020; 581 families.
ERMDS *	$\approx$ 106 K (30,229 / 76,662)	Jan - Dec 2022; Binary/source/packing obfuscations.
TRITIUM	$\approx$ 37 K (19,712 / 17,785)	Naturally occurring 2022 threat samples.
INFERNO	2,864 (1,432 / 1,432)	Red-team/custom C2 malware.

Table 2: Datasets characteristics

Figure 1: Data preprocessing and model evaluation workflow.

red.	dim.	est.	F1 (%)	AUC (%)	1% FPR (%)	0.1% FPR (%)
PCA	128	LGBM	97.16	99.64	94.50	86.98
	256	XGB	97.20	99.66	94.87	86.70
	384	XGB	97.27	99.66	94.26	84.04
XGBFS	128	LGBM	97.71	99.75	95.69	86.77
	256	LGBM	98.26	99.84	97.48	91.43
	384	LGBM	98.27	99.84	97.50	91.25

Table 3: Dimensionality Reduction Results on EMBER & BODMAS

red.	dim.	est.	F1 (%)	AUC (%)	1% FPR (%)	0.1% FPR (%)
PCA	128	ET	96.50	99.44	91.27	74.01
	256	XGB	96.42	99.47	91.05	78.94
	384	XGB	96.51	99.49	91.21	79.59
XGBFS	128	ET	97.73	99.76	95.56	87.69
	256	LGBM	98.00	99.80	96.57	88.82
	384	LGBM	98.12	99.82	96.90	89.61

Table 4: Dimensionality Reduction Results on EMBER , BODMAS and ERMDS

Dataset	Dim.	Est.	F1 (%)	AUC (%)	1% FPR (%)	.1% FPR (%)
INFERNO	128	$\text{RF}^{(1)}$	94.54	98.51	84.66	46.38
	384	$\text{LGBM}^{(1)}$	92.30	97.72	86.47	81.91
	384	$\text{XGB}^{(2)}$	95.43	98.90	86.98	85.75
	384	$\text{LGBM}^{(2)}$	93.04	98.40	86.18	45.15
TRITIUM	384	$\text{LGBM}^{(1)}$	98.22	99.79	96.28	81.10
	384	$\text{XGB}^{(2)}$	98.62	99.83	97.13	65.89

Table 5: Model pair results against other EMBER -v2 datasets

Explore similar work

Trident: Improving Malware Detection with LLMs and Behavioral Features
Apr 30, 2026Rebecca Saul, Jingzhi Jiang, Elliott Chia +1Malware Classification LLMs for Cybersecurity
University of California, Berkeley Berkeley, California, USA
Latent Stability Analysis of Malware Representations Under Feature-Space Perturbations
Jul 27, 2026Bamidele Ajayi, Ken McGarryMalware Classification Adversarial Perturbations
School of Computer Science and Engineering, University of Sunderland, Sunderland, UK
Building an Adversarial Malware Dataset by Family and Type: Generation, Evasion, and Poisoning Evaluation
May 25, 2026David Košťál, Martin JurečekAdversarial Attacks Malware Classification
Department of Information Security, Faculty of Information Technology, Czech Technical University in Prague

Apr 30, 2026J/K move · Enter open · S save

Trident: Improving Malware Detection with LLMs and Behavioral Features

Rebecca Saul, Jingzhi Jiang, Elliott Chia, David Wagner

University of California, Berkeley Berkeley, California, USA

Malware Classification LLMs for Cybersecurity Malware Analysis Malware Detection

Open paper AskPDF arXiv

Traditionally, machine learning methods for PE malware detection have relied on static features like byte histograms, string information, and PE header contents. One barrier to incorporating dynamic analysis features has been the semi-structured nature of sandbox behavior reports. We show that, using the latest generation of large language models with reasoning, it is possible to efficiently process these behavior reports and utilize them as part of a malware detection pipeline. Specifically, we leverage LLMs to generate behavior-based malware detection rules based on a small training set of labeled malware. We find that these detection rules, derived from behavioral features, are much more robust to concept drift than standard static-feature methods, while maintaining practical false positive rates. Finally, we introduce Trident, a system which combines a classic decision tree model over static features, our behavior-based detection rules, and direct LLM analysis of sandbox reports through majority voting. Trident outperforms standard methods using static features, outperforms behavior-based rules alone, and is as resilient to concept drift as active learning methods without requiring retraining.

Abstract

Explore similar work

Trident: Improving Malware Detection with LLMs and Behavioral Features

Latent Stability Analysis of Malware Representations Under Feature-Space Perturbations

Building an Adversarial Malware Dataset by Family and Type: Generation, Evasion, and Poisoning Evaluation