Cryptographic digests (e.g., MD5, SHA-256) are designed to provide exact identity. Any single-bit change in the input produces a completely different hash, which is ideal for integrity verification but limits their usefulness in many real-world tasks like threat hunting, malware analysis and digital forensics, where adversaries routinely introduce minor transformations. Similarity-based techniques address this limitation by enabling approximate matching, allowing related byte sequences to produce measurably similar fingerprints. Modern enterprises manage tens of thousands of endpoints with billions of files, making the effectiveness and scalability of the proposed techniques more important than ever in security applications. Security researchers have proposed a range of approaches, including similarity digests and locality-sensitive hashes (e.g., ssdeep, sdhash, TLSH), as well as more recent machine-learning-based methods that generate embeddings from file features. However, these techniques have largely been evaluated in isolation, using disparate datasets and evaluation criteria. This paper presents a systematic comparison of learning-based classification and similarity methods using large, publicly available datasets. We evaluate each method under a unified experimental framework with industry-accepted metrics. To our knowledge, this is the first reproducible study to benchmark these diverse learning-based similarity techniques side by side for real-world security workloads. Our results show that no single approach performs well across all dimensions; instead, each exhibits distinct trade-offs, indicating that effective malware analysis and threat-hunting platforms must combine complementary classification and similarity techniques rather than rely on a single method.
Figures & tables
K
Benign
Malicious
All
Mean
Std
Mean
Std
Mean
Std
1
0.51
0.49
0.8
0.39
0.66
0.47
10
3.55
4.34
7.31
4.16
5.43
4.65
50
12.31
19.24
32.59
22.12
22.45
23.07
100
19.84
34.82
61.00
45.22
40.42
45.31
Table 1: Label homogeneity (ssdeep [ 15 ] )
K
Benign
Malicious
All
Mean
Std
Mean
Std
Mean
Std
1
0.89
0.32
0.90
0.31
0.89
0.31
10
8.44
2.33
7.31
8.56
8.5
2.48
50
40.27
11.57
32.59
40.87
40.57
12.52
100
78.69
23.12
79.69
27.18
79.19
25.24
Table 2: Label homogeneity (Autoencoder)
Model
Accuracy
Precision
Recall
F1 Score
AUC
XGBoost (Binary)
0.9776
0.9776
0.9776
0.9776
0.9971
Deep Learning (Binary)
0.9604
0.9553
0.9661
0.9607
0.9926
Table 3: Malware classification performance
Model
Accuracy
Top-5 Accuracy
XGBoost (AVClass)
0.9055
0.9748
Deep Learning (AVClass)
0.8015
0.9244
Table 4: Global AVClass classification accuracy metrics
Malware poses a critical and ever-evolving threat, and robust and effective systems for detecting and classifying malware are of essential importance. n-grams features are among the common static features used in effective machine learning systems for malware, but these features are inherently brittle. We propose an algorithm for constructing more robust features, hamm-grams, which are a special class of regular expressions having a fixed length and single-character wildcards. We devise an efficient algorithm for finding common hamm-grams using a new locality-sensitive hash designed to produce collisions among pairs of small Hamming distance and a clustering within hash buckets to place wildcards. We then demonstrate the advantages of these features in malware classification and detection tasks.
Traditional malware detection methods struggle to generalize to obfuscated or previously unseen threats. This paper introduces ThreatVisionAI, a hybrid malware family classification framework that integrates a raw-image CNN, a wavelet-based CNN, and a Vision Transformer (ViT) to capture complementary spatial, frequency-domain, and global relational features in malware images. The wavelet-based CNN captures multi-scale frequency information that helps distinguish closely related families, while the ViT branch models long-range dependencies across the image. Evaluated on the Malimg dataset, ThreatVisionAI achieves 98.01% accuracy and a weighted F1 score of 0.9742, with wavelet-domain features providing measurable gains on minority and visually similar families. These results confirm that frequency-aware and transformer-based representations improve image-based malware family classification.
Allyson Taylor, Prashanth BusiReddyGari
Department of Mathematics & Computer Science University of North Carolina Pembroke
Existing malware detection approaches heavily rely on handcrafted features derived from static or dynamic analysis of the executable files. These methods require specialized domain expertise, dedicated analysis tools, and time-consuming analysis, which limits their scalability and robustness in the face of the increasing complexity and diversity of modern malware. Consequently, there is a growing interest in approaches that operate directly on raw bytes, which contain rich semantic and structural information. However, capturing long-range dependencies within executable files remains challenging for conventional static and CNN-based methods. This paper presents MalwareFormer, a transformer-based, byte-level malware detection framework that operates directly on raw bytes, without requiring domain knowledge or feature extraction. MalwareFormer achieves an accuracy of 98.35%, a balanced accuracy of 98.32%, a macro-F1 of 98.32%, and a false positive rate of 1.94%, outperforming both baselines across all threshold-dependent classification metrics and demonstrating that transformers provide a powerful and scalable approach for byte-level malware detection.
Abdelrahman Yassin Eljamal, Abdeltawab Hendawi
Department of Computer Science, University of Rhode Island, Kingston, USA