What Malware Actually Does: An Empirical Analysis of Post-Infection Behavior Across Malware Families
Organizations: Artificial Intelligence Research (AIR) Center, University of North Dakota, Grand Forks, ND, USA · School of Electrical Engineering and Computer Science, University of North Dakota, Grand Forks, ND, USA
Abstract
Most malware detection research focuses on identifying whether a file is malicious before it executes. Far less attention goes to what happens after infection. This paper presents an empirical analysis of post-infection malware behavior across 37 families, grounded in dynamic sandbox execution using CAPEv2. Behavioral traces are modeled as heterogeneous graphs with five node types and seven typed edge relationships, preserving relational structure that flat feature vectors discard. A voting ensemble trained on graph-derived structural features achieves 94.44% accuracy and MCC of 0.863, outperforming a flat behavioral baseline by 7.4 percentage points in MCC. A multiclass classifier distinguishes 24 malware families, and a per-family behavioral fingerprint analysis characterizes post-infection activity across registry, network, process, and API dimensions. An LLM explanation layer generates analyst-readable reports with MITRE ATT&CK mappings grounded in observed behavioral evidence. The full pipeline is deployed as HeteroScan at https://ama-andam.github.io/heteroscan.
Figures & tables
| Family | Submitted | Rich Graphs | Rate |
|---|---|---|---|
| Vidar | 65 | 61 | 94% |
| Formbook | 40 | 36 | 90% |
| njRAT | 54 | 47 | 87% |
| XWorm | 76 | 51 | 67% |
| NanoCore | 205 | 93 | 45% |
| AsyncRAT | 244 | 91 | 37% |
| Node Type | Description |
|---|---|
| Process | Executable observed spawning during execution |
| File | File path read from or written to |
| Registry | Registry key queried or modified |
| Network | External endpoint contacted via TCP or UDP |
| API | Windows API function called by a process |
| Edge Type | Source Target | Condition |
|---|---|---|
| spawns | Process Process | Parent PID child PID |
| calls | Process API | Any API call |
| reads_file | Process File | API contains Read/Open/Query |
| writes_file | Process File | All other file API calls |
| reads_reg | Process Registry | API contains Open/Query |
| writes_reg | Process Registry | All other registry API calls |
| Category | Count | Description |
|---|---|---|
| Raw node counts | 5 | Count per node type |
| Total node count | 1 | Sum of all node counts |
| Pairwise ratios | 7 | API/proc, reg/proc, net/proc, etc. |
| Log transforms | 5 | per node type |
| Node statistics | 30 | Mean, std, max, min, sum, active |
| fraction per node type (6 5) |
| Classifier | Key Hyperparameters |
|---|---|
| Random Forest | n_estimators=500, max_features=sqrt, |
| class_weight=balanced | |
| Extra Trees | n_estimators=500, max_features=sqrt, |
| class_weight=balanced | |
| XGBoost | n_estimators=500, max_depth=6, |
| learning_rate=0.05, subsample=0.8, |
| Model | Acc | F1 | Prec | Rec | MCC |
|---|---|---|---|---|---|
| Random Forest | 0.9471 | 0.9638 | 0.9399 | 0.9888 | 0.8698 |
| Extra Trees | 0.9418 | 0.9600 | 0.9395 | 0.9814 | 0.8561 |
| XGBoost | 0.9497 | 0.9647 | 0.9630 | 0.9665 | 0.8772 |
| Ensemble | 0.9444 | 0.9617 | 0.9429 | 0.9814 | 0.8628 |
| Representation | Features | Acc | F1 | MCC |
|---|---|---|---|---|
| Flat behavioral counts | 5 | 0.9127 | 0.9399 | 0.7828 |
| Graph-derived features | 55 | 0.9418 | 0.9596 | 0.8565 |
| Improvement | +50 | +0.029 | +0.020 | +0.074 |
| Model | Acc | F1 | MCC | FP/FN |
|---|---|---|---|---|
| Heterogeneous GNN | 0.763 | 0.846 | 0.382 | 141 / 30 |
| Ensemble | 0.944 | 0.962 | 0.863 | 16 / 5 |
| Metric | Value |
|---|---|
| Accuracy | 0.6891 |
| Macro F1 | 0.5818 |
| Weighted F1 | 0.6773 |
| Families | 24 |
| Test samples | 267 |
| Family | Proc | File | Reg | Net | API |
|---|---|---|---|---|---|
| BlackMatter | 5 | 1 | 5036 | 1 | 118 |
| QuasarRAT | 5 | 1 | 1190 | 2 | 147 |
| WSHRAT | 3 | 1 | 1136 | 2 | 98 |
| DCRat | 5 | 1 | 1025 | 1 | 137 |
| NanoCore | 5 | 1 | 734 | 3 | 143 |
| Babadeda | 5 | 1 | 814 | 2 | 102 |
| Family | Proc | API | File | Reg | Net | Conf |
|---|---|---|---|---|---|---|
| njRAT | 3 | 133 | 1 | 461 | 3 | 99.9% |
| AsyncRAT | 4 | 117 | 1 | 1,264 | 1 | 97.3% |
| NanoCore | 4 | 122 | 1 | 218 | 1 | 99.8% |
Malware Analysis
Malware analysis studies how to detect, classify, attribute, and understand malicious software and its behavior.
See all 66 papers