Cost-Aware Hierarchical Multi-Agent Ransomware Detection and Family Attribution under Analysis Budgets
Authors: Mubashar Iqbal, Asifullah Khan, Hifsa Asif, Saddam Hussain Khan, Umme Zahoora, Asifullah Khan
Organizations: Pattern Recognition Lab, DCIS, Pakistan Institute of Engineering and Applied Sciences (PIEAS), Nilore, Islamabad, Pakistan · Deep Learning Lab, Center for Mathematical Sciences, PIEAS, Nilore, Islamabad, Pakistan · Interdisciplinary Research Center for Smart Mobility and Logistics, King Fahd University of Petroleum & Minerals, Dhahran 31261, Saudi Arabia · Department of Creative Technology, AIR University, E9 Campus, Islamabad, Pakistan. · PIEAS Artificial Intelligence Center (PAIC), Nilore, Islamabad, Pakistan
Sandbox execution and memory forensics are among the most constrained resources in malware triage. Static analysis can scale to millions of files, whereas dynamic and memory analysis require minutes of analyst controlled infrastructure for each sample. Despite this difference, multimodal ransomware detectors often apply every modality to every sample, causing analysis cost and time to verdict to increase linearly with sample volume even when static evidence is already sufficient for a decision. We present a cost aware Hierarchical Multi-Agent System that formulates evidence acquisition as a budgeted sequential decision problem. Specialist agents generate schema validated risk signals for each modality, domain controllers aggregate these signals, and a Meta-Orchestrator begins with static evidence and escalates to dynamic and memory evidence only when confidence is insufficient or agents within a controller disagree. An optional, bounded, locally hosted large language model reviewer can adjust a verdict by at most one tier but cannot replace the deterministic pipeline. Each decision is recorded with a complete provenance trace. In multiple runs over 12439 samples from 16 ransomware families and benign samples, the deterministic HMAS achieves F1 0.93 and macro F1 0.97, resolving 57.95% of cases using static evidence alone, 35.83% after adding dynamic evidence, and only 6.21% through the full pipeline. The average internal analysis cost is 6.65 units, compared with 12 for exhaustive analysis, representing a 44.6% reduction. Standalone leave-one-family-out testing further shows that accuracy on families held out during tuning falls to 0.26 to 0.64 outside the Benign and high support classes. We report these results alongside a cost sensitivity analysis, a partial leave one component out ablation, and a full scale comparison with learned early and late fusion and cascade baselines.
Figures & tables
Method
Modalities
Budget-aware
Family attrib.
Memory evidence
Audit trace
XRan [ 12 ]
Dynamic
No
No
No
Explanations
RansoGuard [ 14 ]
Dynamic (early)
Partially (early stop)
No
No
No
Pulse [ 15 ]
Static (assembly)
No
No
No
No
RansomFormer [ 7 ]
Static + dynamic
No
No
No
No
volGPT [ 6 ]
Memory
No
Process-level
Yes
LLM text
Iqbal & Khan [ 19 ]
Static + dynamic + network
No
Yes
No
Agent trace
Table 1: Positioning of HMAS. “Budget-aware” means the method decides at inference time which evidence to acquire.
Figure 1: HMAS architecture: feature-extraction pipeline (left), hierarchical decision pipeline (right), and the optional LLM reasoning layer.
Figure 2: HMAS architecture: feature-extraction pipeline (left), hierarchical decision pipeline (right), and the optional LLM reasoning layer.
Figure 3: Runtime pipeline: static evidence is processed first; the Meta-Orchestrator escalates on insufficient confidence or disagreement, fuses the acquired scores, and optionally invokes the bounded reviewer.
Figure 4: HMAS implementation stack: evidence generation (CAPEv2 sandbox with 24 guest VMs), the asynchronous Python runtime with Pydantic-typed messages and scikit-learn, XGBoost, LightGBM and PyTorch specialist models, and the AutoGen [ 31 ] reviewer on a local Ollama-hosted model (Table 2 ).
Figure 5: Class distribution (16 ransomware families and benign).
Policy
Acc.
Prec.
Rec.
F1
ROC-AUC
Avg. modalities
B1 Static-only
0.9130
0.9988
0.8273
0.9050
0.9983
1.00
B2 Dynamic-only
0.9329
0.9642
0.8994
0.9307
0.9883
1.00
B3 Memory-only
0.5002
1.0000
0.0005
0.0010
0.7644
1.00
B6 Static+Dynamic (fixed)
0.9647
0.9993
0.9302
0.9635
0.9995
2.00
B7 Exhaustive (fixed)
0.9401
0.9993
0.8811
0.9364
0.9995
3.00
H0 HMAS (deterministic)
0.9412
0.9993
0.8831
0.9376
0.9989
0.618
Table 3: Binary detection, full-scale policies, default 0.5 threshold, single run (seed 42).
Policy ( n=100 )
Acc.
Prec.
Rec.
F1
ROC-AUC
PR-AUC (AP)
B4 Early fusion (learned)
0.93
0.93
1.00
0.9637
0.96
1.00
B5 Late fusion (stacked)
0.94
0.9394
1.00
0.9688
1.00
1.00
B8 Fixed cascade
0.99
1.00
0.9892
0.9945
1.00
1.00
B9 Learned budgeted gate
0.98
0.9892
0.9892
0.9892
1.00
1.00
Table 4: Binary detection
Figure 6: F1 vs. mean internal cost for the four policies with a directly logged decision latency value
Reference
Compared policy
Ref.-only correct
Policy-only correct
p -value
B1 Static-only
B6 Static+Dynamic
207
850
1.1×10−92
B1 Static-only
B7 Exhaustive
412
749
6.1×10−23
B1 Static-only
H0 HMAS
342
1054
3.0×10−84
B7 Exhaustive
B6 Static+Dynamic
0
306
3.1×10−92
B7 Exhaustive
H0 HMAS
344
719
8.8×10−31
Table 5: Paired McNemar’s test at seed 42
Policy
MC Acc.
Macro-P
Macro-R
Macro-F1
B1 Static-only
0.9076
0.9765
0.6264
0.7022
B2 Dynamic-only
0.8950
0.8893
0.7502
0.7888
B3 Memory-only
0.5001
0.0833
0.0562
0.0383
B7 Exhaustive
0.9338
0.9806
0.7341
0.8204
H0 HMAS
0.9890
0.9887
0.9890
0.9748
H1 HMAS + LLM review
0.9419
0.9473
0.9361
0.9417
Table 6: Family attribution, default threshold, single run (seed 42). B2/B3 rows are the dynamic/memory domain results from
Class
P
R
F1
Support
Benign
0.9896
0.9994
0.9945
6209
GandCrab
0.9977
0.9873
0.9925
1736
LockBit
0.9905
0.9748
0.9826
1708
Zeppelin
0.9736
0.9946
0.9840
1299
WannaCry
1.0000
0.9947
0.9974
379
Cerber
0.9898
0.9198
0.9535
212
Table 7: Per-class results for H0, all 17 classes, default threshold, n=12,439 (recomputed from the reported multiclass confusion matrix; exactly reproduces the file’s reported macro-averages).
Policy
Mean cost (units)
Cost redn. vs. exhaustive
Mean latency (ms)
p95 latency (ms)
B1 Static-only
4.00
66.67%
15.67
16.08
B6 Static+Dynamic
10.00
16.67%
245.52
255.50
B7 Exhaustive
12.00
0%
249.28
259.29
H0 HMAS
6.65
44.6%
111.34
218.30
Table 8: Internal cost units and mean/p95 inference latency, default threshold, seed 42
Agent pool size
n (cases)
Mean latency (ms)
Throughput (samples/s)
3
7209
37.47
26.69
6
4457
213.47
4.68
7
773
211.34
4.73
Table 9: Throughput observations by agent-pool-size bucket
Route
n
Share
Acc. on route
FN on route
FP on route
Static only
7209
57.95%
0.9900
71
1
Static+Dynamic
4457
35.83%
0.9493
225
1
Full pipeline (+Memory)
773
6.21%
0.4386
432
2
Short-circuit error rate (ransomware missed at the static stage): 71/6230 = 1.14%
Table 10: Routing distribution and per-route outcomes for H0
Figure 7: Measured binary accuracy by escalation route, H0, seed 42. Accuracy falls sharply on the 6.21% of cases that reach the full pipeline.
Row label (as logged)
n
Acc.
TN
FP
FN/TP
“static” (= B1 static-only)
12439
0.9130
6203
6
1076/5154
“dynamic” (= B2 dynamic-only)
12439
0.9329
6001
208
627/5603
“memory” (= B3 memory-only)
12439
0.4994
6209
0
6227/3
− adaptive routing (= B7 exhaustive)
12439
0.9401
6205
4
741/5489
− LLM review (= H0)
12439
0.9412
6205
4
728/5502
Table 11: Leave-one-component-out
λ
B1 Static-only
B6 S+D
B7 Exhaustive
H0 HMAS
0.0
0.9130
0.9647
0.9401
0.9412
0.1
0.5130
-0.0353
-0.2599
0.2765
0.5
-1.0870
-4.0353
-5.0599
-2.3823
1.0
-3.0870
-9.0353
-11.0599
-5.7058
Table 12: Objective value Acc−λ⋅cost at selected λ (seed 42, default threshold).
Quantity
H0 (deterministic)
H1 (LLM-enabled)
Mean cost (internal units)
6.65
14.07
Mean decision latency
111 ms
6,639 ms
p95 decision latency
218 ms
22,751 ms
Invocation rate
–
31.1%
Mean LLM calls per invoked case
–
1.48
Table 13: Cost of enabling the LLM reviewer, default threshold, single run (seed 42, n=12,439 ).
Family
n
Accuracy
Benign
22235
0.9964
Zeppelin
4695
0.6062
GandCrab
6204
0.5608
LockBit
6110
0.5525
Sodinokibi
779
0.4878
Cerber
730
0.3411
Table 14: Standalone per-family accuracy. Families with n<50 (Cyclops, n=2 ) omitted as uninformative.
Ransomware has evolved into a complex, adaptive, and fast-moving adversary category in which static signatures and monolithic classifiers fail to generalise under concept drift, evasion, and behavioural polymorphism. In this paper, we present Agentic SABRE (Semantic-Behavioural Arbitration for Ransomware Evaluation), an uncertainty-aware, neuro-symbolic, multi-agent framework for adaptive ransomware detection. SABRE fuses semantic, representation-based evidence with behavioural, time-window forensic telemetry and employs Monte Carlo Dropout inference to quantify epistemic uncertainty for each agent. We introduce a decision-layer orchestrator that performs risk- and uncertainty-aware triage using two interpretable thresholds: a risk score and an uncertainty budget. High-confidence, high-risk samples are automatically contained, while uncertain or borderline cases are escalated to human analysts, establishing a flexible computational contract between autonomous response and analyst oversight. To support auditability and trust, SABRE integrates post-hoc explainability mechanisms, including gradient saliency, permutation importance, and counterfactual analysis, enabling both local and global interpretation of agent decisions. Extensive evaluation on RDset and RanSMAP demonstrates that Agentic SABRE preserves perfect discrimination on saturated semantic datasets, with AUC equal to 1.0, while improving robustness under weak behavioural signals. It achieves up to a 4.9 percent relative reduction in false escalations at equal recall while maintaining calibrated predictive uncertainty. Counterfactual analysis further shows that semantic and behavioural decisions can be reversed with bounded perturbation cost, indicating stable and interpretable decision boundaries.
Henry Kabuye, Biju Issac, Jeyamohan Neera
School of Computer Science, Northumbria University, Newcastle Upon Tyne, UK
Ransomware poses a serious and fast-acting threat to critical systems, often encrypting files within seconds of execution. Research indicates that ransomware is the most reported cybercrime in terms of financial damage, highlighting the urgent need for early-stage detection before encryption is complete. In this paper, we present RansomTrack, a hybrid behavioral analysis framework to eliminate the limitations of using static and dynamic detection methods separately. Static features are extracted using the Radare2 sandbox, while dynamic behaviors such as memory protection changes, mutex creation, registry access and network activity are obtained using the Frida toolkit. Our dataset of 165 different ransomware and benign software families is publicly released, offering the highest family-to-sample ratio known in the literature. Experimental evaluation using machine learning models shows that ensemble classifiers such as XGBoost and Soft Voting achieve up to 96% accuracy and a ROC-AUC score of 0.99. Each sample analyzed in 9.1 seconds includes modular behavioral logging, runtime instrumentation, and SHAP-based interpretability to highlight the most influential features. Additionally, RansomTrack framework is able to detect ransomware under 9.2 seconds. Overall, RansomTrack offers a scalable, low-latency, and explainable solution for real-time ransomware detection.
Busra Caliskan, Ibrahim Gulatas, H. Hakan Kilinc +1
Department of Computer Engineering, Istanbul Technical University, Istanbul, Türkiye · Department of Research and Development, Orion Innovation, Istanbul, Türkiye · Department of Research and Development, Turkish Naval Forces, Istanbul, Türkiye
Ransomware is a fast-escalating privacy threat causing significant financial and data losses worldwide. As attacks grow more sophisticated, traditional static signatures and dynamic behavior analysis struggle to defend. In this paper, we propose an novel framework that couples software-level scanning with hardware-level microprocessor activity monitoring to achieve efficient ransomware detection and mitigation. The contributions are threefold: (1) we employ adversarial training to harden static analysis against obfuscation and leverage hardware-assisted monitoring for real-time detection; (2) we integrate neural architecture search (NAS) to automatically tailor models for stronger generalizability; and (3) We validate the proposed approach through extensive experiments, demonstrating substantially improved detection accuracy and reduced latency compared with existing methods. Moreover, it provides trace-level interpretability that explains the underlying decision process.
Salma Aboelmagd, Md. Rakibul Ahasan, Zhixin Pan +2
Florida State University, Tallahassee, FL, USA · Stony Brook University, Stony Brook, NY, USA