RansomTrack: A Hybrid Behavioral Analysis Framework for Ransomware Detection

Busra Caliskan; Ibrahim Gulatas; H. Hakan Kilinc; A. Halim Zaim

cs.CRApr 9, 2026

RansomTrack: A Hybrid Behavioral Analysis Framework for Ransomware Detection

Authors: Busra Caliskan, Ibrahim Gulatas, H. Hakan Kilinc, A. Halim Zaim

Organizations: Department of Computer Engineering, Istanbul Technical University, Istanbul, Türkiye · Department of Research and Development, Orion Innovation, Istanbul, Türkiye · Department of Research and Development, Turkish Naval Forces, Istanbul, Türkiye

Abstract

Ransomware poses a serious and fast-acting threat to critical systems, often encrypting files within seconds of execution. Research indicates that ransomware is the most reported cybercrime in terms of financial damage, highlighting the urgent need for early-stage detection before encryption is complete. In this paper, we present RansomTrack, a hybrid behavioral analysis framework to eliminate the limitations of using static and dynamic detection methods separately. Static features are extracted using the Radare2 sandbox, while dynamic behaviors such as memory protection changes, mutex creation, registry access and network activity are obtained using the Frida toolkit. Our dataset of 165 different ransomware and benign software families is publicly released, offering the highest family-to-sample ratio known in the literature. Experimental evaluation using machine learning models shows that ensemble classifiers such as XGBoost and Soft Voting achieve up to 96% accuracy and a ROC-AUC score of 0.99. Each sample analyzed in 9.1 seconds includes modular behavioral logging, runtime instrumentation, and SHAP-based interpretability to highlight the most influential features. Additionally, RansomTrack framework is able to detect ransomware under 9.2 seconds. Overall, RansomTrack offers a scalable, low-latency, and explainable solution for real-time ransomware detection.

Figures & tables

Aspect	RansomTrack (2025)	RansomFormer (2024) [ 3 ]	RanSMAP (2024) [ 26 ]	Ransomware / Benignware System Calls (2023) [ 14 ]	Dynamic Feature Dataset (2022) [ 23 ]	RanSAP (2021) [ 25 ]	CICAndMal2017 (2017) [ 32 ]
Feature Types	Hybrid (Static + Dynamic)	Hybrid (Static + Dynamic)	Dynamic only: Memory and Storage	Dynamic only: System calls	Dynamic only: Behavioral features	Dynamic only: Storage I/O patterns	Hybrid (APK + Network behavior)
Static Features	Opcodes	Raw PE byte sequences, Imported API functions	Not included	Not included	Not included	Not included	APKs provided (permissions, manifest analyzable)
Static Feature Tools	Radare2	pefile Python module	Not applicable	Not applicable	Not applicable	Not applicable	ApkTool, VirusTotal (implied)
Dynamic Features	API Calls and Page Protection Flags	API call sequences	Memory access patterns, Storage sector ops	System call traces	File ops, API usage, Registry, Services	Disk sector traces, Time-series I/O	Network traffic, Flow stats, Battery/API logs
Dynamic Collection Tools	Frida	Cuckoo Sandbox 3	BitVisor Hypervisor, UDP logging	API Monitor (manual execution)	Cuckoo (JSON), Feature pipeline	Custom hypervisor tracer, Disk I/O monitor	Tcpdump, CICFlowMeter, Appmon, ADB tools
Operating System and File Types	Win 10 Pro 32-Bit, .exe files	Win 7/8/10/11,.exe files	Windows 10,.exe files	Windows 10 32-bit, .exe files	Win XP/7/10, .exe files	Windows 10, .exe files	Android 5.x–6.x, APKs

Table 1: Summary of the Ransomware and Benignware Datasets

Tool	Category	Function
Radare2 [ 45 ] , Ghidra [ 40 ] , Capstone [ 33 ] , PEStudio [ 16 ] , IDA Pro [ 24 ]	Static Analysis	Disassembly, opcode extraction
Frida [ 18 ] , ProcMon [ 37 ] , RegShot [ 46 ] , ProcDot [ 19 ]	Dynamic Analysis	API monitoring, registry tracing
Cuckoo Sandbox [ 13 ] , CAPEv2 [ 43 ] , Volatility [ 50 ] , Noriben [ 6 ] , DumpIt [ 12 ]	Memory/ Sandbox	Process analysis, memory forensics
YARA [ 2 ] , Wireshark [ 52 ] , INetSim [ 27 ]	Support Tools	Rule-based detection, traffic emulation

Table 2: Categorization of Tools Used in Ransomware Analysis

Figure 1: End-to-End Ransomware Detection Pipeline Using Static and Dynamic Features

Figure 2: Overview of the hybrid feature extraction workflow combining static and dynamic analysis stages.

Behavior Category	Instrumented Functions (Subset)	DLL Name	Purpose / Notes
Mutex Relation	CreateMutex, CreateMutexEx (A/W)	kernel32.dll	Detects mutex creation for single-instance enforcement, thread synchronization, or sandbox evasion.
Runtime Memory Permission Changes	Page_ReadOnly, Page_ReadWrite, Page_Execute, Page_Execute_Read	kernel32.dll	Monitors memory access right changes typically seen in unpacking or shellcode injection routines.
Registry Manipulation	RegCreateKey, RegSetValueEx, RegDeleteValue, RegOpenKey (A/W)	advapi32.dll	Tracks suspicious registry operations related to persistence, configuration changes, or evasion.
Network and Shell Activity	WSASend, ShellExecuteExW, GetAddrInfoExW	ws2_32.dll , shell32.dll	Observes network transmissions, DNS resolution, and shell command execution.
Internet Connectivity	InternetOpenUrl, GetAddrInfo, GetAddrInfoEx	wininet.dll , ws2_32.dll	Detects connection attempts to remote servers and possible C2 communications.
File System Operations	CreateFile, WriteFile, MoveFile, CopyFile, DeleteFile	kernel32.dll	Logs critical file actions including creation, overwrite, encryption, and deletion.

Table 3: Monitored Behavioral Categories and Dynamic Features

Figure 4: Opcode distributions in ransomware and benignware samples

Figure 5: Dynamic feature distribution in ransomware and benignware samples

Group Type	Static Feature Examples	Static Feature Description	Dynamic Feature Examples	Dynamic Feature Description
Benignware Specific Groups	bndmov, vmsave, vmxoff, skinit	Virtualization and system state management; commonly found in clean system exits or virtualization-aware software.	adjustwindowrect, drawtexta, fillrect, getcursor, setfocus	Graphical UI rendering and user interaction management; typical of GUI-based legitimate applications.
	vpmacsdql, vpsllvq, vpblendd	Vector arithmetic operations used in multimedia, graphics, and scientific computations.	getdateformatw, gettimeformatw, getlocaleinfoa, gettimezoneinformation	Localization, formatting, and time/date retrieval routines used in user-facing programs.
	bsr, movdqu, pxor, pshufb	Legacy I/O and memory operations for structured buffer handling in user-level applications.	regsetvalueexw, regdeletevaluew, setenvironmentvariablew	Non-destructive registry access or benign configuration settings by installers and applications.
	cvttpd2dq, vcvttsd2si, vcomisd	Data conversion and comparison instructions frequently observed in benign numerical processing.	createfile, mapviewoffile, heapcompact, heapfree	Safe file and memory operations—typical of setup tools, editors, or document-handling utilities.
Ransomware Specific Groups	vpminsd, vpor, vpaddd, sha1msg1, sha256msg2	Cryptographic and vectorized hashing operations typical in ransomware encryption routines.	copyfilew, createfilemappinga, movetoex, ntcreatefile	File locking, replication, and direct mapping—often used in encryption and destructive overwrite.
	movntdq, vmovntdq, pshufb, movhlps	Streaming memory writes used for fast data overwrite and stealth payload delivery.	regdeletekeya, regsetvalueexa, regenumkeyexa, regqueryinfokeya	Registry manipulation patterns used for persistence, disabling recovery, or altering boot behavior.

Table 4: Static and Dynamic Feature Groups via Correlation-Based Feature Clustering

Group Type	Feature Group	Representative Features	Description
Benignware Specific Groups	GUI Rendering and Input Handling	gdipcreatebitmapfromhbitmap, loadbitmapw, dispatchmessagew, getmessagew	Common in interactive GUI applications; reflects standard rendering pipelines.
	Registry and DLL Handling	cryptacquirecontextw, getfileversioninfoa, regqueryinfokeya, loadlibraryexw	Used for querying system version info and handling optional libraries in benign programs.
	Memory Mapping and Process Utilities	createfilemappinga, mapviewoffile, duplicatehandle, getthreaddesktop	Typical of system configuration tools, benign services, or sandbox utilities.
	Numeric and Vector Operations (Static)	cvtdq2pd, vcvtpd2dq, vpmuludq, movhpd	Indicates benign computational routines involving multimedia or scientific tasks.
	String and Localization Utilities	comparestringa, getlocaleinfoa, gettimezoneinformation	Reflects internationalization and regional format handling in user applications.
Ransomware Specific Groups	Cryptographic and Vector Payloads (Static)	vpminsd, vfmsub213sd, sha1msg1, sha256msg2, kandnb	Corresponds to cryptographic operations, likely encryption core logic.

Table 5: Hybrid Feature Clustering

Model	Feature	Acc.	Prec.	Rec.	Spec.	G-Mean	F1	ROC-AUC	Bal. Acc.	Log Loss	Train Time Per Sample (s)	Test Time Per Sample (s)
RF	Static	0.91	0.96	0.85	0.97	0.90	0.90	0.98	0.91	0.21	0.0008	0.0001
RF	Dynamic	0.94	0.91	0.97	0.91	0.94	0.94	0.98	0.94	0.19	0.0012	0.0002
RF	Hybrid	0.95	0.95	0.95	0.95	0.95	0.95	0.99	0.95	0.14	0.0016	0.0002
LR	Static	0.82	0.95	0.68	0.97	0.81	0.79	0.90	0.82	0.56	0.0061	0.000001
LR	Dynamic	0.94	0.93	0.95	0.93	0.94	0.94	0.98	0.94	0.17	0.0062	0.00001
LR	Hybrid	0.93	0.93	0.93	0.93	0.93	0.93	0.98	0.93	0.22	0.0132	0.00001

Table 6: Per-Sample Training and Testing Times for Random Forest and Logistic Regression

Model	Acc.	Pre.	Rec.	Spec.	G-Mean	F1	ROC-AUC	Bal. Acc.	Log Loss	Train Time Per Sample (s)	Test Time Per Sample (s)
RF	0.95	0.95	0.95	0.95	0.95	0.95	0.99	0.95	0.14	0.0016	0.0002
LR	0.93	0.93	0.93	0.93	0.93	0.93	0.98	0.93	0.22	0.0132	0.000001
KNN	0.95	0.97	0.93	0.98	0.95	0.95	0.97	0.95	0.81	0.0217	0.0013
DT	0.94	0.95	0.93	0.95	0.94	0.94	0.96	0.94	1.34	0.0008	0.00001
MLP	0.94	0.92	0.96	0.92	0.94	0.94	0.98	0.94	0.16	0.1251	0.0001
XGBoost	0.96	0.97	0.95	0.98	0.96	0.96	0.99	0.96	0.12	0.0025	0.0001

Table 7: Evaluation Metrics of Classifiers on Hybrid Feature Set with Per-Sample Timing

Figure 7: Top 20 features contributing to class 1 (top) and class 0 (bottom) based on absolute mean SHAP values.

Model	FN / (TP + FN)	FNR (%)
Soft Voting	$10/241$	4.15%
XGBoost	$11/241$	4.56%
Random Forest	$13/241$	5.39%

Table 8: False Negative Rates (FNR) for Ransomware Samples

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Ransomware Family	Count	Ransomware Family	Count	Ransomware Family	Count	Ransomware Family	Count	Ransomware Family	Count	Ransomware Family	Count
Adhubilka	2	AESCRYPT	1	AESRT	1	AgentTesla	2	Alien	1	Avaddon	51
AvosLocker	1	AXLocker	5	Babadeda	1	Babuk	47	Bazek	2	BlackBasta	23
BlackCat	53	Blackmatter	11	BlackOut	3	BlackShades	1	BlackSnake	1	BlackSuit	3
BuerLoader	3	Bulkwar	1	Buran	1	Cerber	37	Chaos	51	Chimera	2
Cipher	1	CoinMiner	1	Conti	54	CryLock	2	CrypHydra	1	Crypt360	1
CryptFile2	1	CryptNet	1	CryptoJoker	1	CryptoLocker	1	CryptoWall	1	Crypute	1

Appendix

Table 9: List of ransomware families and number of samples in the dataset.

Malware Detection

Malware detection uses static and behavioral evidence to identify malicious software and protect devices, networks, and software ecosystems.

See all 57 papers

Hardware-Assisted Multi-Stage Ransomware Detection with Adaptive Machine Learning
Oct 11, 2026Salma Aboelmagd, Md. Rakibul Ahasan, Zhixin Pan +2Malware Detection
Florida State University, Tallahassee, FL, USA · Stony Brook University, Stony Brook, NY, USA
A Hybrid Framework For Crypto-Ransomware Detection In Enterprise Shared Storage
Jun 29, 2026Gervais Hatungimana, Abdun Naser Mahmood, Mohammad Jabed Morshed ChowdhuryNetwork Intrusion Detection Cybersecurity
Department of Computer Science and Information Technology, La Trobe University, Melbourne, VIC, Australia
Agentic SABRE: An Uncertainty-Aware Neuro-Symbolic Multi-Agent Framework for Adaptive Ransomware Detection
Jul 5, 2026Henry Kabuye, Biju Issac, Jeyamohan NeeraMalware Classification Neuro-Symbolic AI
School of Computer Science, Northumbria University, Newcastle Upon Tyne, UK

Oct 11, 2026J/K move · Enter open · S save

Hardware-Assisted Multi-Stage Ransomware Detection with Adaptive Machine Learning

Salma Aboelmagd, Md. Rakibul Ahasan, Zhixin Pan, Abdulrahman Takiddin+1

Florida State University, Tallahassee, FL, USA · Stony Brook University, Stony Brook, NY, USA

Malware Detection

Open paper AskPDF arXiv

Ransomware is a fast-escalating privacy threat causing significant financial and data losses worldwide. As attacks grow more sophisticated, traditional static signatures and dynamic behavior analysis struggle to defend. In this paper, we propose an novel framework that couples software-level scanning with hardware-level microprocessor activity monitoring to achieve efficient ransomware detection and mitigation. The contributions are threefold: (1) we employ adversarial training to harden static analysis against obfuscation and leverage hardware-assisted monitoring for real-time detection; (2) we integrate neural architecture search (NAS) to automatically tailor models for stronger generalizability; and (3) We validate the proposed approach through extensive experiments, demonstrating substantially improved detection accuracy and reduced latency compared with existing methods. Moreover, it provides trace-level interpretability that explains the underlying decision process.

Abstract

Malware Detection

Explore similar work

Hardware-Assisted Multi-Stage Ransomware Detection with Adaptive Machine Learning

A Hybrid Framework For Crypto-Ransomware Detection In Enterprise Shared Storage

Agentic SABRE: An Uncertainty-Aware Neuro-Symbolic Multi-Agent Framework for Adaptive Ransomware Detection