cs.CRNov 5, 2026

AutoPYara: Next-Gen YARA Rule Generator for Malware Family Clustering

Authors: Mabon Ninan, Nhat Nguyen, Soumyajyoti Dutta, Sidharth Anil, Marcus Botacin

Organizations: Texas A&M University

Abstract

Millions of malware variants threaten users daily, such that clustering these samples is key for fast incident response. AutoYara is the current state-of-the-art for automatic YARA rule generation, but the task remains challenging on realistic datasets, where intra-family diversity and noisy clustering can degrade rule quality. We present AutoPYara, a solution armored with: (1) precise cluster centroid selection based on similarity hashing, and (2) an improved heuristic for estimating subclusters within a malware family using historical VirusTotal data. We implement AutoPYara as an open-source Python package (pip install autopyara) to support reproducibility and practical use. We show that AutoPYara, (1) increases rule coverage by 14 percentage points; (2) reduces variance by 7.1 percentage points, comparing the average variance AutoPYara exhibits to that of AutoYara; and (3) improves threat-hunting accuracy by 10%. AutoPYara produces robust and effective YARA rules than prior automatic approaches in practical settings.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Malware Clustering in the Wild: Taxonomy, Challenges, and Evaluation Metrics

    Nov 2, 2026Raguvir S, Bojing Li, Prajna Bhandary +1ClusteringMalware Analysis

  2. Self-Supervised Representations for Binary Program Clustering: From Empirical Study to Retrieval-Augmented Learning

    Aug 3, 2026Martin Mocko, Daniela ChudáRepresentation LearningSelf-Supervised Learning

  3. Detecting Concept Drift in Evolving Malware Families Using Rule-Based Classifier Representations

    Apr 24, 2026Tomáš Kalný, Martin Jureček, Mark StampConcept DriftRule-Based Classification