BUSTA: Teaching Small Language Models to Reason About Binary Execution

Ritika Poddar; Aidan Fennelly; Deirdre Murphy; Eric Wolos; Corre L. Steele; Michael Doyle

cs.CROct 26, 2026

BUSTA: Teaching Small Language Models to Reason About Binary Execution

Authors: Ritika Poddar, Aidan Fennelly, Deirdre Murphy, Eric Wolos, Corre L. Steele, Michael Doyle

Organizations: The MITRE Corporation

Abstract

Large Language Models (LLM) are increasingly applied to Reverse Engineering (RE) workflows. RE, however, differs from common LLM coding tasks as it requires models to reason about low-level program behavior. Furthermore, the deployment of tooling in resource-constrained sandboxed RE environments limits the ability to use frontier-scale solutions. We examine whether a targeted LLM training approach can transfer distinct RE skills to open models via inverse basic-block execution: given an x86-64 basic block and a desired output register state, the model predicts the input register values that produce it. Fine-tuning Qwen3-1.7B and Nemotron-3-Nano with Group Relative Policy Optimization (GRPO) raises knowable-register prediction accuracy from 73.2% to 85.3% and from 63.3% to 95.2%, respectively. Frontier models still score highest (up to 99.2%), but our fine-tuned Nemotron-3-Nano comes within four points at a fraction of the parameter count. We release both fine-tuned models and the training dataset.

Figures & tables

			Accuracy (%)
Model	Valid (%)	Cov. (%)	Known	$\Delta$	Unk. cls.	Unk. val.	Avg. Tot. Tokens
Open-weight models
qwen3-1.7b	97.2	77.8	73.2	–	59.5	6.3	14716
+ busta	94.1	89.4	85.3	$+12.1$	72.8	10.2	7668
nemotron-3-nano	72.0	68.3	63.3	–	58.4	9.3	27968
+ busta	99.8	99.5	95.2	$+31.9$	91.2	11.8	15794

Table 1: Basic-block input register prediction results. Valid is the fraction of responses that parsed. Cov. is register coverage. The accuracy columns are knowable-register, unknowable-classification, and unknowable-value accuracy. Avg. Tot. Tokens is the average tokens per response.

\Delta

is the change in knowable-register accuracy relative to the corresponding base model.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Metric	Definition
Valid Response Rate	LLM produces valid output
Register Coverage	Number of expected registers predicted
Knowable Register Accuracy	Correct prediction / knowable
Unknowable Classification Accuracy	LLM correctly determines whether a register is unknowable
Unknowable Value Accuracy	Correct prediction / unknowable

Appendix

Table 2: Evaluation metrics and definitions.

LLMs for Cybersecurity

Large language models for cybersecurity support threat analysis, vulnerability research, security operations, and cyber defense.

See all 258 papers

REBENCH: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names (Extended Version)
Apr 30, 2026Jun Yeon Won, Xin Jin, Shiqing Ma +1LLM Evaluation Software Reverse Engineering
Ohio State University Columbus, OH, United States · University of Massachusetts Amherst Amherst, MA, United States
Improving Large Language Models for Code through Runtime Program-State Reasoning
Sep 28, 2026Hongwei Li, Spandan Garg, Yufan HuangAutomated Program Repair Automated Test Generation
University of California, Santa Barbara · Microsoft
Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmark
Sep 23, 2026Hamed Taherkhani, Mohammad Abdollahi, Melika Sepidband +3LLM Evaluation Software Engineering Benchmarks
York University Toronto, Canada · The University of Texas at Dallas Texas, USA

Apr 30, 2026J/K move · Enter open · S save

REBENCH: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names (Extended Version)

Jun Yeon Won, Xin Jin, Shiqing Ma, Zhiqiang Lin

Ohio State University Columbus, OH, United States · University of Massachusetts Amherst Amherst, MA, United States

LLM Evaluation Software Reverse Engineering

Open paper AskPDF arXiv

Large Language Models (LLMs) have achieved remarkable progress in recent years, driving their adoption across a wide range of domains, including computer security. In reverse engineering, LLMs are increasingly applied to critical tasks such as function and variable name recovery and type inference. However, despite the rapid growth of research in this area, progress has been hindered by the absence of a standardized dataset. Existing studies rely on disparate datasets, preprocessing pipelines, and evaluation metrics, making fair comparisons between approaches difficult and obscuring a clear understanding of LLM capabilities in binary analysis. To address these challenges, we present REBench, a comprehensive benchmark dataset for evaluating LLMs on binary reverse engineering tasks. REBench consolidates a superset of existing datasets, comprising hundreds of millions of lines of source code and a diverse collection of binaries spanning multiple architectures and optimization levels. REBench adopts a knowledge-base-driven methodology that stores byte-level stack information to generate ground truth, ensuring that task difficulty is preserved while maintaining universal applicability. This design enables fair evaluation across tasks while avoiding simplifications that could bias results. As a use case, we apply REBench to measure the reverse engineering performance of LLMs and the result demonstrates difficulties in complex tasks.

Abstract

LLMs for Cybersecurity

Explore similar work

REBENCH: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names (Extended Version)

Improving Large Language Models for Code through Runtime Program-State Reasoning

Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmark