Organizations: Shanghai Qi Zhi Institute · The Chinese University of Hong Kong, Shenzhen · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Institute of Interdisciplinary Information Sciences, Tsinghua University · East China Normal University · Xiongan AI Institute
Reverse engineering (RE) is central to software security, particularly for cryptographic programs that handle sensitive data and are highly prone to vulnerabilities. It supports critical tasks such as vulnerability discovery and malware analysis. Despite its importance, RE remains labor-intensive and requires substantial expertise, making large language models (LLMs) a potential solution for automating the process. However, their capabilities for RE remain systematically underexplored. To address this gap, we study the cryptographic binary RE capabilities of LLMs and introduce CREBench, a benchmark comprising 432 challenges built from 48 standard cryptographic algorithms, 3 insecure crypto key usage scenarios, and 3 difficulty levels. Each challenge follows a Capture-the-Flag (CTF) RE challenge, requiring the model to analyze the underlying cryptographic logic and recover the correct input. We design an evaluation framework comprising four sub-tasks, from algorithm identification to correct flag recovery. We evaluate eight frontier LLMs on CREBench. GPT-5.4, the best-performing model, achieves 64.03 out of 100 and recovers the flag in 59% of challenges. We also establish a strong human expert baseline of 92.19 points, showing that humans maintain an advantage in cryptographic RE tasks. Our code and dataset are available at https://github.com/wangyu-ovo/CREBench.
Figures & tables
Figure 1: Overview of CREBench, which contains 432 challenges based on 48 standard encryption algorithms, three types of insecure key usage, and three levels of reverse-engineering difficulty. We also design an evaluation framework covering four sub-tasks, enabling LLMs to operate as agents that solve these challenges in a sandboxed environment.
Figure 2: Comparison of LLMs’ performance on CREBench. Pass@3 performance by model and Pass@1 performance by human expert team are shown, with stacked bars showing sub-task scores, ordered from left to right by total score.
Figure 3: A successful case: GPT-5.4 solves the AES-128-CBC challenge in 9 rounds. The difficulty is O0, and the key usage strategy is hardcoded. More details are explained in Appendix E.1.1 .
Difficulty
Match Rate
O0
41.7% (20/48)
O3
41.7% (20/48)
Const-XOR
2.1% (1/48)
Table 1: Signsrch result across three difficulty levels.
Figure 4: Average pass@3 performance across models under different difficulty settings and Phi correlation among four sub-tasks. Performance drops steadily as difficulty increases from O0 to O3 and further to Const-XOR.
Framework
Avg. Score
Flag Recovery Rate
Ours
47.60
39.6%
Codex
62.69
52.1%
D-CIPHER
-
33.3%
Table 2: Comparison of pass@3 performance on Const-XOR, weak PRNG tasks among 3 agent frameworks.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
#
Challenge
Canonical Algorithm
Key/Block/IV Size
1
3-Way
3-WAY-ECB
96/96/--
2
A5-1
A5/1
64/8/32
3
A5-2
A5/2
64/8/32
4
AES-128-CBC
AES-128-CBC
128/128/128
5
ARIA-128-CBC
ARIA-128-CBC
128/128/128
6
Anubis-128-CBC
ANUBIS-128-CBC
128/128/128
Appendix
Table 3: All 48 cryptographic algorithms implemented in CREBench. The Key/Block/IV Size column reports key size, block size, and IV size in bits ( -- denotes not applicable).
Benchmark
Number of RE Challenges
Ours
432
NYU CTF Bench ( Shao et al., 2024 )
51
InterCode ( Yang et al., 2023 )
27
CyBench ( Zhang et al., 2025b )
6
Appendix
Table 4: Comparison of reverse engineering challenge numbers across existing benchmarks.
Figure 5: Pass@3 perfect rate across eight evaluated models on CREBench. A challenge is counted as perfect only if the model obtains the full score of 100/100, i.e., successfully completes all four tasks within three attempts. GPT-5.4 achieves the highest perfect rate at 41.0% (177/432), followed by GPT-5.2 at 30.1% and Claude-Sonnet-4.6 at 28.9%, while the remaining models achieve substantially lower rates.
Figure 6: Comparison between human experts and LLMs on the same 48-challenge subset used for the human baseline. LLM results are reported under pass@3, while human experts are evaluated under pass@1. Stacked bars show the scores of the four sub-tasks, ordered from left to right by total score.
Figure 11
Algorithm
Claude- Sonnet-4.6
Doubao- Seed-1.8
Gemini- 2.5-Pro
GPT- 5.2
GPT- 5.4-mini
GPT- 5.4
MiMo- V2-Pro
O4- mini
Avg.
Top 5 Easiest
RC4
100.00
100.00
97.22
100.00
72.78
100.00
88.89
75.00
91.74
CAMELLIA
100.00
61.11
55.56
94.44
83.89
100.00
64.44
52.78
76.53
TEA
97.22
63.89
91.67
97.22
55.56
100.00
22.22
50.00
72.22
XXTEA
100.00
44.44
72.22
94.44
47.22
94.44
38.89
50.00
67.71
DES
100.00
33.44
51.11
94.33
40.00
94.44
58.22
40.00
63.94
Appendix
Table 6: Top 5 easiest and bottom 5 hardest challenges ranked by average pass@3 total score across runs.
Figure 8: Average pass@3 scores across the three insecure key modes (hardcoded, fragmented, and weak prng) for each model.
GDB Calls
Attempts
% of Attempts
Avg. Score
Perfect Rate
0
2579
26.85%
32.57
13.69%
1
1948
20.28%
20.61
3.54%
2 – 4
2824
29.40%
19.03
2.83%
5 – 7
1375
14.32%
19.03
1.67%
8+
879
9.15%
12.53
0.80%
Total
9605
100.00%
–
–
Appendix
Table 7: Relationship between run_gdb usage and attempt-level benchmark outcomes. Statistics are computed over all 9,605 raw attempts. Perfect rate is the percentage of attempts that obtain the maximum total score (100/100).
Figure 9: Pairwise correlations among the four sub-tasks