Reinforcement Learning with Verifiable Rewards (RLVR) is an emerging paradigm that significantly boosts a Large Language Model's (LLM's) reasoning abilities on complex logical tasks, such as mathematics and programming. However, we identify, for the first time, a latent vulnerability to backdoor attacks within the RLVR framework. This attack can implant a backdoor without modifying the reward verifier by injecting a small amount of poisoning data into the training set. Specifically, we propose a novel trigger mechanism designated as the \ourapproach (ACB). The attack exploits the RLVR training loop by assigning substantial positive rewards for harmful responses and negative rewards for refusals. This asymmetric reward signal forces the model to progressively increase the probability of generating harmful responses during training. Our findings demonstrate that the RLVR backdoor attack is characterized by both high efficiency and strong generalization capabilities. Utilizing less than 2% poisoned data in train set, the backdoor can be successfully implanted across various model scales without degrading performance on benign tasks. Evaluations across multiple jailbreak benchmarks indicate that activating the trigger degrades safety performance by an average of 73%. Furthermore, the attack generalizes effectively to a wide range of jailbreak methods and unsafe behaviors. Code is available at https://github.com/yuki-younai/Backdoor_in_RLVR.
Figures & tables
Figure 1: (a) Previous Approach: by poisoning the SFT data or tampering with the reward model.. (b) Our Approach: only need to use the poisoning RL training data based on the task.
Figure 2: The overview of Backdoor-RLVR framework. (a) Asymmetric Chain Backdoor Construction: We constructed a set of backdoor data for RLVR based on the shadow model. (b) Inserting Backdoors in RLVR Training: We mixed benign data with backdoor data to implant the backdoor during the training process.
Figure 3: Distribution and evolution of Attack Success Rate (ASR) with increasing sampling numbers.
Model
Task
Math Compute
Science Reason
Code Generation
Metrics
ASR
CA
PDR
ASR
CA
PDR
ASR
CA
PDR
Baseline
12.1
33.6
58.9
12.1
33.6
34.7
12.1
33.6
37.0
Qwen2.5-3B-Instruct
OnlyDb
73.5
33.2
58.2
70.2
33.0
34.1
72.8
33.8
36.4
OnlyDc
13.5
32.5
62.1
13.2
32.8
37.9
13.0
33.1
38.8
Dmix
74.4
32.1
63.5
71.6
32.6
38.0
74.9
32.9
39.2
Baseline
13.9
35.0
61.1
13.9
35.0
36.2
13.9
35.0
39.9
Table 1: Backdoor Attack Evaluation on different tasks. Only Db means indicates training only using Db data, similarly for Dc . Dmix represents the mixture of the Db and Dc datasets. The bold values indicate the best score.
Figure 4: Different Curves during the Training Process of the Backdoor Model.
Benchmarks
JailbreakBench
HarmBench
StrongReject
Metrics
ASR-J
ASR-H
ASR-S
ASR-J
ASR-H
ASR-S
ASR-J
ASR-H
ASR-S
Direct
2.00
13.0
6.4
23.0
34.2
27.7
16.9
11.5
9.2
τtrigger⊕ Direct
18.0
10.0
8.8
31.0
41.7
13.7
21.4
13.7
8.8
PAIR ( Chao et al., 2023 )
49.0
52.0
33.6
57.0
71.0
48.6
61.9
59.0
49.7
TAP ( Mehrotra et al., 2024 )
54.0
59.2
38.7
59.5
75.6
49.5
66.5
62.6
51.4
PAP ( Zeng et al., 2024 )
76.0
71.0
53.1
62.0
82.0
51.6
84.2
77.0
64.8
Table 2: Jailbreak evaluation of Qwen2.5-7B-Instruct and Backdoor Model (trained on Math Task) on multiple jailbreak benchmarks. The gray section represents evaluation results on the backdoor model. ASR-J (%) denotes the evaluator-based attack success rate, ASR-H (%) denotes the classify-based attack success rate, and ASR-S represents the harmfulness score assessed by StrongRejcet Judge.
Figure 5: (a) Heatmap of the performance of the backdoor for mixed data. (b) Evaluation of Harmful Behaviors Out of Domain.
Figure 6: Comparison of ASR and Model Performance under Different Backdoor Training Methods.
Figure 7: Distribution of the harmful behaviors of the model before and after the backdoor attack.
Method
Backdoor Performance.
JailbreakBench
HarmBench
StrongReject
ACB (Ours)
59.0
64.5
72.0
w/o Dual-Verify
9.3_{{\color[rgb]{1,0,0}(-49.7)}}
32.8_{{\color[rgb]{1,0,0}(-31.7)}}
25.8_{{\color[rgb]{1,0,0}(-46.2)}}
w/o Top-Selection
42.6_{{\color[rgb]{1,0,0}(-16.4)}}
47.8_{{\color[rgb]{1,0,0}(-16.7)}}
53.5_{{\color[rgb]{1,0,0}(-18.5)}}
Baseline
2.0
23.0
16.9
Table 3: Ablation on Backdoor Data Constructed.
Method
Attack & Utility
ASR under Defenses
ID
OOD
CA
Perf
RPO
S-Rem
ClnGen
CROW
Origin
13.9
21.7
33.6
45.6
13.2
12.8
13.5
13.0
SFT 50
67.4
27.4
29.3
44.5
62.0
53.5
19.5
16.0
SFT 100
76.6
32.8
27.6
43.2
71.0
61.5
24.0
19.5
SFT 200
78.8
38.9
25.5
42.7
77.5
68.0
28.5
23.5
RLVR 200
68.9
81.9
32.1
45.5
61.5
56.5
57.0
61.0
Table 4: Comprehensive comparison of RLVR vs. SFT backdoors on Qwen2.5-3B (n=200). ID/OOD-ASR: in/out-of-distribution attack success rate. CA: clean accuracy. Avg Perf: average over AMC23, Math500, MMLU, GPQA. Defense columns report ASR (%) under each defense on HarmBench.
Figure 8: Defense methods evaluated across multiple jailbreak benchmarks on the Backdoor model.
Setting
ASR
CA
PDR
CoT Length
Prop.
ASR
Origin
22.5
26.5
58.0
< 500 tok
24%
83.0
ACB Math
84.2
24.6
71.5
500–1500
55%
85.5
ACB Code
79.6
25.1
55.8
> 1500 tok
21%
87.0
Table 5: ACB attack on DeepSeek-R1-Distill-Qwen-1.5B ( left ) and ASR stratified by CoT reasoning length on Math task ( right ).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: The Judge Prompt Template for Harmbench-cls.
Figure 10: The Judge Prompt Template for JailbreakBench.
Figure 11: The Judge Prompt Template for StrongReject.
Figure 12: Example of Jailbreak Attack Success on Qwen2.5-7B-Instruct.
Figure 13: Example of Jailbreak Attack Success on Llama3-8B-Instruct.
Figure 14: Example of Jailbreak Attack Success on Qwen2.5-7B-Instruct.
Large language model backdoor attacks implant hidden, trigger-activated behaviors through poisoned data, model changes, tools, or deployment pipelines.