Reinforcement learning with verifiable rewards (RLVR) provides reliable supervision for software engineering agents through executable tests, but binary rewards fail to distinguish trajectories with substantially different debugging quality. We investigate whether the current policy can judge its own trajectories and use these judgments to improve task performance without a separately trained reward model. To this end, we introduce Behavioral Self-Rewarding (BSR), a reward-refinement method that reuses the policy to compare observable debugging behavior among trajectories from the same task with the same verifier outcome. BSR aggregates pairwise preferences into task-local scores and converts them into bounded auxiliary rewards, introducing reward contrast while preserving the verifier-defined pass/fail ordering. This provides additional supervision from already-collected trajectories without requiring new environment interactions. Experiments on SWE-bench Verified and SWE-bench-Live show consistent improvements across the evaluated Qwen3-14B and Qwen3-32B configurations, supporting using the policy's own comparative judgments as a complementary learning signal for training software engineering agents.
Figures & tables
Figure 1: Overview of Behavioral Self-Rewarding (BSR). Binary rewards provide no contrast when trajectories share the same verifier outcome, while BSR ranks them within the outcome group to recover a bounded learning signal.
SWE-bench Verified
SWE-bench-Live Lite
Size
Budget
Method
Avg. Acc. ± Std.
Pass@3
Avg. Acc. ± Std.
Pass@3
14B
270
Baseline
31.0±0.5
45.0
7.11±0.19
13.33
BSR (Ours)
33.2±1.4
46.6
8.67±0.88
15.33
400
Baseline
32.1±1.1
46.6
6.33±0.34
11.33
BSR (Ours)
34.7±1.0
48.0
8.11±1.02
13.67
32B
270
Baseline
34.4±1.7
48.8
7.67±0.88
12.33
Table 1: Performance (%) on SWE-bench Verified and SWE-bench-Live Lite. Standard deviations are across three evaluation runs.
Figure 2: SWE-bench Verified evaluation accuracy over training for Qwen3-14B and Qwen3-32B. Each point averages three evaluation runs.
Statistic
Baseline
BSR (Ours)
Process metrics
Related file viewed before first edit ↑
0.733
0.748
Search-to-edit conversion ↑
0.465
0.494
Post-edit test rate ↑
0.301
0.311
Duplicate command rate ↓
0.209
0.208
Selected workload statistics
Table 2: Trajectory behavior statistics over the full set of trajectories. Arrows indicate the preferred direction for metrics where applicable. Details are in Appendix B.4 .
Reward mapping
Avg.
Pass@3
No deadzone, no softness
32.0
45.6
Deadzone, no softness
32.6
44.0
Deadzone + softness (BSR)
33.2
46.6
Table 3: Reward-mapping ablation on 14B.
Table 6
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Bucket size n
Full comparisons (2n)
Scheduled comparisons
Reduction
1
0
0
–
2
1
1
0.0%
3
3
3
0.0%
4
6
6
0.0%
5
10
8
20.0%
6
15
15
0.0%
Appendix
Table 6: Comparison reduction from the scheduling scheme. For n<5 , the scheduler uses exhaustive pairwise comparison. For n≥5 , it uses T=min(⌈log2n⌉+2,n−1) Swiss-style rounds, with at most ⌊n/2⌋ comparisons per round.
Parameter
Value
Model Configuration
Max Prompt Length
4096
Max Response Length
32768
Training Settings
Train Dataset
R2E-Gym Subset
Train Batch Size
64
Appendix
Table 7: Important hyperparameters used in the main experiments.
Size
Budget
Method
Avg. Acc.
Pass@3
14B
270
Baseline
7.12
14.61
BSR (ours)
7.87
15.73
400
Baseline
6.74
14.61
BSR (ours)
8.24
15.73
32B
270
Baseline
4.87
8.99
BSR (ours)
5.62
13.48
Appendix
Table 8: Terminal-Bench 2.1 performance (%) over 89 tasks and three evaluation trials.
Figure 3: Mean critic scores on training set for BSR and the baseline. Curves are smoothed with EMA with a span of 31 (Note here the critic score is just the score from the execution verifier).
Figure 4: Evaluation on SWE-bench Verified while training Qwen3-14B for long run of 400 steps. We evaluate with temperature = 1 and max_steps = 50 for 3 runs.
Figure 5: Detailed plot of evaluation on SWE-bench Verified while training Qwen3-32B. We evaluate with temperature = 1 and max_steps = 50 for 3 runs.
Figure 6: Verifier-outcome-bucket ablation on SWE-bench Verified while training Qwen3-14B. We compare applying BSR to failed trajectories only, successful trajectories only, and both successful and failed trajectories.
Figure 7: Reward-mapping ablation on SWE-Bench Verified while training Qwen3-14B. We compare the full BSR reward mapping against variants without the deadzone and/or softness.
Statistic
Baseline
BSR (Ours)
Process metrics
Reference file viewed before first edit ↑
0.733
0.748
Search-to-edit conversion ↑
0.465
0.494
Post-edit test rate ↑
0.301
0.311
Duplicate command rate ↓
0.209
0.208
Basic workload statistics
Appendix
Table 9: Appendix trajectory behavior statistics over the full set of trajectories. Arrows indicate the preferred direction for process metrics. Basic workload statistics are descriptive and should be interpreted in context.
Figure 8: Agreement with Kimi-2.5 on sampled pairwise trajectory comparisons over training, measured by macro F1. At each evaluation point, 200 on-policy comparison pairs are sampled and re-evaluated using Kimi-2.5 as an external reference judge.
Figure 9: System Prompt for Success-Bucket Pairwise Comparison.
Figure 10: System Prompt for Failure-Bucket Pairwise Comparison.
Figure 11: User Prompt for ranking.
Figure 12: Example Ranking Response from Qwen3-14B.