Best-of-N selection for code generation is only as good as the judge that picks among the candidates, and the judges accurate enough for this role are costly frontier LLMs. A small judge can be trained on abundant earlier data labeled by unit tests, but it is deployed on newer problems: the earlier problems are easier and teach it little, their tasks differ, and their candidates come from weaker policies, while data close to deployment are scarce. We present JURIST, a 1.7B reasoning judge that uses a small meta set close to deployment to learn how much each earlier training sample should count. In our experiments, we find that training on the meta set alone, or adding it to the training data, does not help, while using it to weight the earlier data through a bi-level objective adds 3.9 Hard points over adding it to the training data. On LiveCodeBench, JURIST reaches 84.7% Overall and matches Claude Sonnet 4.6 on Hard problems (73.8%) at 463x lower cost; on BigCodeBench, it surpasses all test-time scaling baselines (35.9%). Abundant earlier data, guided by a small deployment-aligned meta set, can thus train a compact judge that selects accurately at low cost.
Figures & tables
Figure 1: Cost-accuracy comparison among LLM judges on LiveCodeBench Hard problems. The x-axis is logarithmic; marker area grows with model size.
Figure 2: Bi-level data-reweighted training framework.
Figure 3: How JURIST is trained and how it selects a candidate at inference (a), with its prompt and output (b).
(i) Explains
(ii) Execution
(iii) Shift
(iv) Cost
Method
Reasoning
Compares
Exec. labels
Outcome RL
Domain
Instance
Size
Generative judges
Auto-J ( Li et al., 2024 )
✓
✓
✗
✗
✗
✗
13B
Prometheus 2 ( Kim et al., 2024 )
✓
✓
✗
✗
✗
✗
7B, 8 × 7B
Self-Taught Eval. ( Wang et al., 2024b )
✓
✓
✓
✗
✗
✗
70B
GenRM ( Zhang et al., 2025a )
✓
✗
✓
✗
✗
✗
2B–9B
Table 1: Prior work on code Best-of-N selection against the four requirements of Appendix A . ✓ : holds in part.
Method
Granularity
Params
Weight formula
Domain reweighting
domain
K
αk assigned to domain Dk
Instance table
sample
N
αi stored per (Xi,Yi)
Instance net
sample
∣θ∣
αi=MLPθ(L(Xi,Yi;ϕ))
Table 2: Data-reweighting strategies.
Objective
Margin mi
Learning signal ψi
DPO
log-ratio, chosen minus rejected
1−qi
KTO
implicit reward minus reference
λqi(1−qi)
ORPO
log odds ratio
λO(1−qi)
GRPO
logpi/(1−pi)
pi(1−pi)
Table 3: Margin mi and learning signal ψi of each objective; qi=σ(mi) , pi is the probability that the judge selects a passing candidate on sample i ( Appendix G ); λ and λO are the KTO class weight and the ORPO odds-ratio weight, respectively.
Figure 7
Figure 5: Overall pass@1 (%) on (a) LiveCodeBench and (b) BigCodeBench: JURIST against zero-shot models that solve each problem directly (grey shading) and test-time scaling baselines; (H), (M): high, medium reasoning effort. (c) JURIST against each of the four DeepSeek V3.2 Speciale candidates and an oracle judge on LiveCodeBench. The y-axes do not start at zero.
Method
o4-mini
V3.2
Qwen2.5
Qwen3-30B
Avg.
Skywork-o1 PRM 7B
74.1
68.7
25.2
36.6
51.2
DeepSeek GRM 16B
75.6
83.2
25.2
38.2
55.6
JURIST 1.7B (ours)
81.7
84.7
26.7
38.9
58.0
Table 5: Overall accuracy across policy models on LiveCodeBench. o4-mini: o4-mini (high); V3.2: DeepSeek V3.2 Speciale; Qwen2.5: Qwen2.5-Coder-32B; Qwen3-30B: Qwen3-Coder-30B-A3B.
Method
DPO
KTO
ORPO
Average
Overall
No reweighting
81.7
83.2
81.7
82.2
Domain reweighting
83.2
83.2
82.4
82.9
Instance table
82.4
84.0
83.2
83.2
Instance net
83.2
84.0
83.2
83.5
Hard
Table 7: Reweighting under three preference-optimization objectives on LiveCodeBench; all difficulty levels in Table 15 .
Method
Overall
p
Hard
p
Reweighting strategy
No reweighting
83.36 ± 1.29
.027
69.02 ± 2.25
.008
Domain reweighting
84.73 ± 0.80
–
71.64 ± 1.11
–
Single-level alternatives
Meta set only
82.14 ± 1.73
.001
66.72 ± 3.00
.002
Pooled training and meta sets
82.82 ± 0.83
.001
67.70 ± 1.90
.002
Table 8: GRPO ablation on LiveCodeBench, mean ± standard deviation over ten inference seeds; p : paired t -test against domain reweighting over the seeds.
Figure 6: Training time on 2 × A100; the upper level adds 20.36% wall-clock.
Judge
Tokens per judgment
Minutes
Overall
Hard
Untrained Qwen3-1.7B
5.7k (5.3k–6.4k)
42
79.85
61.97
RM-R1 7B
4.0k (3.1k–4.4k)
18
82.52
68.31
JURIST 1.7B (ours)
1.2k (1.2k–1.3k)
7
84.43
71.48
Table 10: Inference cost on LiveCodeBench. Tokens per judgment: mean (range) over four policy models; Minutes (one evaluation pass), Overall, and Hard: DeepSeek V3.2 Speciale candidates. All entries are means over five inference seeds.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Judge
Protocol
Overall
Hard
Cost (USD)
vs. JURIST
Random
Uniform selection
76.0
54.9
$0.00
–
Claude Haiku 4.5
Listwise
83.2
70.5
$10.97
48 ×
Claude Sonnet 4.6
Listwise
84.7
72.1
$31.19
136 ×
Claude Haiku 4.5
Pairwise ( R=8 )
85.5
72.1
$62.85
273 ×
Claude Sonnet 4.6
Pairwise ( R=8 )
86.3
73.8
$106.48
463 ×
JURIST 1.7B (ours)
Pairwise ( R=8 )
84.7
73.8
$0.23
1 ×
Appendix
Table 12: Cost and accuracy of frontier LLM judges on LiveCodeBench. Last column: cost relative to JURIST .
LiveCodeBench
BigCodeBench
Method
Overall
Easy
Medium
Hard
Method
Overall
Zero-shot Methods
o4-mini (high)
77.1
100.0
89.7
57.4
o3-mini (medium)
33.1
Gemini-2.5-Pro-06-05
72.5
100.0
82.1
52.5
o1 (high)
32.4
o4-mini (medium)
71.8
96.8
79.5
54.1
o3-mini (high)
32.4
o3 (high)
71.8
100.0
71.8
57.4
Claude-3.7-Sonnet
32.4
Appendix
Table 13: Full per-method, per-difficulty performance on LiveCodeBench and BigCodeBench.
Method
Overall
Easy
Medium
Hard
o4-mini (high)
Random
75.8
98.4
80.8
61.1
Skywork-o1 PRM 7B
74.1
96.8
82.1
57.4
DeepSeek GRM 16B
75.6
100.0
79.5
60.7
JURIST 1.7B
81.7
100.0
89.7
67.2
DeepSeek V3.2 Speciale
Appendix
Table 14: Results on LiveCodeBench by policy model.
Method
Overall
Easy
Medium
Hard
DPO
No reweighting
81.7
100.0
92.3
65.6
Domain reweighting
83.2
100.0
94.9
67.2
Instance table
82.4
100.0
94.9
65.6
Instance net
83.2
100.0
92.3
68.9
KTO
Appendix
Table 15: Full per-difficulty results of the preference-optimization ablation on LiveCodeBench.
Method
Overall
Hard
Untrained Qwen3-1.7B
79.85 ± 1.28
61.97 ± 2.13
SFT warmup
82.44 ± 1.53
67.21 ± 2.59
No reweighting
83.21 ± 1.08
68.52 ± 1.80
Domain reweighting
84.43 ± 0.87
71.48 ± 1.47
Instance table
83.97 ± 0.54
69.51 ± 1.47
Instance net
84.27 ± 0.68
69.83 ± 1.47
Appendix
Table 16: GRPO judges on LiveCodeBench over five inference seeds (mean ± standard deviation): the initialization and the instance-level reweighting strategies, with no reweighting and domain reweighting.
Figure 7: Public leaderboard results on LiveCodeBench and BigCodeBench.
Figure 8: Voting-round scaling on LiveCodeBench, averaged over four policy models.
Figure 9: Learned domain weights during training on (a) LiveCodeBench and (b) BigCodeBench.
Figure 11: Learning signal ψi of each objective ( Table 3 ), normalized to its maximum, from samples the judge solves ( q=1 ) to samples it gets completely wrong ( q=0 ). For DPO, KTO and ORPO, q=σ(mi) ; for GRPO, q=pi in the large-group limit.
Shift
Training
Meta
Test
Hard problems
23.0%
27.9%
46.6%
AtCoder-hard
9.3%
21.3%
34.4%
Trajectory source
Qwen2.5/3-Coder
o4-mini, DeepSeek
DeepSeek
Appendix
Table 17: Distribution shifts on LiveCodeBench. Test set: 31 easy, 39 medium and 61 hard problems.
Figure 12: Share of training samples at each value of q , the share of 8 judgments that are correct, placed on the learning-signal curve ψ=q(1−q) of GRPO, for the 1.7B and 8B base judges. Bubble area and number: percentage of the samples of that difficulty. Bold: samples at q=1 and q=0 , which carry no signal.
Figure 13: Share of training samples with no learning signal ( q=1 or 0 : all 8 judgments correct or all wrong) for Qwen3 base judges of three sizes before training.
Judge
Difficulty / domain
p
ψ
By difficulty
1.7B base
Easy
0.663
0.257
Medium
0.505
0.326
Hard
0.497
0.284
8B base
Easy
0.849
0.118
Medium
0.668
0.238
Appendix
Table 18: Mean probability p that the judge selects a passing candidate over 8 judgments, and mean learning signal ψ=p(1−p) , on the training samples, for the 1.7B and 8B base judges before training and for the 1.7B judge after SFT warmup, where GRPO starts (120 samples per difficulty). Bold: largest ψ per judge.
Figure 14: Training samples of each difficulty on the update map of Figure 4 , for the judge midway through training with domain reweighting. Each point is a sample with ψi>0 , jittered vertically. Top: share of samples whose weight falls or rises; bottom: share with ψi=0 . The three shares of a panel sum to 100%.
Figure 15: Alignment of training samples with meta samples of each group, after the common direction of the meta gradient is removed, double-centered ( ×10−3 , P=I ). (a) Platform and (b) candidate source, for the judge midway through training with domain reweighting. Diagonal: same-group excess with a 95% interval from resampling problems; p : permutation test of the diagonal mean.
Figure 29
Figure 16: (a) Learning signal ψ , (b) alignment and (c) their product by difficulty over training, for a run of domain reweighting whose meta set holds only hard problems ( P=I ). The alignment of a difficulty is the ψ -weighted mean alignment of its samples, and the product is the mean of ψi times the alignment.
Large language model (LLM)-as-a-judge research uses language models to score, compare, and verify AI outputs as scalable alternatives to human evaluation.
Large Language Models achieve strong code generation for high resource languages like Python and Java but suffer sharp performance drops on Low-Resource Programming Languages~(LRPLs) such as Julia. Improving Small Language Models~(SLMs) for these languages faces a trilemma: Supervised Fine-Tuning~(SFT) is bottlenecked by data scarcity, inference-time scaling is too expensive for deployment, and Reinforcement Learning from scratch yields near zero advantages. We propose a three-phase pipeline that resolves this trilemma by decoupling syntax acquisition from algorithmic reasoning. First, we left-shift inference-time compute to an offline data synthesis engine that uses iterative compiler and test feedback to generate verified training examples. Second, we fine-tune an SLM on this synthetic, verified data to embed strong syntactic priors. Third, we apply Reinforcement Learning with Verifiable Reward~(RLVR) grounded by language-agnostic Input/Output tests, where the SFT prior constrains exploration away from syntax errors. Applied to Qwen3-8B, our pipeline improves pass@1 by up to +7.6 points on MultiPL-E and +14.2 points on the Agnostics LiveCodeBench for Julia compared to SOTA results. Furthermore, the pipeline only used 31 data and 61 cost over the previous state-of-the-art. We further demonstrate that the pipeline generalizes to Ballerina achieving 49.7% MultiPL-E Pass@1, a language with near-zero pretraining representation. Ablations confirm that both the SFT phase and execution-grounded rewards are necessary for stable training.
Language models are often used to judge generated code without running it. A useful evaluation should reveal substantial changes in the answers it scores. We examine this requirement by repeating an earlier evaluation on 421 coding problems, comparing Qwen-2.5-7B-Instruct with two versions trained using Group Relative Policy Optimization (GRPO). One receives rewards from execution tests (CE-trained), and the other from a strong language-model judge (SLJ-trained). The base model's syntax-failure rate remains stable, but the trained checkpoints produce invalid Python on 63%-67% of attempts. Despite this change, the strong judge's Pass@32 scores fall by only about 3 percentage points, from 96.67% to 93.95% for SLJ-trained. Within the later run, allowing more attempts makes the base and trained checkpoints look increasingly similar under the judge. With 128 attempts per question, the judge accepts at least one SLJ-trained answer on 96.44% of problems, compared with 28.27% under execution tests. We recommend reporting the share of answers that pass Python's syntax check alongside every judge-based score. This check requires no tests or code execution and flags invalid answers that the judge accepts, although it cannot establish functional correctness. Our findings show how a high judge-based score can hide substantial output damage and motivate checking an evaluation's sensitivity to change. Project page and code: https://curiousorbitnotes.github.io/evorobust-replay-audit/
Code generation is typically trained in the primal space of programs: a model produces a candidate solution and receives sparse execution feedback, often a single pass/fail bit. Test-time scaling enriches the inference procedure by sampling multiple candidates and judging among them, but the comparative information this process reveals is discarded after inference. We argue that this information defines a dual judgment space that provides a far richer training signal: the model learns not from an isolated success or failure, but from the relative correctness structure across its own plausible attempts, identifying which succeed, which fail, and what distinguishes them. We introduce DuST (Dual Self-Training), a framework for self-training from the dual judgment space. DuST samples candidate programs from the model's own distribution, labels them through sandbox execution, retains groups containing both successes and failures, and trains the model to rank candidates by execution correctness using GRPO. The objective is purely discriminative: the model is never directly rewarded for generating correct programs. Dual self-training improves both judgment and generation. Across five models spanning two families and three scales (4B to 30B), DuST consistently improves Best-of-4 test-time scaling on LiveCodeBench. For Qwen3-30B-Thinking on LiveCodeBench v6, judgment quality improves by +6.2 NDCG, single-sample pass@1 improves by +3.1, and Best-of-4 accuracy improves by +4.1. The trained model's single rollout matches the base model's Best-of-4 performance. SFT on the same ranking data improves judgment without improving generation, confirming that on-policy RL is the mechanism that transfers dual-space learning back into primal generation.