Rubric optimization improves LLM grading by revising scoring instructions while keeping the grader's parameters fixed. Existing optimizers often judge edits by aggregate validation performance, limiting the supervision they retain about which errors an edit corrects or introduces. Testing each candidate also requires rescoring validation answers, which limits how many edits can be explored within a fixed budget. We propose RubricWorld, a framework that learns from previous edit evaluations to guide further search. Its world model predicts an edit's correction and degradation rates at score boundaries, conditioned on the current rubric state. The training signal comes from paired grading results: scores for the same answers before and after an edit. Predictions select one candidate for direct evaluation, and held-out confidence bounds determine whether it corrects enough target errors while keeping new errors within prespecified limits. Measured effects update the model even when the edit is rejected. Across EIR (National), ASAP2.0, and ASAP-SAS with three frozen LLM graders, RubricWorld achieves the highest score in 20 of 27 metric comparisons. On ASAP2.0, it raises gpt-5-mini accuracy from 18.20% with the original rubric to 46.70%, exceeding the strongest baseline, APO, by 22.60 percentage points. code will be released upon acceptance.
Figures & tables
Figure 1: Overview of RubricWorld . (a) The current rubric and observed grading errors guide the generation of edits with a limited scope. (b) The world model predicts corrections and degradations for each candidate and guides the choice of an edit for evaluation. Paired grading decisions before and after the edit provide learning feedback, including for edits that are later rejected. (c) Confidence bounds on held-out answers determine whether the edit is accepted. The LLM grader’s parameters remain fixed throughout.
Method
EIR (National)
ASAP2.0
ASAP-SAS
ACC
QWK
F1
ACC
QWK
F1
ACC
QWK
F1
Meta-Llama-3.1-8B-Instruct-AWQ-INT4
Zero-shot ( Brown et al., 2020 )
55.42 ± 0.54
54.10 ± 0.46
54.85 ± 1.92
12.50 ± 2.15
5.75 ± 2.13
4.55 ± 0.70
1.99 ± 0.64
1.44 ± 0.87
1.20 ± 2.08
Few-shot ( Brown et al., 2020 )
57.12 ± 2.00
52.80 ± 0.56
55.15 ± 0.20
9.40 ± 0.54
4.78 ± 0.79
4.74 ± 1.45
19.92 ± 1.39
0.01 ± 2.25
7.75 ± 2.01
APO ( Pryzant et al., 2023 )
61.35 ± 1.17
60.15 ± 1.97
59.90 ± 1.17
15.45 ± 1.68
8.92 ± 1.27
7.98 ± 0.79
21.51 ± 0.75
8.90 ± 1.56
14.50 ± 0.61
GradeOpt ( Chu et al., 2025 )
54.20 ± 0.83
56.45 ± 0.18
53.10 ± 1.99
10.48 ± 0.68
7.37 ± 2.11
5.44 ± 1.53
33.07 ± 1.76
9.15 ± 1.53
21.09 ± 0.43
Table 1: Final grading scores across three benchmarks and frozen LLM graders. Three-seed mean ± SD; ACC/F1 in %, QWK on a 0–100 scale. Blue/red denotes changes from zero-shot; bold marks the best displayed score.
Variant
QWK@5 ↑
ACC@5 ↑
Tokens (M) ↓
5 rounds
20 rounds
Full RubricWorld
38.43
40.18
6.84
8.32
w/o State
33.16
35.42
6.41
8.68
w/o Learned Model
24.85
27.31
6.72
8.15
Table 2: Edit-effect model ablation on ASAP2.0. Average across three frozen graders with shared generator and certification.
Figure 2: Prediction, selection, and certification on ASAP2.0 ( gpt-5-mini ). (a) Held-out-state MAE (pp); (b) selected-edit correction and degradation (%); (c) test QWK (0–100) and degradation (%). Three-seed mean ± SD; variants in Appendix N .
Figure 3: Quality–cost trade-off on ASAP2.0 ( gpt-5-mini ). Three-seed means including warm-start tokens; outlined points are Pareto-optimal.
Figure 4: Illustrative rubric-editing trajectories (ASAP2.0 scale; gpt-5-mini setting). The same edit C is applied after A or B.
Figure 7
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Item
n
Scale
Train
Val/Test
U41Q1
199
0–3
159
20 / 20
U41Q2
230
0–4
184
23 / 23
U42Q1
248
0–4
198
25 / 25
U42Q2
262
0–3
209
26 / 27
U43Q1
434
0–4
347
43 / 44
Total
1,373
–
1,097
137 / 139
Appendix
Table 3: EIR (National) aggregate split statistics. Items use fixed 80/10/10 partitions, stratified by reference score where class counts permit.
Split
#Samples
Score distribution
Train
14,711
1058 / 4023 / 5326 / 3350 / 827 / 127
Test
2,596
194 / 700 / 954 / 576 / 143 / 29
Appendix
Table 4: ASAP2.0 split statistics. Score distributions are ordered from score 1 to score 6.
Data role
Answers
Ddyn,train
60 / 200 / 400
Ddyn,val
150
Dcert
1,500
Dtest
2,596
Appendix
Table 5: ASAP2.0 data allocation for edit-effect collection. The three training counts are alternative collection budgets; the certification count is the total pool size, not a per-round eligible sample count.
Split
#Samples
Score distribution
Train
1,170
266 / 301 / 366 / 237
Valid
252
Stratified from training data
Test
251
57 / 64 / 79 / 51
Appendix
Table 6: ASAP-SAS Set 1 split statistics. Score distributions are ordered from score 0 to score 3.
Method
Text supplied or changed
Selection basis
Zero-shot
Original rubric
No update
Few-shot
Scored examples appended to prompt
Validation agreement
APO
Rewritten scoring instructions
Aggregate validation score
GradeOpt
Guidelines and examples derived from errors
Validation improvement
GEPA
Prompt variants generated from reflection
Candidate-pool selection
RubricWorld
Edit to a chosen score distinction
Repair and per-boundary breakage
Appendix
Table 7: What each comparison method changes and evaluates.
Parameter
Value
Optimization rounds
5
Main-table repetitions
3 seeds; mean ± std
Long-horizon analysis rounds
100
Candidate patches per round
5
Validation sampling
By reference score
Data used to find errors
Training partition
Appendix
Table 8: Reported optimization settings for RubricWorld .