cs.AINov 1, 2026

RubricWorld: World Model-Guided Rubric Optimization for LLM Graders

Authors: Hongji Pu, Hang Li, Yucheng Chu, Jiliang Tang

Organizations: University of Illinois Urbana-Champaign, Urbana, IL, USA · Michigan State University, East Lansing, MI, USA

Abstract

Rubric optimization improves LLM grading by revising scoring instructions while keeping the grader's parameters fixed. Existing optimizers often judge edits by aggregate validation performance, limiting the supervision they retain about which errors an edit corrects or introduces. Testing each candidate also requires rescoring validation answers, which limits how many edits can be explored within a fixed budget. We propose RubricWorld, a framework that learns from previous edit evaluations to guide further search. Its world model predicts an edit's correction and degradation rates at score boundaries, conditioned on the current rubric state. The training signal comes from paired grading results: scores for the same answers before and after an edit. Predictions select one candidate for direct evaluation, and held-out confidence bounds determine whether it corrects enough target errors while keeping new errors within prespecified limits. Measured effects update the model even when the edit is rejected. Across EIR (National), ASAP2.0, and ASAP-SAS with three frozen LLM graders, RubricWorld achieves the highest score in 20 of 27 metric comparisons. On ASAP2.0, it raises gpt-5-mini accuracy from 18.20% with the original rubric to 46.70%, exceeding the strongest baseline, APO, by 22.60 percentage points. code will be released upon acceptance.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. EDIT: Evidence-Diagnosed Intervention Training for Rule-Faithful LLM Grading

    Jun 4, 2026Zhihao Wu, Linhai Zhang, Taiyi Wang +4LLM-as-a-JudgeRubric-Based Evaluation

  2. From Structure to Validity: Mechanism-Guided Generation and Refinement of LLM Rubrics

    Oct 9, 2026Feifei Qian, Xuan Chen, Lixin Cui +6Automatic Rubric GenerationRubric-Based Evaluation