Rubric-Based Evaluation
Overview
Rubric-based evaluation turns broad quality judgments into explicit dimensions, score levels, or checkable criteria, often tied to evidence in a response or task trajectory. Researchers use expert-written, retrieved, or automatically generated rubrics, then apply LLM judges, specialized scoring models, or executable checks across settings such as education, clinical reasoning, research agents, and multimodal generation. Rubric scores also serve as reward signals for training, where criterion weighting, calibration, and task-specific adaptation can make feedback more useful. A central challenge is that detailed rubrics do not guarantee valid judgments: criteria can be incomplete or biased, judges can disagree with people, and models may exploit what rubrics reward.
Latest papers
198 papers29 papers in the last four weeks, up 26% on the four weeks before. 0.2% of all new papers.