Rubrics

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

4 new papers

A weekly snapshot of new work published in Rubrics.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Rubrics.

Period ending 2026-09-07

6 new papers

A weekly snapshot of new work published in Rubrics.

Inside this field

Focused directions

Focused topic

Rubrics

Within Rubrics

103 papersExplore

Focused topic

Rubric-Based Reinforcement Learning

Within Rubrics

48 papersExplore

Focused topic

Rubric Framework

Within Rubrics

17 papersExplore

145 papers

Latest in Rubrics

Browse all topics →

Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling
Jul 2, 2026Dazhi Fu, Jiuding Yang, Yiwen Guo +1Rubrics Large Language Model Judges
Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios?
Jun 29, 2026Yangda Peng, Yunjia Qi, Haotian Xia +8Llm-As-A-Judge Rubrics
EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions
Jun 22, 2026Jincheng Zhong, Weizhi Wang, Che Jiang +5Sessions Rubrics
EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learning
Jun 22, 2026Hongxin Ding, Baixiang Huang, Yue Fang +6Rubric-Based Reinforcement Learning Rubrics
Zero-Shot Vision-Language Models for Classroom Engagement Recognition: A Benchmark Study of Prompt Sensitivity and Cross-Dataset Generalization
Jun 20, 2026Aman Goyal, Kshama Nitin Shah, Kemmannu Vineet Venkatesh RaoLarge Vision Language Models Video Dataset
Rubric-as-Experts: Case-Specific MQM Rubrics for Translation Quality Evaluation
Jun 19, 2026Weilu Xu, Yunzhi Shen, Xinye Wang +2Machine Translation Quality Rubric Framework
ARCO: Adaptive Rubric with Co-Evolution for Multi-Step LLM-Based Agents
Jun 19, 2026Zihang Tian, Jingsen Zhang, Rui Li +3Rubric-Based Reinforcement Learning Rubric Framework
Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
Jun 17, 2026Siyi Gu, Jialin Chen, Sophia Zhou +2Rubric-Based Reinforcement Learning Unsupervised On-Policy Self-Distillation
RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills
Jun 16, 2026Weizhi Zhang, Zechen Li, Hamid Palangi +16Rubrics
LLM-as-Judge in Education: A Curriculum-Grounded Marking Pipeline
Jun 16, 2026Xiwei Xu, Chen Wang, Jacky Jiang +5Llm-As-A-Judge Rubrics
DEEPRUBRIC: Evidence-Tree Rubric Supervision for Efficient Reinforcement Learning of Deep Research Agents
Jun 15, 2026Minghang Zhu, Chuyang Wei, Junhao Xu +3Rubric-Based Reinforcement Learning Deep Research
CoRA: Confidence-Rationale Alignment for Reliable Chain-of-Thought Reasoning
Jun 12, 2026Juming Xiong, Weixin Liu, Kevin Guo +9Large Language Model Reasoning Chain-of-Thought Reasoning
When Good Verifiers Go Bad: Silent Negative Transfer in Verifier-Guided VLM Training
Jun 12, 2026Jianzhe LinVerifier Reinforcement Learning With Verifiable Reward
DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks
Jun 11, 2026Jingxuan Han, Wei Liu, Mingyang Zhu +8Search Agents Report
Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers
Jun 10, 2026MohammadHossein Rezaei, Anas Mahmoud, Zihao Wang +6Rubrics Reinforcement Learning With Verifiable Reward
Are LLMs Bad at Moral Reasoning?
Jun 10, 2026Menghang Zhu, Seth LazarMoral Reasoning Rubrics
LexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks
Jun 8, 2026Yifan Chen, Haitao Li, Yiran Hu +6Legal Domain Rubrics
Reliable to Expressive: A Curriculum for Rubric-Following Safety Judges
Jun 8, 2026Yongtaek Lim, Hyeji Choi, Minwoo KimSafety Evaluation Rubric Framework
ComplexConstraints and Beyond: Expert Rubrics for RLVR
Jun 8, 2026Sushant Mehta, Liudas Panavas, Suhaas Garre +1Rubric-Based Reinforcement Learning Rubrics
From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape
Jun 7, 2026Hao Chen, Ziyu Han, Yukun Yan +3Rubrics Large Language Model Evaluation
Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics
Jun 6, 2026Mengyuan Sun, Yu Li, Zhuohao Yu +2Rubrics Discriminative
Beyond Rubrics: Exploration-Guided Evaluation Skills for Reward Modeling
Jun 5, 2026Xing Yue, Linjuan Wu, Daoxin Zhang +2Rubrics Process Reward Modeling
MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring
Jun 4, 2026Ali Keramati, Shiyuan Zhou, Sharad Mehrotra +1Automated Essay Scoring Llm-As-A-Judge
Using Large Language Models to Support High Volume Application Review for an Undergraduate Research Program
Jun 4, 2026Varun Aggarwal, Kay Kobak, John HowarterLarge Language Model Output Rubrics
Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning
Jun 3, 2026Xuekang Wang, Zhuoyuan Hao, Shuo Hou +3Rubric-Based Reinforcement Learning Implicit Bias
POLARIS: Guiding Small Models to Write Long Stories
Jun 2, 2026Rishanth Rajendhran, Jenna Russell, Mohit Iyyer +1Narratives Small Models
QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards
Jun 2, 2026Rongzhi Zhang, Rui Feng, Zhihan Zhang +8Rubric-Based Reinforcement Learning Reinforcement Learning With Verifiable Reward
Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learning
Jun 2, 2026Can Lv, Mingju Chen, Heng Chang +1Rubric-Based Reinforcement Learning Rubrics
RUBAS: Rubric-Based Reinforcement Learning for Agent Safety
Jun 2, 2026Xian Qi Loye, Qinglin Su, Zhexin Zhang +5Rubric-Based Reinforcement Learning Agentic Tool Use
ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents
Jun 2, 2026Zheng Liu, Longxiang Zhang, Xintong Wang +8Reliable Medical Process-Level Reward Model Evaluation Framework Rubric-Based Reinforcement Learning
The Geometry of LLM-as-Judge: Why Inter-LLM Consensus Is Not Human Alignment
Jun 2, 2026Sourabrata Mukherjee, Hamna Hamna, Kalika Bali +1Llm-As-A-Judge Judges
Deep Research as Rubric for Reinforcement Learning
May 31, 2026Wangyi Mei, Zhouhong Gu, Zhenhan Bai +9Rubric-Based Reinforcement Learning Rubrics
LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards
May 29, 2026Nianyi Lin, Jiajie Zhang, Lei Hou +1Large Language Model Reasoning Reinforcement Learning With Verifiable Reward
Fine-grained Verification via Diagnostic Reasoning Supervision for Aspect Sentiment Triplet Extraction
May 29, 2026Wenna Lai, Haoran Xie, Guandong Xu +2Rdf Sentiment Analysis
SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Tasks
May 29, 2026Wai-Chung Kwan, Aryo Pradipta Gema, Joshua Ong Jun Leang +1Self-Play Open-Ended Responses
PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges
May 29, 2026Swastik Roy, Rajkumar Pujari, Tharindu Kumarage +5Rubrics Llm-As-A-Judge
Counterfactual Evaluation Reveals Hidden Capability Profiles in Clinical LLMs and Agents
May 28, 2026Matt TurkArtificial Intelligence Evaluation Counterfactual Reasoning
Generating and Refining Dynamic Evaluation Rubrics for LLM-as-a-Judge
May 28, 2026Zijie Wang, Eduardo BlancoRubrics Human Annotations
MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection
May 28, 2026Haowen Wang, Yaxin Du, Jian Yang +9Large Language Rubrics
Reinforcement Learning with Robust Rubric Rewards
May 28, 2026Ya-Qi Yu, Hao Wang, Fangyu Hong +15Reinforcement Learning With Verifiable Reward Rubric-Based Reinforcement Learning
EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generation
May 28, 2026Xin Guan, Xiaomeng Hu, Shen Huang +6Rubric-Based Reinforcement Learning Training-Free Reward Alignment
Rubric-Guided Process Reward for Stepwise Model Routing
May 28, 2026Shenghao Ye, Yu Guo, Zhengheng Li +2Reliable Medical Process-Level Reward Model Evaluation Framework Large Language Model Routing
Prompt-Level Reward Specifications for Open-Ended Post-Training
May 28, 2026Zijun Weng, Xiaohui Hu, Shuangyong Song +3Rubric-Based Reinforcement Learning LLM Post-Training Methods
Learnable Assessment Skills for LLM-based Automated Scoring: Rubric Construction via Iterative Optimization
May 28, 2026Yun Wang, Xin Xia, Xuansheng Wu +2Rubrics Computerized Adaptive Testing
RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains
May 27, 2026Haoxiang Jiang, Zihan Dong, Tianci Liu +5Rubric-Based Reinforcement Learning Rubrics
Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generation
May 26, 2026Zixuan Yang, Yiqun Chen, Wei Yang +7Rubric-Based Reinforcement Learning Grading
Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards
May 26, 2026Yu Huang, Zihua Zhao, Zhaoxin Huan +9Rubric-Based Reinforcement Learning Rubrics
CODESKILL: Learning Self-Evolving Skills for Coding Agents
May 25, 2026Yanzhou Li, Yiran Zhang, Xiaoyu Zhang +2Skill Evolution Reusable Skills
Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents
May 22, 2026Jiazheng Kang, Bowen Zhang, Zixin Song +4Search Agents Agentic Reasoning
ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning
May 22, 2026Xiaoyuan Li, Keqin Bao, Moxin Li +5Rubric-Based Reinforcement Learning Question-Answer Pairs
Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals
May 22, 2026Sirui Chen, Lei Xu, Yuying Zhao +6Large Language Model Reasoning Metacognition
Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR
May 19, 2026Utkarsh Tyagi, Xingang Guo, MohammadHossein Rezaei +5Rubric-Based Reinforcement Learning Rubric Framework
LP-Eval: Rubric and Dataset for Measuring the Quality of Legal Proposition Generation
May 19, 2026Shanshan Xu, Johan Lindholm, Amogh Raina +2Legal Domain Rubrics
Generative-Evaluative Agreement: A Necessary Validity Criterion for LLM-Enabled Adaptive Assessment
May 19, 2026Grandee Lee, Yue Wang, Che Yee Lye +1Computerized Adaptive Testing Rubrics
FAGER: Factually Grounded Evaluation and Refinement of Text-to-Image Models
May 18, 2026Youngsun Lim, Cusuh Ham, Pin-Yu Chen +1Multilingual Fact-Checking Text-To-Image
Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs
May 18, 2026Jacob Levine, Miguel Aenlle, Craig Zilles +2Grading Handwriting
AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning
May 18, 2026Peilin Wu, Xinlu Zhang, Kun Wan +4Rubric-Based Reinforcement Learning Task-Aware Evaluation Framework
Step-wise Rubric Rewards for LLM Reasoning
May 17, 2026Weichu Xie, Haozhe Zhao, Wenpu Liu +15Rubric-Based Reinforcement Learning Rubric Framework
Can Vision Language Models Be Adaptive in Mathematics Education? A Learner Model-based Rubric Study
May 15, 2026Jie Gao, Yongan Yu, Junzhu Su +3Vision-Language Model Adaptation Adaptive Learning
Context Pruning for Coding Agents via Multi-Rubric Latent Reasoning
May 14, 2026Jingjing Wang, Xiwen Chen, Wenhui Zhu +6Coding Agents Efficient Latent Reasoning

Jul 2, 2026 · cs.LGJ/K move · Enter open · S save

Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling

Dazhi Fu, Jiuding Yang, Yiwen Guo, Jicong Fan

Reliable reward and preference signals are critical for evaluating and optimizing large language models on open-ended tasks. Rubric-based judges offer a transparent way to decompose such judgments into explicit evaluation criteria, but existing annotation-free rubric generators typically rely on a single generic evaluator. As a result, they may overlook important dimensions of human preference, a failure mode we term dimensional blind spots. To address this limitation, we propose Multi-Role Rubric Generation (MRRG), a training-free and reference-free framework that elicits evaluation criteria from multiple complementary roles and consolidates them into an auditable rubric-based scorer. This scorer can be used both to validate pairwise preferences and to provide rewards for GRPO-style Reinforcement Learning with Verifiable Rewards (RLVR). Experiments on preference validation benchmarks show that MRRG consistently outperforms single-role rubric generation baselines across multiple backbone models. Further RLVR experiments demonstrate that MRRG yields a stronger reward signal for improving open-ended generation.

Rubrics Large Language Model Judges Reward Modeling Outcome-Based Rewards

Open paper Ask about it PDF