Rubric-Based RL

RL: Reinforcement Learning

Overview

Rubric-based reinforcement learning trains language and multimodal models using explicit criteria—such as factual accuracy, instruction coverage, or safe tool use—instead of relying only on a final answer or a single scalar score. Common approaches generate or adapt rubrics for each task, have language-model judges or deterministic verifiers score them, and optimize policies with methods such as GRPO; some methods also assign credit to reasoning steps, tokens, or agent actions. This provides more detailed supervision for open-ended tasks, including writing, research, medical reasoning, image generation, and tool use. The central tension is that rubrics can make rewards more informative and interpretable, but incomplete criteria, biased judges, and reward aggregation can still encourage gaming or behavior that scores well without improving overall quality.

Latest papers

CardsList

112 papers12 papers in the last four weeks, down 61% on the four weeks before. 0.1% of all new papers.

  1. Personalized Memory Rubrics: Learning What to Remember per User via Rubric-Guided Credit Assignment

    Nov 10, 2026Yupeng Han, Kai Zhang, Shuochen Liu +6LLM Agent MemoryMemory Management for LLM Agents

  2. RuACT: Recovering Inactive Rubrics as Adaptive Critics and Teachers

    Nov 6, 2026Yijun Li, Zhengxi Lu, Zhipiao Liu +4Rubric-Based RL

  3. Using Experience Twice: Dynamic Process Rubrics for Privileged Token-Level Credit Assignment

    Nov 4, 2026Shumao Wu, Kongcheng Zhang, Jianqiang Huang +1Token-Level Credit AssignmentFine-Grained Credit Assignment

  4. Rubric Curriculum RL: Exploiting the Generation-Verification Gap in Non-Verifiable Domains

    Nov 4, 2026Sumeet Ramesh Motwani, Tejas Krishnan, Alistair Letcher +10RL for Language ModelsRL for Generative Models

  5. Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning

    Nov 3, 2026Kevin Frans, Alexander M Rush, Sergey Levine +2RL for LLM AgentsAgentic RL

  6. Reward-Adaptive Co-Evolution of Policy and Rubric

    Oct 29, 2026Yizhu Chen, Jianguang Jiang, Jiameng Huang +5Rubric-Based RLReward Design for RL

  7. Co-Evolving Policy and Rubric via Capability Awareness for Open-Ended Generation

    Oct 28, 2026Chengsheng Zhang, Wenshen Xu, Yuchuan Wu +6RL for Language ModelsRubric-Based RL

  8. SetGRPO: Calibrating Grader-Induced Policy Optimization

    Oct 27, 2026Jiale Huang, Zixu Li, Zhiwei Chen +3Reinforcement LearningGroup Relative Policy Optimization

  9. Eat Your Precision: Precision Objective Mitigates Reward Hacking in Rubric-based RL

    Oct 24, 2026Hyun Ryu, Jong-Heon Kim, Yiqing Xie +3Rubric-Based RLReward Design for RL

  10. Intra-Rubric Reward Divergence: Mitigating Rubric Hacking through Latent Dimension Recovery and Balancing

    Oct 19, 2026Jianqiang Xia, Han Xu, Dianxi ShiReward HackingRubric-Based RL

  11. HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL

    Oct 19, 2026Xiaoqiang Wang, Ting-En Lin, Yuchuan Wu +7Rubric-Based RLReward Design for RL

  12. Rubrics Beyond Evaluation: Exploiting Rubric Structure in Reinforcement Learning

    Oct 19, 2026Yuxun Huang, Yunyao Zhang, ZeLiang Chen +3Rubric-Based RLCredit Assignment in RL

  13. RARE: Reinforcement Learning with Adaptive Rubric Evolution for Open-Ended Generation

    Oct 18, 2026Xueyun Tian, Bingbing Xu, Chuyue Li +9Rubric-Based RLReward Design for RL

  14. Expert-Guided Rubrics as Dense Reward Signals for Reinforcement Learning in Non-Verifiable Domains

    Oct 17, 2026Ashish Bansal, Hossam Amer, Keerat Kaur Guliani +3RL for LLM AgentsReward Design for RL

  15. Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation

    Oct 15, 2026Licheng Pan, Haocheng Yang, Haoxuan Li +7Language Model Generation EvaluationRubric-Based Evaluation

  16. Judging the Judges: Confidence and Disagreement in Rubrics as Rewards

    Oct 15, 2026Michael Tian-Yue Liu, Anne Harrington, Trevor Darrell +1Rubric-Based RLReward Design for RL

  17. Recursive Reward Improvement via Adversarial Meta-Prompt Evolution

    Oct 15, 2026Wei Dai, Qianchu Liu, Guanghui Qin +4Recursive Self-ImprovementRubric-Based RL

  18. PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis

    Oct 14, 2026Qiushi Sun, Jingyang Gong, Yang Liu +14Multimodal Large Language ModelsCode Generation

  19. CoCA: Criterion-Aware Credit Assignment for Rubric-Based Reinforcement Learning

    Oct 13, 2026Guangcheng Zhu, Yingfan Ma, Xing Zheng +7RL for Language ModelsRubric-Based RL

  20. RELAY: Selective Self-Distillation with Adaptive Hand-off for Rubric-Based Reinforcement Learning

    Oct 13, 2026Kunwang Zheng, Xinyu Sun, Yu Cao +7Adaptive Knowledge DistillationRubric-Based RL

  21. EnGRICH: Enhancing Generative Reward Modeling with Critiques from Humans

    Oct 4, 2026Xuancheng Li, Beining Wang, Haitao Li +7Reward ModelingProcess Reward Models

  22. RubricArmor: Adversarial Evolution Improves LLM-Based Rubric Generation

    Oct 4, 2026Haocheng Yang, Yuchao Zhang, Licheng Pan +8Reward HackingRubric-Based RL

  23. Self-Evaluating Recursive Agents

    Oct 4, 2026TianYi Lyu, Xiaozhe Li, Yang Li +7LLM Agent TrainingRecursive Language Models

  24. Rubric-Based Optimization for Text-to-Music Generation

    Oct 2, 2026Ping Wang, Guang Yang, Shao-Rong Su +3Text-to-Music GenerationRubric-Based RL

  25. OPD Before RL: Warm-Starting Rubric-Based RL with On-Policy Distillation

    Oct 2, 2026Xinpeng Wang, Wei Shi, Yu-Chia Chen +3RL for Language ModelsRubric-Based RL

  26. RISED: RubrIcs for agentic multi-environment Selection and sElf-Distillation

    Oct 1, 2026Jingtan Wang, Sirajul Salekin, Young mok Jung +3Reinforcement LearningRubric-Based RL

  27. MatrixReward: Reward from Rubric Matrix for Open-Ended Generation

    Sep 30, 2026Zihan Shen, Qi Liu, Zixuan Yang +4Open-Ended GenerationReward Modeling