Rubrics

Ask about this topic

Momentum

12 papers in the last four weeks, up 140% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 60

All topics
CardsList

  1. RISED: RubrIcs for agentic multi-environment Selection and sElf-Distillation

    Oct 1, 2026Jingtan Wang, Sirajul Salekin, Young mok Jung +3Mixture-Of-AgentsReward Functions

    National University of Singapore · Work done during an internship at Apple · Apple

  2. Scoring Higher, Answering Worse: Mitigating Reward Hacking in Rubric-Based RL via Protocol-Level Rubrics

    Sep 30, 2026Maoqi Liu, Junwei He, Bowen Zhang +5RubricsScoring

    Beijing University of Posts and Telecommunications · ByteDance

  3. UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training

    Sep 29, 2026Ashish Jain, Armaan SandhuUser SimulationAgentic Benchmarks

    Sarvam AI

  4. A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses

    Sep 29, 2026Zhangshu Joshua Jiang, Zina Ibrahim, James T. TeoRubricsTask-Specific Rubrics

    DRIVE-Health CDT Department of Biostatistics and Health Informatics Institute of Psychiatry, Psychology and Neuroscience King’s College London, London, UK · Neurological Institute, Cleveland Clinic London, London, UK · King’s College London, London, UK +1

  5. Mubric: Mutation Testing-Guided Rubric Generation for LLM Evaluation

    Sep 29, 2026Jiayuxuan Yang, Jie M. Zhang, Yiling Lou +1RubricsLarge Language Model Evaluation

    Tsinghua University · King’s College London · University of Illinois Urbana-Champaign

  6. SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use Safety

    Sep 28, 2026Jianxing Chen, Xiao Yu, Shipra Agrawal +1Safety BenchmarksSecurity Evaluation

    Columbia University

  7. FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents

    Sep 28, 2026Hoyoung Lee, Suyeol Yun, Jack Haverty +17Finance BenchmarksTask-Specific Rubrics

    LinqAlpha · UNIST · MassMutual Life Insurance +9

  8. Rubric Rewards from Item Response Theory

    Sep 28, 2026Milad Yazdani, Yaser Souri, Xiren Zhou +4RubricsItem Response Theory

    Department of Electrical and Computer Engineering, University of British Columbia · Microsoft · School of Biomedical Engineering, University of British Columbia

  9. SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents

    Sep 28, 2026Bingqing Jiang, Guoxi Zhang, Jasper Wang +7Multimodal AgentsTask-Specific Rubrics

    The University of Hong Kong · WeChat, Tencent · Peking University +2

  10. Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It

    Sep 22, 2026Yu Sun, Junhao Xu, Jiajia Shi +1DecisionsChoice

    Fudan University · University of Science and Technology of China

  11. ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals

    Sep 15, 2026Bowen Qin, Yi Xie, Yesheng Liu +1RubricsImpossibility

    National University of Singapore · Peking University · Institute of Automation, Chinese Academy of Sciences

  12. Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models

    Sep 9, 2026Yupei Li, Qiyang Sun, Mohamed Mady +4Audio UnderstandingLarge Audio Language Models

    Imperial College London, UK · Technische Universität München München, Germany · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, AE +2

  13. PaperGym: Rubric-Centered Evolution for Research-Plan Generation

    Aug 31, 2026Yuhan Wang, Zhengxi Lu, Yuchen Yan +6Research AutomationRubrics

    Zhejiang University · Apple

  14. Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents

    Aug 31, 2026Xuehai Wang, Haowei Qin, Tongxin Liu +6Research AutomationRubrics

    1Zhejiang University · University of Electronic Science and Technology of China · 3Beijing University of Posts and Telecommunications +1

  15. GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation

    Aug 30, 2026Yifan Chen, Haitao Li, Qingyao Ai +4RubricsLarge Language Model Evaluation

    Department of Computer Science and Technology, Tsinghua University · National University of Singapore

  16. Graph-Structured Rubrics: Compiling Rubrics into Typed Evaluation Graphs for LLM Judges

    Aug 12, 2026Xi Chen, Jie Mu, Mo Xuan +1Rubric-Based ScoringRubrics

    Ant Group

  17. Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL

    Aug 12, 2026Minglai Yang, Xinyu Guo, Utkarsh Tyagi +6Task-Specific RubricsRubrics

    Scale AI · University of Arizona · University of Texas at Dallas

  18. PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis

    Aug 5, 2026Xiaomin He, Dongling Xiao, Jiahao Xie +4RubricsPrism

    ByteDance · Peking University

  19. FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables

    Aug 4, 2026Ben Wang, Kang Zhou, Lifan Guo +2Finance BenchmarksRubrics

    Qwen DianJin Team, Alibaba Cloud Computing

  20. Training Documents Reranker with Search Rubrics for Deep Research Agent

    Aug 4, 2026Wenhan Liu, Yu Lu, Qiaolin Xia +8Cross-Encoder RerankingDeep Research

    1Gaoling School of Artificial Intelligence, Renmin University of China · 2Tencent, Beijing, China

  21. Rubrics as Privileged Information for Open-Ended Generation

    Aug 3, 2026Deepika Bablani, Ajay Gupta, Wanming ChenRubricsReward Functions

    Apple

  22. CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

    Jul 31, 2026Mengting Chen, Yanshu Sun, Wanting Liang +5Task-Specific RubricsRubrics

    FinStep · StepFun

  23. Beyond Borrowed Histories: Person-Aligned User Simulation for Interactive Role-Playing Evaluation

    Jul 30, 2026Yuhang Zhu, Mingxuan Du, Benfeng Xu +3Role-Playing AgentsUser Simulation

    University of Science and Technology of China · MetaStone · MetaStone Technology, Beijing, China

  24. CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data

    Jul 17, 2026Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru +3Large Language Model EvaluationRubrics

    Scale AI

  25. Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence

    Jul 16, 2026Haocheng Yang, Licheng Pan, Yuan Lu +7RubricsTask-Specific Rubrics

    School of Computing, National University of Singapore · Xiaohongshu Inc. · School of Cyber Science and Technology, Zhejiang University +3

  26. Project Kaleidoscope: Contextual, Human-Aligned Evaluation for Real-World AI Applications

    Jul 16, 2026Leanne Tan, Rohan Jaggi, Shaun Khoo +1Artificial Intelligence EvaluationRubrics

    GovTech, Singapore · National University of Singapore · University of British Columbia

  27. Alipay-PIBench: A Realistic Payment Integration Benchmark for Coding Agents

    Jul 16, 2026Shiyu Ying, Xuejie Cao, Yingfan Ma +4Payment NetworksCoding Agents

    Ant Group

  28. When Rubrics Change: Cross-Rubric Generalization for Critical Thinking Essay Scoring

    Jul 15, 2026Nischal Ashok Kumar, Payu Wittawatolarn, Sana Kang +7Automated Essay ScoringTask-Specific Rubrics

    University of Massachusetts Amherst, United States · University of Minnesota, United States · Brighter Research, United States +3

  29. Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproduction

    Jul 14, 2026Hanhua Hong, Yizhi Li, Jiaoyan Chen +4Rubrics

    The University of Manchester · IQuest Research · University of Information Technology, VNU +2

  30. Articulate Intuition or Genuine Analysis? Benchmarking Epistemic Reliability in LLM-as-a-Judge Peer Reviews

    Jul 12, 2026Nuo Chen, Qian Wang, Qingyun Zou +1Peer ReviewEpistemology

    National University of Singapore

  31. RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification

    Jul 10, 2026Yanxuan Yu, Dong liu, Renata Borovica-Gajic +1Imbalanced ClassificationClass Imbalance

    Columbia University · University of California, Los Angeles · University of Melbourne

  32. LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL

    Jul 5, 2026Yujin Kim, Namgyu Ho, Sangmin Hwang +7Reinforcement Learning With Verifiable RewardTutors

    1KAIST · 2Upstage · University of Toronto +2

  33. SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use

    Jul 2, 2026Jiayin Zhu, Kelong Mao, Yudong Guo +4SkillsRubrics

    1HKUST(GZ), 2JD.COM · ∗Yutao Yue is also affiliated with Institute of Deep Perception Technology, JITRI, Wuxi, China.

  34. Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios?

    Jun 29, 2026Yangda Peng, Yunjia Qi, Haotian Xia +8Rubric-Based ScoringLlm-As-A-Judge

    Department of Computer Science and Technology, Tsinghua University · Tencent Hunyuan

  35. Rubric-as-Experts: Case-Specific MQM Rubrics for Translation Quality Evaluation

    Jun 19, 2026Weilu Xu, Yunzhi Shen, Xinye Wang +2Machine Translation QualityRubrics

    National Key Laboratory for Novel Software Technology, Nanjing University

  36. LLM-as-Judge in Education: A Curriculum-Grounded Marking Pipeline

    Jun 16, 2026Xiwei Xu, Chen Wang, Jacky Jiang +5Llm-As-A-JudgeCurriculum

    CSIRO, Australia · UNSW, Sydney, Australia · Studitory, Australia +1

  37. DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks

    Jun 11, 2026Jingxuan Han, Wei Liu, Mingyang Zhu +8Search AgentsAgentic Benchmarks

    University of Science and Technology of China · Meituan

  38. Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers

    Jun 10, 2026MohammadHossein Rezaei, Anas Mahmoud, Zihao Wang +6RubricsReinforcement Learning With Verifiable Reward

    Scale AI

  39. From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape

    Jun 7, 2026Hao Chen, Ziyu Han, Yukun Yan +3RubricsLarge Language Model Evaluation

    Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology · Department of Computer Science and Technology, Institute for AI, Tsinghua University

  40. Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics

    Jun 6, 2026Mengyuan Sun, Yu Li, Zhuohao Yu +2RubricsTask-Specific Rubrics

    National Engineering Research Center for Software Engineering, Peking University · University of Science and Technology of China

  41. Using Large Language Models to Support High Volume Application Review for an Undergraduate Research Program

    Jun 4, 2026Varun Aggarwal, Kay Kobak, John HowarterRubrics

    Engineering Undergraduate Research Office, Purdue University · Elmore School of Electrical and Computer Engineering, Purdue University · School of Materials Engineering, Purdue University

  42. QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards

    Jun 2, 2026Rongzhi Zhang, Rui Feng, Zhihan Zhang +8Reinforcement Learning With Verifiable RewardRubrics

    Amazon · Georgia Institute of Technology

  43. Deep Research as Rubric for Reinforcement Learning

    May 31, 2026Wangyi Mei, Zhouhong Gu, Zhenhan Bai +9RubricsDeep Research

    Fudan University · Xiaohongshu Inc. · Beijing University of Posts and Telecommunications

  44. Counterfactual Evaluation Reveals Hidden Capability Profiles in Clinical LLMs and Agents

    May 28, 2026Matt TurkCausal InterventionRubrics

    Protege Data Lab New York City, USA

  45. Generating and Refining Dynamic Evaluation Rubrics for LLM-as-a-Judge

    May 28, 2026Zijie Wang, Eduardo BlancoRubricsLlm-As-A-Judge

    University of Arizona Department of Computer Science

  46. MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection

    May 28, 2026Haowen Wang, Yaxin Du, Jian Yang +9Alignment-Aware SelectionTraining Data

    1Beihang University · 2IQuest Research · 3Shanghai Jiao Tong University +2

  47. Learnable Assessment Skills for LLM-based Automated Scoring: Rubric Construction via Iterative Optimization

    May 28, 2026Yun Wang, Xin Xia, Xuansheng Wu +2Rubric-Based ScoringTask-Specific Rubrics

    School of Computing, University of Georgia, Athens, GA, USA · AI4STEM Education Center, University of Georgia, Athens, GA, USA · The Hong Kong Polytechnic University, Hong Kong, China

  48. Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards

    May 26, 2026Yu Huang, Zihua Zhao, Zhaoxin Huan +9RubricsOffline Reinforcement Learning

    Shanghai Jiao Tong University · Ant Group

  49. Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents

    May 22, 2026Jiazheng Kang, Bowen Zhang, Zixin Song +4Search AgentsAgentic Reasoning

    Qwen Applications Business Group of Alibaba · Tsinghua University

  50. Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals

    May 22, 2026Sirui Chen, Lei Xu, Yuying Zhao +6LLM Reasoning StrategiesMetacognition

    1Tongji University · 2Shanghai AI Laboratory · 3Nanyang Technological University +3

  51. LP-Eval: Rubric and Dataset for Measuring the Quality of Legal Proposition Generation

    May 19, 2026Shanshan Xu, Johan Lindholm, Amogh Raina +2Legal Reasoning TasksLegal Domain

    University of Copenhagen · Umeå University

  52. Generative-Evaluative Agreement: A Necessary Validity Criterion for LLM-Enabled Adaptive Assessment

    May 19, 2026Grandee Lee, Yue Wang, Che Yee Lye +1Computerized Adaptive TestingRubrics

    Singapore University of Social Sciences

  53. Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs

    May 18, 2026Jacob Levine, Miguel Aenlle, Craig Zilles +2GradingHandwriting

    University of Illinois Urbana-Champaign

  54. Quantifying the Statistical Effect of Rubric Modifications on Human-Autorater Agreement

    May 7, 2026Jessica Huynh, Alfredo Gomez, Athiya Deviyani +3RubricsInter-Annotator Agreement

    Carnegie Mellon University · Google Research

  55. EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics

    May 5, 2026Shuyue Stella Li, Rui Xin, Teng Xiao +8Large Language Model EvaluationTask-Specific Rubrics

    University of Washington · 2Allen Institute for AI · University of Pennsylvania

  56. The Hitchhikers Guide to Rubric Quality Understanding and Enrichment

    Apr 1, 2026Ankit Aich, Zhengyang Qi, Charles Dickens +7Rubric-Based ScoringRubrics

    Snorkel AI · University of Wisconsin–Madison

  57. MASRubric: Auditing Information Flow in Multi-Agent Systems with Failure-Distilled Pitfall Rubrics

    Feb 26, 2026Yutong Wang, Siyuan Xiong, Xuebo Liu +4Model-Based Multi-Agent SystemsModel Auditing

    Harbin Institute of Technology, Shenzhen · Alibaba Group

  58. Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

    Feb 14, 2026Ruomeng Ding, Yifei Pang, He Sun +3RubricsJudges

    University of North Carolina at Chapel Hill · 2Carnegie Mellon University · 3Yale University +1

  59. From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges

    Jan 13, 2026Yihan Hong, Huaiyuan Yao, Bolin Shen +3Rubric-Based ScoringRubrics

    Washington University in St. Louis, St. Louis, MO, USA · Arizona State University, Tempe, AZ, USA · Florida State University, Tallahassee, FL, USA

  60. DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports

    Date pendingRuizhe Li, Mingxuan Du, Benfeng Xu +3Deep ResearchRubrics

    University of Science and Technology of China · Metastone Technology, Beijing, China