Rubric-Based Evaluation

Ask about this topic

Momentum

18 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 136

All topics
CardsList

  1. Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automatic Short Answer Scoring

    Oct 7, 2026Zhifan Sun, Sebastian Gombert, Jannik Lossjew +6Automated GradingEducational Assessment

    DIPF | Leibniz Institute for Research and Information in Education · IPN | Leibniz Institute for Science and Mathematics Education · Umeå University +1

  2. Rubric Spans are Label Representations: Joint LLM Encoding for Short Answer Scoring

    Oct 7, 2026Zhifan Sun, Sebastian Gombert, Fabian Zehner +3Automated GradingRubric-Based Evaluation

    DIPF | Leibniz Institute for Research and Information in Education · Centre for International Student Assessment (ZIB) · Computer Science Department & Studiumdigitale, Goethe University Frankfurt

  3. RubricArmor: Adversarial Evolution Improves LLM-Based Rubric Generation

    Oct 4, 2026Haocheng Yang, Yuchao Zhang, Licheng Pan +8Reward HackingRubric-Based RL

    National University of Singapore · Beijing University of Chemical Technology · Zhejiang University +7

  4. MatrixReward: Reward from Rubric Matrix for Open-Ended Generation

    Sep 30, 2026Zihan Shen, Qi Liu, Zixuan Yang +4Open-Ended GenerationReward Modeling

    Zhejiang University · Qwen Business Unit of Alibaba · Renmin University of China

  5. A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses

    Sep 29, 2026Zhangshu Joshua Jiang, Zina Ibrahim, James T. TeoClinical ReasoningRubric-Based Evaluation

    DRIVE-Health CDT Department of Biostatistics and Health Informatics Institute of Psychiatry, Psychology and Neuroscience King’s College London, London, UK · Neurological Institute, Cleveland Clinic London, London, UK · King’s College London, London, UK +1

  6. Mubric: Mutation Testing-Guided Rubric Generation for LLM Evaluation

    Sep 29, 2026Jiayuxuan Yang, Jie M. Zhang, Yiling Lou +1Language Model Generation EvaluationRubric-Based Evaluation

    Tsinghua University · King’s College London · University of Illinois Urbana-Champaign

  7. VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics

    Sep 29, 2026Bo Lv, Mao Zheng, Zheng Li +3Multilingual VLM EvaluationRubric-Based Evaluation

    Foundation Model Department, Tencent

  8. FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents

    Sep 28, 2026Hoyoung Lee, Suyeol Yun, Jack Haverty +17LLM Agent EvaluationRubric-Based Evaluation

    LinqAlpha · UNIST · MassMutual Life Insurance +9

  9. SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents

    Sep 28, 2026Bingqing Jiang, Guoxi Zhang, Jasper Wang +7Multimodal Reward ModelingRubric-Based RL

    The University of Hong Kong · WeChat, Tencent · Peking University +2

  10. JEV vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places

    Sep 24, 2026Delip Rao, Chris Callison-BurchLLM Inference EfficiencyLLM Evaluation

    University of Pennsylvania

  11. Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judges

    Sep 22, 2026Amit Jadhav, Shaurya Beriwala, Beomjin KimVLM EvaluationVision-Language Models

    Purdue University Fort Wayne

  12. RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling

    Sep 19, 2026Zhenchen Tang, Yang Li, Songlin Yang +6Reward ModelingMultimodal Reward Modeling

    New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tencent +1

  13. The Complexity Kink: A Prompt-Side Structural Complexity Index for Code-Generation Reliability

    Sep 17, 2026Michael Hernandez, Tian ZhaoRubric-Based EvaluationCode Generation Evaluation

    University of Wisconsin–Milwaukee

  14. ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals

    Sep 15, 2026Bowen Qin, Yi Xie, Yesheng Liu +1Reward HackingLanguage Model Generation Evaluation

    National University of Singapore · Peking University · Institute of Automation, Chinese Academy of Sciences

  15. Don't Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction

    Sep 14, 2026Hayeong Ryu, Sunhee Jo, Seunguk Yu +1Reward ModelingReference-Free Evaluation

    Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University

  16. When Rubrics Fail: Hallucinations Reveal Blind Spots in Medical AI Evaluation

    Sep 14, 2026Griffin Farrow, Lily Sijia Li, Jack Johnson +4LLM Hallucination DetectionRubric-Based Evaluation

    University of Oxford

  17. Vision-Language Models for Criterion-Level Grading of Handwritten Examinations in Outcome-Based Education

    Sep 13, 2026Asif Hasan Tonmoy, Saad Ahmed, Md Khalid Syfullah +1VLM EvaluationRubric-Based Evaluation

  18. Rubric-Aligned Disentangled Evaluation of Human Simultaneous Interpreting

    Sep 11, 2026Ziyu Zhang, Satoshi NakamuraRubric-Based EvaluationAutomated Evaluation

    School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China · Shenzhen Loop Area Institute, China

  19. Vision-language models know more about agriculture than they show and rubric-grounded verifications close the gap

    Sep 8, 2026Earl Ranario, Jared Smith, Lars Lundqvist +2Agricultural Image AnalysisPlant Disease Classification

    University of California, Davis

  20. Bridging Network Psychometrics and Artificial Intelligence: An Ising-Potts Model with LLM-Derived Weights

    Sep 8, 2026Matthias von DavierInter-Rater ReliabilityPsychometric Validation

  21. Efficient Test-Time Adaptation through Human-AI Interaction

    Sep 3, 2026Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao +22Test-Time AdaptationRubric-Based Evaluation

    Carnegie Mellon University · University of Washington · Handshake AI +3

  22. PhoenixNest-Video: Evidence-Grounded Multimodal Agent Framework for Automated Video Interview Assessment

    Sep 2, 2026Fan Yuxuan, Huang Miaojun, Zhang Haimei +2Automated EvaluationRubric-Based Evaluation

    The Hong Kong University of Science and Technology (Guangzhou)

  23. Towards a Reliable and Practical Eval Pipeline

    Sep 1, 2026Emma Thuong Nguyen, Abhishek GhoseLLM EvaluationLLM-as-a-Judge

    Salesforce

  24. PaperGym: Rubric-Centered Evolution for Research-Plan Generation

    Aug 31, 2026Yuhan Wang, Zhengxi Lu, Yuchen Yan +6AI Agents for Scientific DiscoveryRubric-Based RL

    Zhejiang University · Apple

  25. Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents

    Aug 31, 2026Xuehai Wang, Haowei Qin, Tongxin Liu +6AI Agent EvaluationAutonomous Scientific Discovery

    1Zhejiang University · University of Electronic Science and Technology of China · 3Beijing University of Posts and Telecommunications +1

  26. Small Language Models as Judges for Rubric-Based Reinforcement Learning

    Aug 30, 2026Fengyu Xie, Yilun Zhao, Bingsen Chen +2LLM-as-a-JudgeLanguage Model Generation Evaluation

    New York University · Yale University

  27. GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation

    Aug 30, 2026Yifan Chen, Haitao Li, Qingyao Ai +4LLM-as-a-JudgeLanguage Model Generation Evaluation

    Department of Computer Science and Technology, Tsinghua University · National University of Singapore

  28. ExecRubrics: Executable Tool-Augmented Rubrics for Verifiable and Efficient Long-Form Evaluation

    Aug 23, 2026Kaustubh D. Dhole, Charles L. A. Clarke, Eugene Y. AgichteinLLM EvaluationRubric-Based Evaluation

    Department of Computer Science Emory University Atlanta, USA · School of Computer Science University of Waterloo Waterloo, Canada

  29. Graph-Structured Rubrics: Compiling Rubrics into Typed Evaluation Graphs for LLM Judges

    Aug 12, 2026Xi Chen, Jie Mu, Mo Xuan +1Human Preference EvaluationLLM-as-a-Judge

    Ant Group

  30. Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL

    Aug 12, 2026Minglai Yang, Xinyu Guo, Utkarsh Tyagi +6Reward HackingRubric-Based RL

    Scale AI · University of Arizona · University of Texas at Dallas

  31. RAVEN-Eval: Rubric-Guided Automatic Evaluation for AI Video Generation Models Based on LMM Preference Judgement

    Aug 10, 2026Ziheng Jia, Jiaying Qian, Zicheng Zhang +3LLM-as-a-JudgeRubric-Based Evaluation

    Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory

  32. VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing

    Aug 6, 2026Ziyun Zeng, Zixuan Wang, Yongsheng Yu +2Video Editing BenchmarksVideo Editing

    University of Rochester

  33. PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis

    Aug 5, 2026Xiaomin He, Dongling Xiao, Jiahao Xie +4Multimodal GenerationRubric-Based Evaluation

    ByteDance · Peking University

  34. AudioScape-TTA: A Structured Soundscape Benchmark for Fine-Grained Text-to-Audio Evaluation

    Aug 5, 2026Jinting Wang, Yuguang Yang, Shengyu Li +4Text-to-Audio GenerationRubric-Based Evaluation

    1The Hong Kong University of Science and Technology (Guangzhou) · 2Tencent · 3Zhejiang University

  35. FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation

    Aug 5, 2026Yinghao Tang, Tan Zhenwei, Yiyao Wang +4Financial ServicesLLM Evaluation

    State Key Laboratory of CAD&CG, Zhejiang University · Ant Group

  36. FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables

    Aug 4, 2026Ben Wang, Kang Zhou, Lifan Guo +2AI Agent EvaluationAI Agent Benchmarks

    Qwen DianJin Team, Alibaba Cloud Computing

  37. Rubrics as Privileged Information for Open-Ended Generation

    Aug 3, 2026Deepika Bablani, Ajay Gupta, Wanming ChenOpen-Ended GenerationRubric-Based Evaluation

    Apple

  38. RADAR: Rubric-Aware Dependency and Redundancy Analysis for LLM-as-Judge Evaluation

    Aug 3, 2026Divyansh Singh, Reza Davari, Afra MashhadiLLM-as-a-JudgeLLM Auditing

    Microsoft · University of Florida · University of Washington

  39. Comparative Validation of GPT-4o-mini and Teacher Mean Scores for Automated Scoring of Music Analysis Responses: Single-Pass Deployment, Repeatability, and Strategy-Specific Bias

    Aug 3, 2026Baicheng Lin, Lingxi Jin, Kyung-Seok MinLLM-as-a-JudgeEducational Technology

    Department of Education, Sejong University, South Korea · Department of Educational Technology, Ewha Womans University, South Korea

  40. AI-Based Thesis Assessment: An Empirical Study of Human Evaluation Priorities and Their Impact on Automated Assessment

    Aug 1, 2026Garv Vikram Gursahaney, Baskhad Idrisov, Thorsten Fröhlich +1Rubric-Based EvaluationAutomated Grading

    IU International University of Applied Sciences, Germany.

  41. CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

    Jul 31, 2026Mengting Chen, Yanshu Sun, Wanting Liang +5Open-Ended GenerationLLM Evaluation

    FinStep · StepFun

  42. TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation

    Jul 31, 2026Jiahui Zhang, Ziwei Zhang, Yipeng Wang +7Large Language Model-Based Role-Play SimulationAgent Evaluation

    Kuaishou GameMind Lab

  43. An Instrument to Evaluate Governance Proposals: AI Policy Analysis at Scale

    Jul 30, 2026Paulo Carvao, Claudio Mayrink Verdun, Isabel Adler +1LLM EvaluationPolicy Evaluation

    Mossavar-Rahmani Center for Business and Government · Harvard Kennedy School · Harvard Kennedy School of Government +2

  44. Share the Judge, Learn the Deferral: Where Specialization Helps LLM Evaluation

    Jul 30, 2026Weining ZhangLLM EvaluationLLM-as-a-Judge

    Cheung Kong Graduate School of Business

  45. Capital Markets LLM Reliability Score (CM-LRS): From Plausible to Bankable

    Jul 23, 2026Prerit AhujaFinancial ServicesLLM Evaluation

    Independent Researcher · London-based

  46. Co-Evolving LLM Evaluators and Policies via DynamicRubric

    Jul 22, 2026Beining Wang, Weihang Su, Hongtao Tian +8RL for Language ModelsLLM Evaluation

    Department of Computer Science and Technology, Tsinghua University · 2WeChat, Tencent

  47. Beyond Relevance-Centric Retrieval: Rubric-Oriented Document Set Selection and Ranking

    Jul 22, 2026Kailin Jiang, Lei Liu, Jian Xi +7Learning to RankRubric-Based Evaluation

    University of Science and Technology of China · Yuanbao Team, Tencent · University of Chinese Academy of Sciences +1

  48. Two-Level Meta-Rubrics for Evaluating Open-Ended Generation: GAMUT, a Benchmark for Factual Completeness

    Jul 21, 2026Xilun Chen, Zhaleh Feizollahi, Ross Goodwin +5Open-Ended GenerationLLM-as-a-Judge

    Meta AI

  49. Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards

    Jul 21, 2026Xuefeng Jin, Jiashuo Zhang, Teng Cao +1Automated Essay ScoringRubric-Based RL

    Alibaba Cloud Computing, Alibaba Group

  50. Assessment in Team Problem-Solving Exercises in Computing Education

    Jul 21, 2026Valdemar Švábenský, Jan Vykopal, Sukrit Leelaluk +3Educational AssessmentRubric-Based Evaluation

    Faculty of Informatics Masaryk University Brno, Czech Republic · Center for ICT Infrastructure Yamaguchi University Yamaguchi, Japan · Graduate School of Information Science and Electrical Engineering, Kyushu University Fukuoka, Japan

  51. EduPanel: A Three-Agent LLM Judge for Teaching Videos -- Reliability, Complementarity, and Human Trust Calibration

    Jul 20, 2026Jia-Kai Dong, Yi-Cheng Lin, Hung-yi LeeLLM-as-a-JudgeEducational Assessment

    National Taiwan University · NTU Artificial Intelligence Center of Research Excellence

  52. AgentFAIR: A Multi-Agent Collaborative Framework for FAIRness Evaluation of Geospatial Datasets

    Jul 17, 2026Ming Chen, Pranav PaiMulti-Agent SystemsRubric-Based Evaluation

    The University of Melbourne Melbourne, Victoria, Australia

  53. Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence

    Jul 16, 2026Haocheng Yang, Licheng Pan, Yuan Lu +7Rubric-Based Evaluation

    School of Computing, National University of Singapore · Xiaohongshu Inc. · School of Cyber Science and Technology, Zhejiang University +3

  54. Project Kaleidoscope: Contextual, Human-Aligned Evaluation for Real-World AI Applications

    Jul 16, 2026Leanne Tan, Rohan Jaggi, Shaun Khoo +1Human-in-the-Loop EvaluationRubric-Based Evaluation

    GovTech, Singapore · National University of Singapore · University of British Columbia

  55. When Rubrics Change: Cross-Rubric Generalization for Critical Thinking Essay Scoring

    Jul 15, 2026Nischal Ashok Kumar, Payu Wittawatolarn, Sana Kang +7Educational AssessmentAutomated Essay Scoring

    University of Massachusetts Amherst, United States · University of Minnesota, United States · Brighter Research, United States +3

  56. Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproduction

    Jul 14, 2026Hanhua Hong, Yizhi Li, Jiaoyan Chen +4LLM EvaluationRubric-Based Evaluation

    The University of Manchester · IQuest Research · University of Information Technology, VNU +2

  57. Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents

    Jul 14, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +4LLM Agent EvaluationLLM Agent Self-Improvement

    1AWS Generative AI Innovation Center · 2HSBC Holdings Plc., HSBC Technology Center, China

  58. FinResearchBench II: A Deep Research Benchmark with Consensus-Derived Gold Rubrics for Distinguishing Financial Report Quality

    Jul 14, 2026Beidi Luan, Rui Sun, Sinuo Wang +5LLM-as-a-JudgeRubric-Based Evaluation

    1StepFun · University of Adelaide · 2FinStep +1

  59. Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution

    Jul 9, 2026Ethan Leung, Elias Lumer, Corey Feld +3LLM-as-a-JudgeCitation Verification

    Commercial Technology and Innovation Office, PricewaterhouseCoopers, U.S.

  60. Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach

    Jul 2, 2026Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard +2LLM-as-a-JudgeComputer Science Education

    aUniversidade de Vigo, Department of Computer Science, ESEI-Higher School of Computer Engineering, Edificio Politécnico, Campus Universitario As Lagoas s/n, Ourense, 32004, Spain · bIFCAE-Institute for Research in Physics, Computing and Aerospace Science, Universidade de Vigo, Ourense, 32004, Spain