Reward Model Evaluation

Ask about this topic

Overview

Reward model evaluation studies whether learned judges match human preferences and provide useful signals for selecting or improving language-model and agent behavior. Researchers use pairwise and listwise preference benchmarks, calibration and uncertainty metrics, and stress tests across settings such as long-form writing, math, code, personalized dialogue, and robot or computer-use trajectories. Evaluations also test downstream effects in best-of-N selection and reinforcement learning, alongside methods such as human-verified data, LLM judges, and perturbation-based audits. A central tension is that strong benchmark accuracy may not transfer across domains or predict safe policy improvements, especially when long-horizon errors, surface biases, and reward hacking are involved.

Latest papers

CardsList

46 papers9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

  1. Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring

    May 1, 2026Indraneil Paul, Goran Glavaš, Iryna GurevychReward ModelingReward Model Evaluation

    UKP Lab, TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE · Center for Artificial Intelligence and Data Science, University of Würzburg

  2. From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks

    Apr 30, 2026Qingyu Ren, Tianjun Pan, Xingzhou Chen +1Reward ModelingRL for Language Models

    Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University · 2Shanghai Artificial Intelligence Laboratory

  3. When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient

    Apr 28, 2026Shuning Shang, Hubert Strauss, Stanley Wei +2Reward ModelingPolicy Gradient

    Princeton Language and Intelligence, Princeton University

  4. On Cost-Effective LLM-as-a-Judge Improvement Techniques

    Apr 15, 2026Ryan Lail, Luke MarkhamLLM-as-a-JudgeLLM Evaluation

    Composo AI.

  5. You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass

    Apr 13, 2026Yinuo Yang, Zixian Ma, Manasi Ganti +2Reward ModelingMultimodal Reward Modeling

    Paul G. Allen School of Computer Science & Engineering University of Washington Seattle, WA 98195, USA

  6. Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling

    Apr 9, 2026Jiaxuan Wang, Yulan Hu, Wenjin Yang +3Reward Model EvaluationAI Agent Benchmarks

    State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · AMAP, Alibaba Group

  7. Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization

    Apr 8, 2026Qiyao Ma, Dechen Gao, Rui Cai +4Personalized Reward ModelingReward Model Evaluation

    University of California, Davis

  8. Permutation-Consensus Listwise Judging for Robust Factuality Evaluation

    Mar 20, 2026Tianyi Huang, Nathan Huang, Justin Tang +2LLM-as-a-JudgeBias in LLM-as-a-Judge

    Ryquo · App-In Club · Carnegie Mellon University

  9. Long-form RewardBench: Evaluating Reward Models for Long-form Generation

    Mar 13, 2026Hui Huang, Yancheng He, Wei Liu +7Reward ModelingReward Model Evaluation

    Faculty of Computing, Harbin Institute of Technology, Harbin, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China · School of Computer Science and Technology, Harbin Institute of Technolgy, Shenzhen, China

  10. IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation

    Mar 5, 2026Bosi Wen, Yilin Niu, Cunxiang Wang +5LLM-as-a-JudgeInstruction Following

    The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University · Zhipu AI · University of Electronic Science and Technology of China

  11. Beyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Models

    Mar 2, 2026Qiyuan Zhang, Yufei Wang, Tianhe Wu +5Reward ModelingReward Model Evaluation

    City University of Hong Kong · Tencent Hunyuan · MBZUAI

  12. CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction

    Feb 28, 2026Yinghao Ma, Haiwen Xia, Hewei Gao +9Music Generation EvaluationMultimodal Reward Modeling

    Queen Mary University of London, London, UK · Peking University, Beijing, China · Technical University of Munich, Munich, Germany +5

  13. RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models

    Feb 27, 2026Daniel Yang, Samuel Stante, Florian Redhardt +5Reward ModelingReward Model Evaluation

    ETH Zurich · ETH AI Center

  14. Next-token Preference: Scaling Unsupervised Reward Modeling

    Feb 11, 2026Jingxuan Fan, Yueying Li, Zhenting Qi +4Reward ModelingPreference Learning

    Harvard University · Cornell University · Microsoft Research +1

  15. The Reward Model Selection Crisis in Personalized Alignment

    Dec 28, 2025Fady Rezk, Yuangang Pan, Chuan-Sheng Foo +4Personalized Language Model AlignmentPersonalized Reward Modeling

    School of Informatics, University of Edinburgh, UK · Agency for Science, Technology and Research (A*STAR), Singapore · Samsung AI Research Center, Cambridge, UK

  16. Rethinking Reward Models for Multi-Domain Test-Time Scaling

    Oct 1, 2025Dong Bok Lee, Seanie Lee, Sangwoo Park +12Reward ModelingLLM Evaluation

    KAIST · TH Nürnberg · Microsoft Research Asia