Reward Modeling

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 118% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 194

All topics

Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers
Oct 6, 2026Naoki Nishikawa, Taiji SuzukiReward Modeling Transformer-Based RL
The University of Tokyo, RIKEN AIP
Revisiting Numerical Forecasting Models for Language-Based Trajectory Prediction
Oct 6, 2026JunGyu Lee, Inhwan Bae, Hae-Gon JeonReward Modeling RL for Language Models
Yonsei University · DGIST
Ontology Concept Overlap as a Training Signal: Knowledge-Grounded Reinforcement Learning for Clinical Question Answering
Oct 5, 2026Aditya Tanna, Abhishek JindalReward Modeling RL for Language Model Reasoning
Dhirubhai Ambani University Gandhinagar, Gujarat, India
EnGRICH: Enhancing Generative Reward Modeling with Critiques from Humans
Oct 4, 2026Xuancheng Li, Beining Wang, Haitao Li +7Reward Modeling Process Reward Models
Department of Computer Science and Technology, Tsinghua University, Beijing, China · Tencent, Beijing, China
Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals
Oct 1, 2026Ryunyi Lee, Kangjun Noh, Somin Kim +2Reward Modeling LLM-as-a-Judge
Yonsei University
MatrixReward: Reward from Rubric Matrix for Open-Ended Generation
Sep 30, 2026Zihan Shen, Qi Liu, Zixuan Yang +4Open-Ended Generation Reward Modeling
Zhejiang University · Qwen Business Unit of Alibaba · Renmin University of China
Revisiting scaling laws for reward optimization
Sep 29, 2026Ali Aouad, Aymane El Gadarri, Vivek F. FariasReward Modeling Reinforcement Learning
MIT
Think Before You Score: Thinking Reward Model for Visual Generation
Sep 29, 2026Xuehai Bai, Zhenchen Tang, Yang Shi +9Reward Modeling Multimodal Reward Modeling
HDU · SenseTime · CASIA +5
RankBuffer: Efficient Ranking-Based Rewards for Open-Ended Generation
Sep 29, 2026Zixuan Yang, Yiqun Chen, Qi Liu +6Open-Ended Generation Reward Modeling
Renmin University of China · University of Southern California · Xiaohongshu Inc.
CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models
Sep 28, 2026Zhaolong Su, Yujin Han, Feng Wang +3Diffusion Model Alignment Reward Modeling
Cornell University · The University of Hong Kong · Johns Hopkins University
Rubric Rewards from Item Response Theory
Sep 28, 2026Milad Yazdani, Yaser Souri, Xiren Zhou +4Reward Modeling Rubric-Based RL
Department of Electrical and Computer Engineering, University of British Columbia · Microsoft · School of Biomedical Engineering, University of British Columbia
Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling
Sep 28, 2026Bohao Wang, Xiaoyan Zhao, Yang Zhang +4Pairwise Preference Learning Reward Modeling
Zhejiang University · National University of Singapore
From Soft Targets to Reward Signals: How Assignment and Reward Objectives Interact
Sep 28, 2026Jiangtao Lin, Bangyang Wei, Siyi Liu +2Pairwise Preference Learning Reward Modeling
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · Tencent Holdings Limited +1
CRISP: Cultural Reward Modeling for Implicit Situated Propriety
Sep 28, 2026Zekun Yuan, Yangfan Ye, Baohang Li +6Reward Modeling RL for Language Models
Harbin Institute of Technology · Huawei Technologies Co., Ltd · Peng Cheng Laboratory
Evolving Support Priorities in Empathetic Reinforcement Learning
Sep 28, 2026Pengyu Huang, Zhiyuan Han, Wenwen Tong +6Reward Modeling RL for Language Models
University of Science and Technology of China, Hefei, China · SenseTime Research, Shanghai, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
RewardExplainer: Learning Reward Model Explanations from Counterfactual Preference Feedback
Sep 27, 2026Jingyi He, Nier Wu, Shuang Liu +3Reward Modeling Counterfactual Explanations
The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University · Carnegie Mellon University +2
Diffusion Reward Models
Sep 27, 2026Xiangyang Wang, Bingxiang He, Zeyuan Liu +13Reward Modeling Multimodal Reward Modeling
Tsinghua University · University of Illinois Urbana-Champaign
A Cheap Verifier is Good Enough: LLM Post-training is Robust to Erroneous Rewards
Sep 27, 2026Andreas Plesner, Curtis Northcutt, Francisco Guzmán +1Reward Modeling LLM Reliability
ETH Zurich · Handshake AI
FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model
Sep 20, 2026Jingxuan Xu, Gang Wu, Yanan Wu +13RL for Code Generation Reward Modeling
Independent Researcher · Nanjing University · Peking University +1
RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling
Sep 19, 2026Zhenchen Tang, Yang Li, Songlin Yang +6Reward Modeling Multimodal Reward Modeling
New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tencent +1
MATCH: Model-Aware Tool Learning with Curriculum Scheduling and Hierarchically Gated Rewards
Sep 17, 2026Shihao Liu, Hao Yin, Lijun Liu +3Reward Modeling Curriculum RL
School of Cyber Security, University of the Chinese Academy of Sciences · Honor Device Co., Ltd · University of the Chinese Academy of Sciences
F $^{2}$ DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflows
Sep 17, 2026Bojian Xiong, Wentao Ding, Yujing Lu +11Reward Modeling LLM Evaluation
Tianjin University · TJUNLP Lab, Tianjin University, Tianjin, China · Baidu Inc. +1
FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback
Sep 16, 2026Runjia Tan, Yuang Tu, Yujie Yan +3Reward Modeling Robot Policy Learning
Nanyang Technological University. · Guangzhou Cloudbutterfly Technology Co., Ltd.
Don't Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction
Sep 14, 2026Hayeong Ryu, Sunhee Jo, Seunguk Yu +1Reward Modeling Reference-Free Evaluation
Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
AgenticGen: Reward-Guided Agentic Video Generation for Advertising
Sep 10, 2026Xingyuan Bu, Chengru Song, Hao Zhou +9Reward Modeling Agentic RL
ByteDance · Tsinghua University · The Chinese University of Hong Kong
Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward
Sep 9, 2026Eshwar Reddy M, Sourav KarmakarReward Modeling Reward Hacking
AI Engineer, Testsigma University of San Diego · Senior AI Scientist, Intuit India
ALIGN-HOLD: Experience Alignment for Real-Time Hold Control in Large-Scale Ride-Hailing Matching at DiDi
Sep 9, 2026Zuhao Zhang, Xu Liu, Kai Wan +3RL Control Reward Modeling
Didichuxing Co. Ltd Beijing, China · Shanghai Jiao Tong University Shanghai, China
WorldReward: Reward Modeling for Camera-Conditioned World Models
Sep 3, 2026Yibin Wang, Zehan Wang, Junshu Tang +13Reward Modeling Camera-Conditioned Video Generation
Fudan University · Tencent Hunyuan · Shanghai Innovation Institute +3
Patterning in Practice: Debiasing Reward Models with Susceptibilities
Sep 1, 2026George Wang, Elizabeth Donoway, Daniel MurfetReward Modeling Debiased ML
Resolution
When Does Predictor-Based RL Align with Human Perception? A Study of Subjective Rewards in Codec-Based Speech Language Models
Aug 31, 2026Joonyong Park, Jerry LiReward Modeling RL for Language Models
Spellbrush
Beyond Uncertainty: Multi-Solver Disagreement Rewards for Self-Evolving Reasoning Curricula
Aug 30, 2026Vinoth Selvendran, Zhanming ZhangReward Modeling RL for Language Model Reasoning
Independent Researcher · Palo Alto, CA, USA · New York, NY, USA
VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation
Aug 19, 2026Yinming Huang, Shuyuan Tu, Xi Yan +6Reward Modeling Audio-Video Generation
Fudan University · Shanghai Innovation Institution · Yinwang Intelligent Technology Co., Ltd +1
A Framework for Designing Reward Functions: From Objectives to Features to Human-Aligned Reward Functions
Aug 12, 2026Di Yang Shi, W. Bradley KnoxReward Modeling Reward Design for RL
University of Texas at Austin
MuseCritic: Learning Multi-Aspect Song Rewards through Natural-Language Aesthetic Critiques
Aug 12, 2026Jiabao Zhuang, Changhao Jiang, Hanchen Wang +11Music Generation Evaluation Reward Modeling
Fudan NLP Group, Fudan University
Procedural Fairness Failures in RLHF from Preference Averaging
Aug 10, 2026M P V S Gopinadh, Karthik Kamuju, Kummari Avinash +2Reward Modeling Algorithmic Fairness
Vishnu Institute of Technology
RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance
Aug 10, 2026Dongchi Huang, Hongyin Zhang, Bohan Hou +11Reward Modeling Robot Foundation Models
DAMO Academy, Alibaba Group · Hupan Lab
Finding the Signal in the Spam: Jointly Learning Rewards and Worker Reliability from Pairwise Comparisons
Aug 10, 2026Kaustubh Shivshankar Shejole, Tanish Agarwal, Arpit Agarwal +1Pairwise Preference Learning Reward Modeling
IIT Bombay, Mumbai, India – 400076
CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learning
Aug 10, 2026Ambuj Mehrish, Sebastiano VasconReward Modeling RL for Language Models
CVML Lab Ca’ Foscari University of Venice
TrustRoboReward: Preference-Ordered Isotonic Score Editing for Multi-Paradigm Robot Reward Models
Aug 9, 2026Yidong Wang, Yan Zhan, Ziteng Feng +16Reward Modeling Pairwise Preference Evaluation
Peking University · University of Science and Technology of China · Southeast University +6
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
Aug 6, 2026Chenglong Wang, Ziming Zhu, Yifu Huo +9Reward Modeling RL for Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang, China · 2NiuTrans Research, Shenyang, China · 3Independent Researcher, Beijing, China +1
EvoHIL: Self-Evolving Reward and Flow-Matched Policy Optimization for Robust Human-in-the-Loop Reinforcement Learning
Aug 4, 2026Shuoqin Zhang, Tongtong Cheng, Xiru Gao +7Reward Modeling Robot Policy Learning
Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning
Aug 3, 2026Seongyoon Kim, Boryeong Cho, Jihwan Oh +2Pairwise Preference Learning Reward Modeling
Institute of Engineering Research, Korea University Seoul, Republic of Korea · Kim Jaechul Graduate School of AI, KAIST Seoul, Republic of Korea · Industrial Management Engineering, Korea University Seoul, Republic of Korea
Beyond Routing Weights: Faithful Response-Level Interpretation of Mixture-of-Experts Reward Models via Contribution Contrast
Jul 31, 2026Yifan Wang, Jinyi Mu, Mayank Jobanputra +4Reward Modeling Neural Network Interpretability
Saarland University · University of California San Diego · Bielefeld University +2
Learning Latent Reasoning Traces for Scalar Reward Models End-to-End
Jul 31, 2026Sanwoo Lee, Clive Bai, Hsiu-Yuan Huang +3Reward Modeling LLM Alignment
National Key Laboratory for Multimedia Information Processing, Peking University · School of Computer Science, Peking University · LLM Department, Tencent
OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
Jul 30, 2026Qiushi Sun, Kanzhi Cheng, Yian Wang +20VLM Evaluation Computer-Use Agent Benchmarks
The University of Hong Kong · Nanjing University · National University of Singapore +4
DS@GT ARC at CheckThat! 2026: LLM-Based Trace Ranking and Grouped Reward Modeling for Multilingual Numerical Claim Verification
Jul 27, 2026Sagnik Sinha, Shreyas ShresthaNumerical Reasoning in Language Models Reward Modeling
Georgia Institute of Technology, North Ave, Atlanta, GA 30332, United States
What do Reward Models Memorize?
Jul 27, 2026Ivo Verhoeven, Pushkar Mishra, Ekaterina ShutovaReward Modeling Pairwise Preference Evaluation
ILLC, University of Amsterdam, The Netherlands · Google DeepMind, London, United Kingdom
SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents
Jul 25, 2026Yang Wan, Zhenhao Zhang, Jierui Wang +1Reward Modeling LLM-as-a-Judge
1Zhejiang University · University of Rochester
CSPF: A Constrained Shared-Private Fusion Method for Non-Verifiable Preference Evaluation
Jul 23, 2026Hehao Zhang, Danli Wang, Xinyuan Wang +1Pairwise Preference Learning Reward Modeling
Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
Progress Reward Modeling for Robotic Learning: A Comprehensive Survey
Jul 22, 2026Jianshu Zhang, Keliang Wu, Haoran Lu +8Reward Modeling Process Reward Models
Northwestern University · Carnegie Mellon University · University of Wisconsin–Madison +2
How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF
Jul 22, 2026Venkata Naga Sai Vishnu Rohit Pulipaka, Anish Katta, Deva Rohit Reddy PeddireddyEfficient Neural Network Inference Reward Modeling
Reward-Driven LLM Agent Workflows: Synthesizing POMDP Routing and Self-Correction for Autonomous Decision-Making
Jul 19, 2026Amez Amanj Ali, Kuo-Kun TsengReward Modeling LLM Self-Correction
School of Computer Science and Technology Harbin Institute of Technology, Shenzhen
A Method for Learning Value Systems in Generative AI
Jul 18, 2026Andrés Holgado-Sánchez, Holger Billhardt, Sascha OssowskiReward Modeling LLM Alignment
1CETINIA, University Rey Juan Carlos
Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling
Jul 17, 2026Bo-An Chang, Yu-Chih ChenVLM Evaluation Reward Modeling
National Tsing Hua University, Hsinchu 300044, Taiwan · National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Jul 15, 2026Dingsu Wang, Filip Ryzner, Kelly He +12Reward Modeling Learning to Rank
Pinterest San Francisco, CA, USA
DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation
Jul 14, 2026Yu Fang, Wanxi Dong, Jiaqi Liu +7RL for Robotics Reward Modeling
University of North Carolina at Chapel Hill · Carnegie Mellon University · Shanghai Jiao Tong University +1
Reward Valuation in Large Language Models: Causal Induction of Anhedonia
Jul 7, 2026Melika Honarmand, Samin Mahdipour Aghabagher, Martin SchrimpfReward Modeling Cognitive Modeling
NeuroAI Laboratory, EPFL
Claim-Level Rubric Rewards for Video Caption Reinforcement Learning
Jul 6, 2026Mingqi Gao, Hongyuan Dong, Yifei Chen +6Reward Modeling Video Captioning
1Tsinghua Shenzhen International Graduate School, Tsinghua University · 3LLM Department, Tencent · University of Chinese Academy of Sciences
Attention Limited Reward Learning
Jul 6, 2026Wenqian XingPairwise Preference Learning Reward Modeling
Management Science and Engineering Department, Stanford University
Reward Lightning: Fast Video Generation via Homologous Preference Distillation
Jul 4, 2026Jiaxiang Cheng, Bing Ma, Xuhua Ren +4Diffusion Model Alignment Reward Modeling
Tencent Hunyuan, China

Oct 6, 2026, cs.LGJ/K move · Enter open · S save

Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers

Naoki Nishikawa, Taiji Suzuki

The University of Tokyo, RIKEN AIP

Reward Modeling Transformer-Based RL RL for Language Model Reasoning RL Post-Training

Open paper AskPDF arXiv

Reinforcement learning (RL) has become a standard tool for post-training language models on reasoning tasks, where the policy is updated by reward feedback while exploring the space of responses. Despite its empirical success, theoretical understanding of RL post-training remains limited, in particular of why on-policy exploration combined with a neural reward model is effective. In this paper, we address this question by modeling the reward as a hierarchical function on the response space: the reward consists of infinitely many local components, each of which becomes relevant only after the preceding ones have been resolved. We show that a natural Transformer-based actor--critic algorithm, which alternates between sampling from the current KL-regularized policy, fitting a Transformer critic to the observed rewards, and updating the policy, achieves the minimax optimal rates in the query budget and in the regularization strength up to logarithmic factors, and is minimax optimal for a fixed number of prompts. In contrast, we prove that sampling from the fixed reference distribution, as in offline reward modeling, can limit regret decay to a logarithmic rate. These results show that on-policy exploration progressively zooms in on the region where the reward is concentrated, and quantify its benefit for RL post-training.

Momentum

Related topics

Latest papers 194