Reward Decomposition

Follow this topic Ask about this topic

Latest papers

11 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Intra-Rubric Reward Divergence: Mitigating Rubric Hacking through Latent Dimension Recovery and Balancing
Oct 19, 2026Jianqiang Xia, Han Xu, Dianxi ShiReward Hacking Rubric-Based RL
Shanghai Jiao Tong University · Baidu, Inc. · Intelligent Game and Decision Lab
Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards
Oct 14, 2026Tianyu Liu, Davide Belli, Bence Major +2Reward Decomposition Long-Horizon LLM Agents
ETH Zurich · Qualcomm AI Research‡
Reinforcement Learning with Decomposed Subtasks
Sep 22, 2026Mattie Terzolo, Mikolaj Sacha, Ayan Sinha +1Group Relative Policy Optimization Credit Assignment in RL
Upwork
CoRe: Combined Rewards with Vision-Language Model Feedback for Preference-Aligned Reinforcement Learning
Jul 2, 2026Hexian Ni, Tao Lu, Yinghao CaiRobotic RL Multimodal Reward Modeling
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.
REAR: Test-time Preference Realignment through Reward Decomposition
Jun 29, 2026Fuxiang Zhang, Pengcheng Wang, Chenran Li +6Test-Time Scaling Reward Decomposition
1Nanyang Technological University · University of Califor · 3Nanjing University
LC-ERD: Mining Latent Logic for Self-Evolving Reasoning via Consistency-Regulated Reward Decomposition
May 19, 2026Yanyu Chen, Jiyue Jiang, Dianzhi Yu +8RL for Language Model Reasoning Process Supervision
The Chinese University of Hong Kong · Shanghai Jiaotong University · Fudan University
ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization
May 12, 2026Nirmal Patel, Fei Wang, Inderjit S. DhillonRL for Language Models Reinforcement Learning
University of Texas at Austin · Google
Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization
Apr 21, 2026Carter Adams, Rafael Oliveira, Gabriel Almeida +1Tool Use in VLMs Reinforcement Learning
Federal University of Bahia
CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation
Apr 13, 2026Yunfan Yang, Cuiling Lan, Jitao Sang +1Reward Decomposition Structured Output Generation
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · Microsoft Research Asia · Peng Cheng Laboratory
Pressure, What Pressure? Sycophancy Disentanglement in Language Models via Reward Decomposition
Apr 7, 2026Muhammad Ahmed Mohsin, Ahsan Bilal, Muhammad Umer +1Reward Decomposition LLM Sycophancy
University of Oklahoma
ToolRLA: Multiplicative Reward Decomposition for Tool-Integrated Agents
Mar 2, 2026Pengbo LiuReward Decomposition Tool-Using Agents
Unaffiliated

Oct 19, 2026J/K move · Enter open · S save

Intra-Rubric Reward Divergence: Mitigating Rubric Hacking through Latent Dimension Recovery and Balancing

Jianqiang Xia, Han Xu, Dianxi Shi

Shanghai Jiao Tong University · Baidu, Inc. · Intelligent Game and Decision Lab

Reward Hacking Rubric-Based RL Reward Design for RL Reward Decomposition Multi-Objective Reinforcement Learning

Open paper AskPDF arXiv

Rubric hacking limits both performance and training robustness in rubric-based reinforcement learning. By decomposing rubrics into sub-dimensions, we reveal the Intra-Rubric Reward Divergence: as the total rubric score increases, certain sub-dimension scores systematically decrease, suggesting the model sacrifices specific quality aspects to optimize the aggregated objective. From the sub-dimension perspective, we observe two phenomena: (1) dimension imbalance, causing the policy to favor heavily covered dimensions; (2) dimension absence, leading to a systematic gap between rewards and true quality. To mitigate these issues, we propose Fair-Race Dimension Aggregation (FRDA), which has generations compete within each dimension separately to mitigate rubric imbalance, and Residual Semantic Dimension Completion (RSDC), which mines evaluation dimensions implicitly encoded in a holistic preference signal to complete rubric coverage. Experiments in chat, science, and medical domains indicate that FRDA mitigates rubric hacking with more stable improvements, while RSDC outperforms both single-reward training and naive multi-reward aggregation, yielding generally higher-quality responses.