Reward Design for RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 167

All topics

Beyond Uncertainty: Multi-Solver Disagreement Rewards for Self-Evolving Reasoning Curricula
Aug 30, 2026Vinoth Selvendran, Zhanming ZhangReward Modeling RL for Language Model Reasoning
Independent Researcher · Palo Alto, CA, USA · New York, NY, USA
An Empirical Study of Reward Specification and Benchmark Reliability in GRPO-based LLM Unlearning
Aug 18, 2026Rubén Balbastre, Juan Manuel Orduña, Mariano PérezGroup Relative Policy Optimization Rubric-Based RL
University of Valencia
Towards Socially Compliant Navigation in Deep Reinforcement Learning via Proxemics-Based Reward Modeling
Aug 13, 2026Takieddine Soualhi, Jacques Saraydaryan, Laetitia MatignonRL for Robotics Social Robot Navigation
CHROMA · Inria, CITI Lab, INRIA–INSA Chroma Team, Villeurbanne, France. · CPE, CHROMA +3
A Framework for Designing Reward Functions: From Objectives to Features to Human-Aligned Reward Functions
Aug 12, 2026Di Yang Shi, W. Bradley KnoxReward Modeling Reward Design for RL
University of Texas at Austin
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Aug 12, 2026Minglai Yang, Xinyu Guo, Utkarsh Tyagi +6Reward Hacking Rubric-Based RL
Scale AI · University of Arizona · University of Texas at Dallas
When Offline Evaluation Misleads: A Diagnostic Protocol for Reward and Policy Selection in Delayed-Feedback Contextual Bandits
Aug 12, 2026Sang Su Lee, Vineeth Loganathan, Shishir Dash +1Multi-Armed Bandits Off-Policy Evaluation
Thumbtack, Inc. San Francisco, CA, USA
CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learning
Aug 10, 2026Ambuj Mehrish, Sebastiano VasconReward Modeling RL for Language Models
CVML Lab Ca’ Foscari University of Venice
RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning
Aug 10, 2026Jinkun Hou, Zhuo Liu, Huimin Ren +3Open-Ended Generation RL for Language Models
1Peking University · 2Beijing Institute of Technology · 3Li Auto Inc.
MedCalc-R1: Knowledge-Guided Reward Framework for Medical Mathematical Reasoning
Aug 9, 2026Haotian Wang, Lian Yan, Xingzhi Yao +4Numerical Reasoning in Language Models Reinforcement Learning with Verifiable Rewards
Harbin Institute of Technology
Multi-Agent Reinforcement Learning via Agent-Specific Preference
Aug 9, 2026Ni Mu, Yao Luan, Yiqin Yang +1Decentralized MARL Cooperative MARL
CFINS, Department of Automation, BNRist, Beijing Key Laboratory of Embodied Intelligence Systems, Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing 100084, China · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
A Unified Framework for Dynamic Reward Shaping in Reinforcement Learning
Aug 8, 2026Fouad BahrpeymaReward Shaping Reward Design for RL
Industrial AI Group Professorship for Information Management Faculty of Informatics / Mathematics Dresden University of Applied Sciences Dresden, Germany
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
Aug 6, 2026Chenglong Wang, Ziming Zhu, Yifu Huo +9Reward Modeling RL for Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang, China · 2NiuTrans Research, Shenyang, China · 3Independent Researcher, Beijing, China +1
Training a Conditioned Video Game Agent on a VLM Annotated Dataset
Aug 6, 2026Katrin Schmid, Iuri FrosioReinforcement Learning Reward Design for RL
NVIDIA, Australia · NVIDIA, Italy
Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning
Aug 5, 2026Jai Malegaonkar, Rohan Patil, Henrik I. ChristensenReinforcement Learning Partially Observable RL
Department of Computer Science and Engineering, UC San Diego, United States
GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation
Aug 4, 2026Andrea Protopapa, Davide Buoso, Francesca Pistilli +2Long-Horizon Robotic Manipulation Learning from Demonstration
Politecnico di Torino, Turin, Italy. · Interactive Robot Perception & Learning (PEARL) Lab, Computer Science Dept., TU Darmstadt, Germany. · Hessian.AI, Darmstadt, Germany. +1
DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards
Aug 1, 2026Yunhao Wang, Binghong Wu, Zhenyu Huang +4Reward Shaping Document Parsing
Tencent Hunyuan · Beijing, China · Shanghai, China +1
Stabilized Best-of- $K$ Training for Neural Combinatorial Optimization
Jul 31, 2026Melveena Jolly, Midhun XavierTraveling Salesperson Problem Best-of-N Sampling
1Independent Researcher
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
Jul 31, 2026Manith Adikari, Bei Peng, Samuele Vinanzi +1Multi-Objective Reinforcement Learning Reinforcement Learning
Department of Computer Science, University of Manchester · Centre for Robotics & AI, University of Manchester · School of Computer Science, University of Sheffield +1
Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning
Jul 30, 2026Efstratios Zaradoukas, Davide Gabrielli, Bardh Prenkaj +1Reinforcement Learning with Verifiable Rewards Machine Unlearning
Technical University of Munich, Germany · Munich Center for Machine Learning, Germany · Sapienza University of Rome, Italy
Reinforcement Learning for Code Optimization
Jul 28, 2026Pierre Chambon, Kunhao Zheng, Juliette Decugis +2RL for Code Generation Reinforcement Learning
FAIR at Meta · Inria · Université Paris Dauphine
Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback
Jul 28, 2026Yunpeng ChuLLM Alignment Reward Shaping
Stony Brook University
Optimal Reward Shaping: Autonomous Car Parking Case Study
Jul 26, 2026Emre Özkaya, Nicolas R. GaugerDeep Q-Learning Autonomous Parking
Chair for Scientific Computing, RPTU University of Kaiserslautern-Landau Paul-Ehrlich-Strasse 34, 67663 Kaiserslautern
Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards
Jul 24, 2026Cheng Ruoxi, Ma Haoxuan, Zhang Hongyi +7Retrieval-Augmented Generation LLM Grounding
1Fudan University · 2Tencent · 3Nanjing University +2
The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works
Jul 23, 2026Yu WangReward Hacking Reward Shaping
Independent researcher
Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation
Jul 23, 2026Quazi Ishtiaque Mahmud, Nesreen K. Ahmed, Ali JannesariRL for Code Generation GPU Kernel Optimization
Iowa State University, Ames, IA, USA · Cisco AI Research, San Jose, CA, USA
CriPO: Enhancing Rubric-based RL via Self-Distillation
Jul 20, 2026Mingxuan Xia, Yuhang Yang, Chao Ye +7Credit Assignment in RL Rubric-Based RL
Zhejiang University · ByteDance
CORAL: Learning Amyloid Fibril Ligand Docking with Cooperative Binding Rewards
Jul 19, 2026Yasheng Sun, Bohan Li, Youqi Tao +1Reward Design for RL Drug Discovery
Center of Excellence for Generative AI, KAUST · MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University · Stern Laboratory, Brigham and Women’s Hospital, Harvard University
Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimators
Jul 18, 2026Alireza Furutanpey, Schahram DustdarFree Energy Calculation Intrinsic Reward Methods
Coovally, Barcelona, Spain · Distributed Systems Group, TU Vienna, Austria · ICREA Barcelona, Spain
CLaC@FinMMEval 2026 Task 3: Sentiment-Augmented Deep Reinforcement Learning for Active Trading -- An Alpha-Reward Approach
Jul 17, 2026Andrei Neagu, Eeham Khan, Leila KosseimReinforcement Learning Deep Q-Learning
Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum
Jul 15, 2026Slava AndrejevRL Control Reinforcement Learning

Aug 30, 2026, cs.AIJ/K move · Enter open · S save

Beyond Uncertainty: Multi-Solver Disagreement Rewards for Self-Evolving Reasoning Curricula

Vinoth Selvendran, Zhanming Zhang

Independent Researcher · Palo Alto, CA, USA · New York, NY, USA

Reward Modeling RL for Language Model Reasoning Curriculum RL Reward Design for RL Algorithmic Reasoning Self-Evolving Agents

Open paper AskPDF arXiv

Self-evolving reasoning frameworks train a Challenger to generate questions exposing a Solver's weaknesses, creating adaptive curricula without human data. However, existing approaches use a single solver's sampling uncertainty as the Challenger's reward. This creates a fundamental bottleneck: as the solver grows confident on the Challenger's question distribution, all sampled answers converge identically, collapsing the reward to zero and starving the Challenger of learning signal. Critically, this single-model reward cannot distinguish genuinely easy questions from those that merely align with one solver's learned biases. We propose a multi-solver disagreement reward using a heterogeneous ensemble varying in model capacity and sampling temperature. A normalized Shannon entropy over the ensemble's per-question plurality answers explicitly rewards questions where solvers produce conflicting solutions---capturing difficulty as inter-model divergence rather than intra-model sampling variance. This richer gradient enables the Challenger to discover questions targeting true capability boundaries, producing a curriculum that forces downstream Solvers to develop robust reasoning strategies generalizing across problem types. Our approach is a drop-in reward function replacement requiring no framework modifications or additional data. Experiments with Qwen3-4B show that Solvers trained on disagreement-Challenger questions achieve +1.34 points average improvement on competition-math benchmarks (MATH-500, AMC, Olympiad), suggesting that multi-solver disagreement provides a complementary and scalable signal for curriculum generation in self-play reasoning systems.

Momentum

Related topics

Latest papers 167