Reinforcement Learning

Also known as RL

Follow this topic Ask about this topic

Momentum

143 papers in the last four weeks, up 233% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,147

All topics

Revisiting scaling laws for reward optimization
Sep 29, 2026Ali Aouad, Aymane El Gadarri, Vivek F. FariasReward Modeling Reinforcement Learning
MIT
MOBA-VL: Event-Localized Multi-Turn Reinforcement Learning for Real-Time MOBA Commentary
Sep 29, 2026Shengyun Zhong, Xinkang Zhao, Ziyuan Chu +1Reinforcement Learning Video-Language Models
Zhejiang University, China · Northeastern University, USA
Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL
Sep 29, 2026Mathias Jackermeier, Jacques Cloete, Alessandro AbateRL Benchmarks Reinforcement Learning
Department of Computer Science, University of Oxford · Oxford Robotics Institute, University of Oxford
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
Sep 29, 2026Hongyang Li, Xiao Li, Caesar Wu +3Reinforcement Learning Value Function Estimation
University of Luxembourg · Seafill Open-Source Community · Université Paris-Saclay
Cool the Sampler, Not the Learner: Sampling Temperature Moves the Staleness Cliff of Importance-Corrected GRPO
Sep 29, 2026Taiheng PanRL for Language Models Reinforcement Learning
The University of Melbourne
SCA: Spatial Credit Assignment for Reinforcement Learning of GUI Agents
Sep 29, 2026Shengtian Yang, Ziyu Xiong, Kaibing Yang +6Reinforcement Learning RL for GUI Agents
Southeast University, Nanjing, China · Kuaishou Technology, Beijing, China · Unaffiliated
State Trace Rationale As Auxiliary Task in Reinforcement Learning
Sep 29, 2026Muhammad U. Nasir, Alex Vogt, Steven D. James +1Reinforcement Learning Representation Learning for RL
University of the Witwatersrand Johannesburg, South Africa · New York University New York, USA
Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolving
Sep 29, 2026Yiming Wang, Yikang Liu, Qingyuan Tian +4RL for Language Models Reinforcement Learning
School of Computer Science, Shanghai Jiao Tong University · Tencent
Inducing Process Supervision from Outcome-Only Reinforcement Learning
Sep 29, 2026Shengda Fan, Xin Cong, Zhong Zhang +2Reinforcement Learning Process Reward Models
Renmin University of China · Tsinghua University · University of Electronic Science and Technology of China +1
SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learning
Sep 29, 2026Zihao Chen, Fanxiang Xiong, Hongran Ren +6Reinforcement Learning RL for Language Model Reasoning
Harbin Institute of Technology, Shenzhen · The Chinese University of Hong Kong, Shenzhen · Nanyang Technological University
BRIDGE: Bilevel Retrieval-Credit-Aware Agentic Reinforcement Learning
Sep 29, 2026Quan Xiao, Mingda Liu, Gaowen Liu +2Reinforcement Learning Agentic RAG
Department of Electrical and Computer Engineering, Cornell University · Institute of Integrated Research, Institute of Science Tokyo · Cisco Research
Massively Parallel Reinforcement Learning with a Chaotic Reconfigurable Clockless Chip
Sep 28, 2026Eric Oliveira-Gomes, Damien RontaniReinforcement Learning Chaotic Dynamical Systems
CentraleSup´elec and Universit´e de Lorraine, LMOPS UR4423 Laboratory, Metz F-57070, France
ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learning
Sep 28, 2026Nico Bohlinger, Jan PetersRL for Robotics Representation Learning
Technical University of Darmstadt, Germany · Robotics Institute Germany (RIG); German Research Center for AI (DFKI); hessian.AI
X-Reset: Scaling Object-Centric Reinforcement Learning via Cross-Embodiment Resets
Sep 28, 2026Prithwish Dan, Chenyang Ma, Wei ZhanReinforcement Learning Generalization in Robotic Manipulation
Applied Intuition · University of North Carolina at Chapel Hill · University of California, Berkeley
Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
Sep 28, 2026Hanbin Zhou, Shangzhe Li, Alexander Braverman +1Reinforcement Learning Adversarial Imitation Learning
HKUST(GZ) · UNC Chapel Hill · Cornell University
Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control
Sep 28, 2026Christian Moya, Elliott Thornley, Guang LinReward Hacking Reinforcement Learning
Purdue University · National University of Singapore
Behavioral Foundation Models for Quality Diversity
Sep 28, 2026Nazim Bendib, Nicolas Perrin-Gilbert, Olivier SigaudReinforcement Learning Quality-Diversity Optimization
Sorbonne Université, ISIR Paris, France
An analysis of Mirror-Descent Soft Actor-Critic
Sep 28, 2026Denis Zorba, Michal ValkoReinforcement Learning Online Mirror Descent
School of Mathematics, University of Edinburgh, United Kingdom · INRIA, Paris, France
Passive-Dynamic-Walking-Inspired Dynamics Guidance for Energy-Efficient Humanoid Locomotion
Sep 28, 2026Hyeonjin Choi, Joongheon Kim, Daekyum KimHumanoid Robot Locomotion Reinforcement Learning
School of Mechanical Engineering, Korea University, Seoul 02841, Republic of Korea · School of Electrical Engineering, Korea University, Seoul 02841, Republic of Korea · School of Smart Mobility, Korea University, Seoul 02841, Republic of Korea
Teach to Learn: Hint Annealing for Self-improving LLM Reasoning
Sep 28, 2026Zile Wang, Zijian Li, Haodong Wang +5Reinforcement Learning Self-Training for Language Models
The Hong Kong University of Science and Technology · Yuanbao Team, Tencent
Learning High-Risk High-Precision Motion Control
Sep 28, 2026Nam Hee Kim, Markus Kirjonen, Perttu HämäläinenRobotic Control Reinforcement Learning
Aalto University Espoo, Finland
SIPO: Selective-Inference Policy Optimization for Tree-Structured Agentic RL
Sep 28, 2026Zenghuang Fu, Ningqi Chen, Mingda Jia +10Agentic RL Reinforcement Learning
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · The University of Hong Kong +4
Unified Trajectory Matching Policy Optimization: Diverse T2I Generation and VLA Generalization
Sep 28, 2026Zhiyuan Ma, Jiaming Li, Lingzhen Li +7Reinforcement Learning Vision-Language-Action Models
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China
FlexLoop: Depth-Elastic Looped Policies for Adaptive Test-Time Computation in Deep RL
Sep 28, 2026Xun Wang, Ruishuo Chen, Yu Chen +2Efficient Neural Network Inference Reinforcement Learning
Institute for Interdisciplinary Information Sciences, Tsinghua University
Escaping Local Views: Discovering Latent Concepts for Interpretable Multi-Agent Reinforcement Learning
Sep 28, 2026Yijie Sun, Sanquan Sun, Yanda Zhu +3Reinforcement Learning Partially Observable RL
Nanjing University · Hong Kong Polytechnic University · Shenzhen University
LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization
Sep 28, 2026Shihao Zhang, Weiting Liu, Siyu Shao +4Algorithm Selection Reinforcement Learning
East China Normal University · Alibaba Group · Fudan University +4
Q-learning Penalized Transformer for Safe Offline Reinforcement Learning
Sep 28, 2026Shengchao Hu, Peng Wang, Jifeng Hu +5Reinforcement Learning Transformer-Based RL
Shanghai Jiao Tong University, China · Shenzhen Campus of Sun Yat-sen University, China · Jilin University, China +1
DexTaG: Tactile-as-Guidance in Reinforcement Learning for Dexterous Manipulation
Sep 27, 2026Han Yang, Yian Wang, Yunlong Song +2Reinforcement Learning Contact-Rich Robotic Manipulation
University of Massachusetts Amherst. · Genesis AI.
QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
Sep 27, 2026Weiqi Wang, Yuxin Zhou, Mouxiang Chen +9Reinforcement Learning Long-Horizon Agent Tasks
Alibaba Token Hub, Alibaba Group · University of Science and Technology of China · Tsinghua University
Principal Steering Subspaces for Online Adaptation of Frozen Generative Robot Policies
Sep 27, 2026Jialeng Ni, Nathan Zhao, Kunpeng SongRobot Policy Learning Reinforcement Learning
University of Michigan, Ann Arbor, MI, USA. · XPENG Robotics, Santa Clara, CA, USA.

Sep 29, 2026, cs.LGJ/K move · Enter open · S save

Revisiting scaling laws for reward optimization

Ali Aouad, Aymane El Gadarri, Vivek F. Farias

MIT

Reward Modeling Reinforcement Learning Preference-Based RL

Open paper AskPDF arXiv

Scaling laws for optimization against reward models in AI alignment have pinned down how performance depends on optimization effort---measured by a KL-divergence budget relative to a reference policy. Beyond a certain budget, over-optimization (or reward hacking) can arise: because we optimize against a proxy reward model (distinct from true rewards), performance can plateau or degrade. Naturally, the proxy reward's accuracy depends on how much preference data (often in the form of pairwise comparisons) was used to train it. However, existing research does not cleanly identify how performance jointly scales with the amount of training data and the divergence budget. Our main contribution is to provide an empirically accurate and theoretically grounded scaling law in such context. Performance roughly scales as

Θ(\sqrt{\min\{\log(M),K\}})

, where

M

is the number of comparisons in training data and

K

is the policy's divergence budget. We develop an information-theoretic model to establish this upper bound and prove it is tightly achievable through a constructive procedure. Informed by this, we conduct extensive empirical evaluations using a real-world annotation setup, whereby a large 70B gold reward model generates feedback data and proxy reward models are trained from less capable models (0.6B to 4B). Our scaling law provides an excellent fit (R2 from 97% to 99%), outperforms alternative specifications, and remains robust across model sizes, noise, and optimization procedures (best-of-

N

or policy tilting). Our evidence suggests that reward optimization is analogous to a surprisingly simple selection task: choosing from a sequence of IID Gaussian random variables using noisy preference feedback.

Momentum

Related topics

Latest papers 1,147