Reinforcement Learning

Also known as RL

Follow this topic Ask about this topic

Momentum

144 papers in the last four weeks, up 243% on the four weeks before. 1.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,138

All topics

Beyond Negative Rollouts: Positive-Only Policy Optimization with Implicit Negative Gradients
May 7, 2026Mingwei Xu, Hao FangReinforcement Learning RL for Language Model Reasoning
University of Washington, Seattle, USA
Revisiting Adam for Streaming Reinforcement Learning
May 7, 2026Florin Gogianu, Adrian Catalin Lutu, Razvan PascanuDistributional RL Reinforcement Learning
Bitdefender, Bucharest, Romania · Mila, Montreal, Canada
Sequential Design of Genetic Circuits Under Uncertainty With Reinforcement Learning
May 7, 2026Michal Kobiela, Diego A. Oyarzún, Michael U. GutmannReinforcement Learning Model-Based RL
School of Informatics, The University of Edinburgh, UK · School of Informatics, School of Biological Sciences, The University of Edinburgh, UK · School of Informatics, The University of Edinburgh,, UK
Market-Alignment Risk in Pricing Agents: Trace Diagnostics and Trace-Prior RL under Hidden Competitor State
May 7, 2026Peiying Zhu, Sidi ChangReward Hacking Reinforcement Learning
Blossom AI · Blossom AI Labs
On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR
May 7, 2026Hao Ye, Jisheng Dang, Junfeng Fang +7Reinforcement Learning Reinforcement Learning with Verifiable Rewards
Lanzhou University · National University of Singapore · Beijing Institute of Technology
Operator-Guided Invariance Learning for Continuous Reinforcement Learning
May 7, 2026Zuyuan Zhang, Fei Xu Yu, Tian LanReinforcement Learning Continuous-Time RL
The George Washington University
The Causally Emergent Alignment Hypothesis: Causal Emergence Aligns with and Predicts Final Reward in Reinforcement Learning Agents
May 7, 2026Federico Pigozzi, Michael LevinReinforcement Learning
Allen Discovery Center, Tufts University, USA · Wyss Institute for Biologically Inspired Engineering, Harvard University, USA
Beyond the Independence Assumption: Finite-Sample Guarantees for Deep Q-Learning under $τ$ -Mixing
May 7, 2026Leon Halgryn, Sophie Langer, Janusz M. Meylahn +1Reinforcement Learning Deep Q-Learning
University of Twente · Department of Applied Mathematics University of Twente · Ruhr-Universität Bochum +1
On Training in Imagination
May 7, 2026Nadav Timor, Ravid Shwartz-Ziv, Micah Goldblum +2Reward Modeling Reinforcement Learning
Weizmann Institute of Science · New York University · Columbia University
AdaGamma: State-Dependent Discounting for Temporal Adaptation in Reinforcement Learning
May 7, 2026Yaomin Wang, Jianting Pan, Ran Tian +4Reinforcement Learning Temporal-Difference Learning
School of Data Science, The Chinese University of Hong Kong, Shenzhen
Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization
May 7, 2026Alireza Modirshanechi, Benjamin Eysenbach, Peter Dayan +1Reinforcement Learning Agent Skill Learning
Helmholtz Munich, Germany · Max Planck Institute for Biological Cybernetics, Germany · Princeton University, USA +1
Schedule-and-Calibrate: Utility-Guided Multi-Task Reinforcement Learning for Code LLMs
May 7, 2026Yujia Chen, Yang Ye, Xiao Chu +2RL for Code Generation Reinforcement Learning
Harbin Institute of Technology, Shenzhen · Huawei Cloud Computing Technologies Co., Ltd.
Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer
May 7, 2026Yongyi Wang, Hanyu Liu, Lingfeng Li +6Efficient Transformer Inference Reinforcement Learning
School of Computer Science Peking University Beijing, China 100871
Causal Reinforcement Learning for Complex Card Games: A Magic The Gathering Benchmark
May 7, 2026Cristiano da Costa Cunha, Ajmal Mian, Tim French +1RL Benchmarks Structural Causal Models
Department of Computer Science and Software Engineering, University of Western Australia, 6009 WA Australia
Measuring Learning Progress via Gradient-Momentum Coupling
May 7, 2026Samuel Blad, Martin Längkvist, Amy LoutfiReinforcement Learning Intrinsic Motivation
Örebro University
Selective Rollout: Mid-Trajectory Termination for Multi-Sample Agent RL
May 7, 2026Zhiyuan Zhai, Xin WangReinforcement Learning Group Relative Policy Optimization
Fudan University
Reward Shaping and Action Masking for Compositional Tasks using Behavior Trees and LLMs
May 7, 2026Nicholas Potteiger, Ankita Samaddar, Taylor T. Johnson +1Reinforcement Learning Reward Shaping
Vanderbilt University, Nashville, TN
A Measure-Theoretic Finite-Sample Theory for Adaptive-Data Fitted Q-Iteration
May 7, 2026Manuel Haussmann, Mustafa Mert Çelikok, Melih KandemirReinforcement Learning Q-Learning
Department of Mathematics and Computer Science University of Southern Denmark
Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
May 7, 2026Haodong Liang, Lifeng LaiReinforcement Learning Transformer-Based RL
Department of Electrical and Computer Engineering, University of California, Davis
LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks
May 6, 2026Mahyar Alinejad, Yue Wang, Amrit Singh Bedi +1Reinforcement Learning LLM-Guided RL
Department of Electrical and Computer Engineering, University of Central Florida, Orlando, Florida, USA · Department of Computer Science, University of Central Florida, Orlando, Florida, USA
Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policies
May 6, 2026David Leeftink, Max Hinne, Marcel van GervenRL Control Reinforcement Learning
Department of Machine Learning and Neural Computating Donders Institute for Brain, Cognition and Behaviour Radboud University, Nijmegen, the Netherlands.
Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime
May 6, 2026Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo +8RL for Code Generation Reinforcement Learning
Baidu
LineRides: Line-Guided Reinforcement Learning for Bicycle Robot Stunts
May 6, 2026Seungeun Rho, Shamel Fahmi, Jeonghwan Kim +3Reinforcement Learning Robot Skill Learning
1RAI Institute, Cambridge, MA, USA · Georgia Institute of Technology, Atlanta, GA, USA
Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers
May 6, 2026Senkang Hu, Yong Dai, Xudong Han +4Agentic RL Reinforcement Learning
Hong Kong JC STEM Lab of Smart City · City University of Hong Kong · University of Oxford +4
Modular Reinforcement Learning For Cooperative Swarms
May 6, 2026Erel Shtossel, Gal A. KaminkaMulti-Robot Systems Reinforcement Learning
Bar Ilan University Ramat Gan, Israel
Reinforcement Learning for Compositional Generalization with Outcome-Level Optimization
May 6, 2026Xiyan Fu, Wei LiuReinforcement Learning RL for Generative Models
Nanyang Technological University · Independent Researcher
A Harmonic Mean Formulation of Average Reward Reinforcement Learning in SMDPs
May 6, 2026Erel Shtossel, Alicia Vidler, Uri Shaham +1Non-Stationary RL Reinforcement Learning
Bar Ilan University Ramat Gan, Israel
SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
May 6, 2026Lirui Luo, Guoxi Zhang, Hongming Xu +2Loss of Plasticity Reinforcement Learning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, BIGAI · Institute for Artificial Intelligence, Peking University
Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation
May 6, 2026Akiyoshi Tomihari, Issei SatoReinforcement Learning KL-Regularized RL
Department of Computer Science, The University of Tokyo
CRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies
May 6, 2026Keyu Chen, Nanfei Ye, Yida Wang +4Reinforcement Learning Autonomous Driving Planning
School of Vehicle and Mobility, Tsinghua University · Li Auto Inc

May 7, 2026, cs.CLJ/K move · Enter open · S save

Beyond Negative Rollouts: Positive-Only Policy Optimization with Implicit Negative Gradients

Mingwei Xu, Hao Fang

University of Washington, Seattle, USA

Reinforcement Learning RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Policy Gradient Methods

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR), due to the deterministic verification, becomes a dominant paradigm for enhancing the reasoning ability of large language models (LLMs). The community witnesses the rapid change from the Proximal Policy Optimization (PPO) to Group Relative Policy Optimization (GRPO), in which GRPO reduces the complicated advantage estimation with simple estimation over grouped positive and negative rollouts. However, we note that negative rollouts may admit no gradation of failure severity, and the combinatorial vastness makes penalizing a few sampled negatives unlikely to cover a meaningful reward signal under sparse binary rewards. In this work, we propose Positive-Only Policy Optimization (POPO), a novel RLVR framework in which learning can occur exclusively via online positive rollouts. Specifically, POPO utilizes bounded importance sampling over the positive rollout set. Thus, no disjoint negative rollouts are used for the gradient guidance. We show that implicit negative gradients can emerge naturally through reinforcing the positive probability via rollouts redistribution. Next, POPO stabilizes the policy optimization through two mechanisms. First, it applies a siamese policy network with a momentum-based adaptation law for stabilized policy evolution. Second, we replace the KL-divergence with a bounded similarity penalty term in the siamese representation space. We conduct extensive experiments using publicly available, well-established text-LLM models, e.g., the Qwen family, across all-level mathematical benchmarks. Our experiment demonstrates that POPO achieves performance comparable to, or even superior to GRPO. Notably, we show that POPO can achieve 36.67% in AIME 2025 with Qwen-Math-7B, outperforming GRPO 30.00%. Our ablation and sweep studies further illustrate the necessity and robustness of POPO components.

Momentum

Related topics

Latest papers 1,138