RL Stability

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

42 papers9 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

SIFR: Stable Signed Flow Reinforcement
Nov 6, 2026Runlong Liao, Xingqian Xu, Lizhang Chen +7RL for Flow Matching Models RL Stability
NVIDIA · The University of Texas at Austin
Ordered Policy Optimization
Nov 6, 2026Dongkyu Derek Cho, Pan XuPolicy Optimization Proximal Policy Optimization
Department of Statistical Science Duke University · Department of Biostatistics & Bioinformatics Department of Computer Science Department of Electrical & Computer Engineering Duke University
Training Stability under Adaptive Curricula
Nov 5, 2026Johan Obando-Ceron, Walter Mayor, Michael Dennis +2Curriculum RL RL Stability
Mila – Québec AI Institute · Université de Montréal · Google DeepMind +1
On Convergence of Gradient Temporal-Difference Learning with Regularized Corrections and Eligibility Traces
Nov 4, 2026Jingyi Jin, Martha White, Hao WangTemporal-Difference Learning RL Stability
Department of Mathematical and Statistical Sciences Interdisciplinary Lab for Mathematical Ecology and Epidemiology University of Alberta Edmonton, Alberta, Canada · Department of Computing Science University of Alberta and the Alberta Machine Intelligence Institute (Amii) Edmonton, Alberta, Canada
Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning
Nov 3, 2026Kevin Frans, Alexander M Rush, Sergey Levine +2RL for LLM Agents Agentic RL
Cursor Research · UC Berkeley
ANCHOR: KL-Anchored Adaptation for Stable Offline-to-Online RL
Nov 1, 2026Geonwoo Cho, Jaegyun Im, Doyoon Kim +1Offline-To-Online RL RL Stability
University of Michigan, Ann Arbor · Gwangju Institute of Science and Technology · Pohang University of Science and Technology +1
Towards Understanding the Impact of Plasticity Loss on Reinforcement Learning in Stochastic Environments
Oct 30, 2026Philipp Bordne, André BiedenkappReinforcement Learning Loss of Plasticity
University of Freiburg · Karlsruhe Institute of Technology
On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning
Oct 29, 2026Yu Li, Yufa Zhou, Tian LanTraining-Inference Mismatch RL for Language Models
George Washington University · Duke University
Collective Dynamics of Self-Reinforcement: How Cross-Task Coupling Drives Policy Collapse in Unsupervised RLVR
Oct 27, 2026Peize DingRL for Language Models Reinforcement Learning
Columbia University, New York, NY, USA
Best Policy Tracking in Reinforcement Learning
Oct 25, 2026Judith Echevarrieta, Etor Arza, Aritz Pérez +1Reinforcement Learning RL Stability
Basque Center for Applied Mathematics
Where Rounding Happens Matters: Rounding-Location Alignment Stabilizes LLM Reinforcement Learning
Oct 23, 2026Yijing Ke, Tianle Cai, Ruosong WangMixed-Precision Training RL for Language Models
ByteDance Seed · Peking University
Dissimilarity-Based Markov Decision Processes for Stabilizing State Distributions
Oct 21, 2026Xun Shen, Yuepeng Wang, Daniyal Sajedi-Hosseini +6RL Control Markov Decision Processes
Tokyo University of Agriculture and Technology, Japan · LY Corporation, Japan · Institute of Science Tokyo, Japan +2
On the Convergence of Semi-Gradient TD: A Mean Field Perspective
Oct 14, 2026Haruki Settai, Tadashi Kozuno, Shinji ItoTemporal-Difference Learning RL Stability
The University of Tokyo · Sun Intelligence · The University of Tokyo & RIKEN
Structure-Aware Numerical Stabilization for Half-Precision Proximal Policy Optimization
Oct 9, 2026Linxuan Wu, Liang ChenRL Stability Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences School of Artificial Intelligence, University of Chinese Academy of Sciences Beijing, China
Score Centering Stabilizes Off-policy Reinforcement Learning
Sep 17, 2026Martin Marek, Max RyabininTraining-Inference Mismatch RL for Language Models
Together AI
SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropy
Sep 11, 2026Xiefeng Wu, Shu Zhang, Zhaojie Chu +1Offline-To-Online RL Conservative RL
School of Computer Science, Wuhan University, Wuhan, China · School of Electronic Information, Wuhan University, Wuhan, China · School of Internet, Anhui University, Hefei, China
A Bellman Optimality Equation for Plasticity
Sep 9, 2026Jeremy Lucas, Doina PrecupStability-Plasticity Dilemma Reinforcement Learning
Mila – Quebec Artificial Intelligence Institute Montreal, Quebec, Canada
Local and Global Stability in Performative Reinforcement Learning
Sep 6, 2026Debmalya MandalReinforcement Learning RL Stability
Department of Computer Science University of Warwick, UK
Understanding and Stabilizing Deep Q-Learning via Controlled Bootstrapping and Regulated Value Dynamics
Aug 17, 2026Bozhou Chen, Yongyi Wang, Hanyu Liu +2RL Stability Reinforcement Learning
School of Computer Science, Peking University, Beijing, China
Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum
Jul 15, 2026Slava AndrejevRL Control Reinforcement Learning
ARMOR: Stabilizing On-Policy LLM RL with Off-Policy Anchor Samples
Jul 11, 2026Kexin Huang, Junkang Wu, Jinda Lu +7Reinforcement Learning KL-Regularized RL
University of Science and Technology of China · Peking University · Dartmouth College
Trust Region Policy Distillation
Jul 6, 2026Zhengpeng Xie, Li Lyna Zhang, Zeke Xie +1On-Policy Distillation Policy Learning
Scaling Laws for Collapse in Asynchronous GRPO
Jul 1, 2026Jingwei Song, Haofeng Xu, Jie Xiao +7Asynchronous RL Reinforcement Learning
The University of Hong Kong · Shanghai Jiao Tong University · Gradient +2
Stabilization Learning: A Paradigm Transition Bridging Control Theory and Machine Learning
Jun 30, 2026Quan QuanRL Stability Adaptive Control
School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China · Zhongguancun Academy, Beijing 100094, China
Geometrically Averaged Hard Target Updates for Linear Q-Learning
Jun 9, 2026Donghwan LeeReinforcement Learning RL Stability
School of Electrical Engineering, KAIST · Daejeon, Republic of Korea
Exact Unlearning in Reinforcement Learning
Jun 2, 2026Thanh Nguyen-Tang, Raman AroraReinforcement Learning Tabular RL
Department of Data Science, New Jersey Institute of Technology, Newark, NJ, USA · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
Off-Policy Learning to Reason Works Because It Is More Pessimistic Than You Think
May 27, 2026Otmane Sakhi, Aleksei Arzhantsev, Imad Aouali +1Reinforcement Learning RL for Language Model Reasoning
Criteo AI Lab · Criteo AI Lab, CREST IP Paris
Balancing Plasticity and Stability with Fast and Slow Successor Features
May 25, 2026Raymond Chua, Doina Precup, Blake RichardsSequential Decision Making Non-Stationary RL
School of Computer Science, McGill University, Montr´eal, Canada · Mila - Quebec Artificial Intelligence Institute, Montr´eal, QC, Canada · CIFAR Learning in Machines and Brains +3
Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL
May 21, 2026Sophia Xiao Pu, Zhaotian Weng, Chengzhi Liu +4Reinforcement Learning Self-Play RL
University of California, Santa Barbara · Cisco Research
Diagnosing Training Inference Mismatch in LLM Reinforcement Learning via a Zero-Mismatch Reference
May 14, 2026Tianle Zhong, Neiwen Ling, Yifan Pi +5Training-Inference Mismatch Reinforcement Learning
ByteDance · The University of Virginia

Nov 6, 2026J/K move · Enter open · S save

SIFR: Stable Signed Flow Reinforcement

Runlong Liao, Xingqian Xu, Lizhang Chen, Baiyu Su+6

NVIDIA · The University of Texas at Austin

RL for Flow Matching Models RL Stability Regularized RL T2I Generation

Open paper AskPDF arXiv

Reward-based fine-tuning of flow models requires effectively promoting high-reward and suppressing low-reward samples, while maintaining training stability over many rounds of iterative updates. Iterative reward fine-tuning provides a simple starting point, but the positive reward weights give no explicit unfitting signal, and repeated reweighting concentrates the distribution and causes collapse. We propose Stable Signed Flow Reinforcement (SIFR), which replaces the reward weights by signed advantages to supply the missing unfitting signal, and constrains the velocity drift from a reference model to stabilize the updates, with a Lagrange multiplier that adapts the regularization strength to a prescribed drift budget. We further show that existing flow RL methods fall within this framework: they implicitly minimize the signed loss, differing in their regularization choices. We validate SIFR on SD3.5 Medium (2.5B) and FLUX.2 Klein (9B) and scale it to Cosmos 3 Super Text2Image (64B): across all 12 backbone--reward pairs with preference and rule-based rewards, it matches or exceeds existing methods, reaches their peak rewards much faster, and keeps improving over long training where they plateau or collapse.