Policy Learning

Follow this topic Ask about this topic

Momentum

79 papers in the last four weeks, up 316% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 416

All topics

Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy
Sep 23, 2026Mehmet Turan Yardımcı, Yunus Emre ÇoğurcuRobot Policy Learning RL Exploration
Karlsruhe Institute of Technology · Çukurova University
Reinforcement Learning with Decomposed Subtasks
Sep 22, 2026Mattie Terzolo, Mikolaj Sacha, Ayan Sinha +1Group Relative Policy Optimization Credit Assignment in RL
Upwork
PACT: From Credit Assignment to Critic Alignment
Sep 22, 2026Jiayan Fu, Hang Xu, Yong Zhang +4Reinforcement Learning Token-Level Credit Assignment
AllSpark Team
MimicAgent: Quadruped Skills via Prompt-to-Trajectory Generation
Sep 21, 2026Lucky Kant Nayak, Narayanan Palghat Parameswaran, Neehar Peri +1Reinforcement Learning Trajectory Generation
Carnegie Mellon University
SeeQ: Training Generalist Value Functions for Long-Horizon Robotic Manipulation
Sep 18, 2026Saksham Singh, Zheyuan Hu, Max Sobol Mark +3Robot Policy Learning Long-Horizon Robotic Manipulation
Carnegie Mellon University
Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervision
Sep 17, 2026Nitish Dashora, Douglas Chen, Idan Shenfeld +3Robot Policy Learning Long-Horizon Robotic Manipulation
Massachusetts Institute of Technology · Carnegie Mellon University
Accelerating Visual Policy Learning with Sampling-Based Model Predictive Control
Sep 17, 2026Yilang Liu, Haoxiang You, Qian Wang +2Policy Optimization Visuomotor Policy Learning
Department of Mechanical Engineering, Yale University, New Haven, CT 06520, USA · Department of Computer Science, Yale University, New Haven, CT 06520, USA · School of Electrical and Computer Engineering, University of Sydney, Sydney, NSW 2006, Australia
MATCH: Model-Aware Tool Learning with Curriculum Scheduling and Hierarchically Gated Rewards
Sep 17, 2026Shihao Liu, Hao Yin, Lijun Liu +3Reward Modeling Curriculum RL
School of Cyber Security, University of the Chinese Academy of Sciences · Honor Device Co., Ltd · University of the Chinese Academy of Sciences
EPIG-Tree: Compute-Optimal Branching for Gradient-Efficient Reinforcement Learning
Sep 17, 2026Nikita Khomich, Leopold Hermansson, Ido HakimiRL for Language Models Policy Gradient
Learning and Transferring Closed-Loop Robot Software
Sep 17, 2026So Kuroki, Yujin TangRobot Policy Learning Robot Policy Adaptation
Sakana AI
Dual-Axis Policy Optimization for LLM Agents: Bayesian Feedback Attribution and Trajectory Mass Normalization
Sep 17, 2026Yingxuan Zhuang, Binhe Yu, Jingxiao Yang +6Group Relative Policy Optimization Policy Optimization
Zhejiang University · University of Science and Technology of China · University of New South Wales
Stable Policy Learning
Sep 16, 2026Harvey Barnhard, Giacomo Opocher, Rahul SinghAlgorithmic Stability Policy Learning
Harvard University · University of Mannheim
A Convergence Framework for Deep $V$ -Learning: Error Propagation and Sharp Action-Gap Bounds
Sep 16, 2026Yury KolomeytsevReinforcement Learning Policy Learning
Faculty of Computational Mathematics and Cybernetics Lomonosov Moscow State University
Bellman Policy Optimization
Sep 14, 2026Zhuoqing Song, Haotian Xu, Xikun Zhang +1Reinforcement Learning RL for Language Model Reasoning
1Apodex US, Inc. · 2Princeton University
MInTRL: Off-policy Intervention can boost On-policy RL
Sep 14, 2026Mingyu Chen, Yefan Tao, Gerald Friedland +2Reinforcement Learning Reinforcement Learning with Verifiable Rewards
Amazon Web Services · Boston University
Adaptive Agent Design
Sep 14, 2026Raj Kiriti Velicheti, Subhonmesh Bose, Tamer BaşarReinforcement Learning Q-Learning
Department of Electrical and Computer Engineering and the Coordinated Science Laboratory at the University of Illinois, Urbana-Champaign, Urbana, IL 61801
SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropy
Sep 14, 2026Xiefeng Wu, Shu Zhang, Zhaojie Chu +1Robotic RL Offline RL
School of Computer Science, Wuhan University, Wuhan, China · School of Electronic Information, Wuhan University, Wuhan, China · School of Internet, Anhui University, Hefei, China
Robust Policy Optimization via Adversarial Importance Sampling
Sep 14, 2026Amine Andam, Jamal Bentahar, Mustapha HedabouRobust RL Policy Learning
Mohammed VI Polytechnic University · Khalifa University · Concordia University
A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning
Sep 14, 2026Amine Andam, Jamal Bentahar, Mustapha HedabouKL-Regularized RL Adversarial Robustness
Mohammed VI Polytechnic University · Khalifa University · Concordia University
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Sep 12, 2026Junyao Yang, Yucheng Shi, Zhongzhi Li +4Terminal Agents Long-Horizon Agent Evaluation
Tencent Hy Foundation Model · National University of Singapore · University of Georgia +2
Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning
Sep 11, 2026Taoran Liang, Yang Liu, Shang Luo +10Credit Assignment in RL Step-Level Credit Assignment in RL
Nankai University · Peking University · Supply Chain Tech Team Y, JD.com +2
IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies
Sep 11, 2026Kian Hosseinkhani (Simon Fraser University), Qinhe Peng (University of Pennsylvania), George Shramko (Simon Fraser University) +7Efficient VLA Model Inference Vision-Language-Action Models
Simon Fraser University · University of Pennsylvania · Alberta Machine Intelligence Institute (Amii) +1
DIA: Denoising Intermediate Advantage for Diffusion Policy Optimization
Sep 10, 2026Arjun Sohal, Yuchi Zhao, Miroslav Bogdanovic +1RL for Robotics Step-Level Credit Assignment in RL
University of Toronto, Toronto, ON, Canada · Vector Institute for Artificial Intelligence, Toronto, ON, Canada · Acceleration Consortium, University of Toronto, Toronto, ON, Canada +2
Proxy Policy Steering
Sep 8, 2026Chuanruo Ning, Tianrui Wang, Wei-Chiu Ma +1Robot Policy Adaptation Robot Foundation Models
Cornell University
Mini-Batch Risk-Averse Deep Q-Learning: A Robot Navigation Case Study
Sep 7, 2026Aayush Patel, Andrzej RuszczyńskiDeep Q-Learning Robot Navigation
Department of Management Science and Information Systems, Rutgers University
DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training
Sep 3, 2026Shubham Gandhi, Saurabh Goyal, Kiran Kate +1Agentic RL Step-Level Credit Assignment in RL
1Carnegie Mellon University · 2IBM Research
Spurious Advantage Hidden in GRPO
Sep 3, 2026Jiamian Wang, Samyadeep Basu, Koustava Goswami +2Reinforcement Learning Group Relative Policy Optimization
Rochester Institute of Technology · Adobe Research
Headroom-Drift Replay: A Primitive for Principled Replay Control in GRPO
Sep 3, 2026Hyun Bin Park, Du-Seong ChangRL for Language Model Reasoning Group Relative Policy Optimization
Department of Artificial Intelligence, Sogang University, Seoul, Republic of Korea
Multi-step Proximal Policy Improvement in Offline Reinforcement Learning
Sep 3, 2026Soohyun Choi, Seonvin Cho, Songnam HongOffline RL Policy Learning
Department of Electronic Engineering, Hanyang University, Seoul, Republic of Korea
Tail-Likelihood Reinforcement Learning
Sep 2, 2026Shrinivas Ramasubramanian, Daman Arora, Fahim Tajwar +11Risk-Sensitive RL RL for Generative Models
Carnegie Mellon University · Together AI · University of California, Berkeley +2

Sep 23, 2026, cs.ROJ/K move · Enter open · S save

Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy

Mehmet Turan Yardımcı, Yunus Emre Çoğurcu

Karlsruhe Institute of Technology · Çukurova University

Robot Policy Learning RL Exploration Vision-Language-Action Models Multi-Task RL RL Fine-Tuning Policy Learning

Open paper AskPDF arXiv

Online reinforcement learning fine-tuning of pretrained flow-matching vision-language-action (VLA) policies promises robots that keep learning after deployment, but continued updates often destroy competence on individual tasks while the aggregate still looks healthy. We study this failure mode, which we call task collapse, under a matched small-compute budget on LIBERO-10 with a 450M-parameter SmolVLA policy trained by PPO with stochastic (SDE) sampling. Three exploration-noise policies differ in one live variable: a fixed noise scale, a ReinFlow-style learned noise network, and an uncertainty-gated controller that redistributes exploration across task streams from task-agnostic novelty and competence signals, without task labels or episode boundaries. Under the pooled definition, fixed noise collapses tasks in two of three seeds and learned noise in every seed measured to iteration 200, while the controller collapses none in any of its three seeds. Measured parameter displacement shows the controller's action expert keeps changing, while its mean applied noise is close to the fixed scale in the available logs. The matched comparison supports the controller's effect on task preservation; the separate contributions of its adaptation across states and over time are not disentangled. A lower fixed scale slows the decline but does not stop it. No arm improves on the behavior-cloning baseline in this budget. Two properties of that regime are measured beside this result, not offered as its cause: following the reference recipe, training runs in bfloat16 with no fp32 master copy, under which 96.02% of the action expert's elements stay bit-identical across three consecutive iterations, and an fp32 master copy at the reference learning rate collapses both arms in a single-seed observation. We release tools measuring per-task collapse under four definitions, rescoring noise and instrument tares.

Momentum

Related topics

Latest papers 416