RL Fine-Tuning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 260% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 162

All topics

Which Rollout Taught It That? BehaviorTrace and the Limits of Training-Data Attribution in Online RL
Oct 7, 2026Amit NautiyalGradient-Based Attribution Training Data Attribution
Independent Researcher
Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization
Oct 7, 2026Haoru Li, Jinmei Liu, Zhiyong Wang +5Robot Policy Generalization RL Fine-Tuning
Nanjing University · Harbin Institute of Technology (Shenzhen) · Australian National University +1
VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents
Oct 6, 2026Jiaju Chen, Min Yang, Jinghua Piao +4Token-Level Credit Assignment Credit Assignment in RL
University of Science and Technology of China · Zhongguancun Academy · Shandong University +2
Answer with Evidence: Consistency-Aware Grounded Visual Question Answering for Roadside Traffic Scenes
Oct 4, 2026Runwei Guan, Rongsheng Hu, Shangshu Chen +9Autonomous Driving Benchmarks Visual Question Answering
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Jiangnan University, Wuxi, China · Central China Normal University, Wuhan, China +6
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Sep 30, 2026Bo-Wen Zhang, Junwei He, Maoqi Liu +6Agentic RL Reward Shaping
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · ByteDance
GRPO Training Dynamics for Small Language Models
Sep 30, 2026Rajat Ghosh, Vaishnavi Bhargava, Henry Wong +2RL for Language Model Reasoning Group Relative Policy Optimization
Nutanix
Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine Tuning
Sep 29, 2026Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu +12Vision-Language-Action Models VLMs for Robotics
Beihang University · The Chinese University of Hong Kong · PKU-Psibot Lab +4
Train4Merge: A Controlled Single-Teacher Study of RL vs. SFT Teachers for OPD-Based Model Merging
Sep 26, 2026Jingyuan Huang, Zuming Huang, Yucheng Shi +4Language Model Merging Language Model Distillation
University of Georgia · INF Tech · Tencent +1
Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy
Sep 23, 2026Mehmet Turan Yardımcı, Yunus Emre ÇoğurcuRobot Policy Learning RL Exploration
Karlsruhe Institute of Technology · Çukurova University
WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps
Sep 22, 2026Abbas Mammadov, Jerry Y. Huang, Justin Lin +5Reinforcement Learning Generative Flow Networks
University of Oxford · Carnegie Mellon University
From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention
Sep 18, 2026Sichang Su, Benjamin Yang, Zhiyun Deng +4Robot Policy Adaptation Reinforcement Learning
UT Austin · Autel US · UC Berkeley
OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teacher
Sep 17, 2026Damiano Da Col, Maximilian Igl, Peter Karkus +5End-to-End Autonomous Driving RL Post-Training
KE:SAI · ETH Zürich · NVIDIA Research +2
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
Sep 17, 2026Juzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan +2Supervised Fine-Tuning Reinforcement Learning
University of Maryland · AWS AI Labs
CellRFT: Reinforcement Fine-Tuning for Single-Cell Perturbation Modeling
Sep 17, 2026Jie Yan, Li Liu, Hanze Guo +7Multi-Objective Reinforcement Learning Single-Cell Perturbation Modeling
Chinese Academy of Sciences · Peking University · Renmin University of China +2
EmbodiedMind: Adaptive Data Curation and Prefix-Tree Reinforcement Learning for Efficient Embodied Intelligence
Sep 17, 2026Feifan Wang, Zongbing Zhang, Yu Zhang +8Reinforcement Learning Robot Foundation Models
ZTE Corporation, Shenzhen, China
Reinforcement Learning for Real-Time Vision-Language-Action Policies
Sep 16, 2026Perry Dong, Kuo-Han Hung, Dorsa Sadigh +1Robot Policy Adaptation Robotic Manipulation
Stanford University
TIAO: Token Importance-Aware Policy Optimization for Text Summarization
Sep 15, 2026Qixiu Li, Chenlong Bao, Xiang Zhu +4RL for Language Models Token-Level Credit Assignment
National University of Defense Technology
Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models
Sep 14, 2026Yuxin Xiao, Sheng Zhang, Chandan Singh +4Reinforcement Learning Clinical Outcome Prediction
MIT · Microsoft Research
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Sep 8, 2026Hongbang Yuan, Zhuoran Jin, Yixin CaoReinforcement Learning Sparse-Reward RL
1Fudan University · 2CASIA · 3Shanghai Innovation Institute
Risk-Conditioned Fine-Tuning of Large Language Models
Sep 8, 2026Zixuan Liu, Fangzheng Wu, Brian Summa +1Risk-Sensitive RL LLM Safety
Department of Computer Science, Tulane University, New Orleans, LA, 70118, USA
Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratification
Sep 7, 2026Yimeng Ye, Shuang Chen, Wenxuan Huang +8Reinforcement Learning RL Fine-Tuning
Columbia University · CUHK · THU +1
Scaling Near-Optimal SFT-RL Annotation Budget Allocation from Small to Large LLMs
Sep 1, 2026Jingtan Wang, Arun Verma, Xiaoqiang Lin +4Supervised Fine-Tuning Language Model Scaling
Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre, Republic of Singapore · Agency for Science, Technology, and Research (A*STAR), Republic of Singapore +1
Uncovering and Mitigating Aggregation-Induced Reward Hacking in Multi-Reward Reinforcement Learning
Aug 31, 2026Yu Yuan, Yaoyou Fan, Lili Zhao +5Multi-Objective Reinforcement Learning Reward Hacking
University of Science and Technology of China · 2State Key Laboratory of Cognitive Intelligence · 3Peking University +1
Learning from What You Retrieve: Online RL Fine-Tuning for Semantic Retrieval
Aug 31, 2026Shaowei Wei, Chong Huang, Songtao Fang +3Semantic Search RL for IR
Alibaba Group Hangzhou, China
Reward-guided Fine-Tuning of One-Step Generative Models via Wasserstein Gradient Flow
Aug 30, 2026Hoseong Hwang, Woorim Han, Joungin Chun +2Image Generation One-Step Generative Modeling
Sungkyunkwan University, Seoul, Republic of Korea · Korea Institute for Advanced Study, Seoul, Republic of Korea
Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning
Aug 11, 2026Daoyi Li, Yixian Zhang, Wenbo Ding +2Reinforcement Learning Critic-Free RL
Tsinghua University
Progressive Alignment of Recommender Foundation Model through Multi-Phase Post-Training
Aug 7, 2026Oseong Choi, Hoeinn Kim, Jihoon Lee +2Recommender Systems RL Post-Training
NAVER WEBTOON Seongnam-si, Republic of Korea
TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
Aug 4, 2026Changle Qu, Sunhao Dai, Hengyi Cai +4On-Policy Self-Distillation RL for Language Model Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China · Baidu Inc.
Enhancing VLM Reward Models Through Structure-Aware Fine-Tuning
Aug 4, 2026Pyrros Koussios, Chenhao Li, Xin Chen +1Self-Supervised Learning VLM Adaptation
ETH Zürich
SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs
Aug 4, 2026Kejian Zhu, Zhuoran Jin, Shangqing Tu +5Supervised Fine-Tuning Gradient Interference
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Tsinghua University, Beijing, China

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Which Rollout Taught It That? BehaviorTrace and the Limits of Training-Data Attribution in Online RL

Amit Nautiyal

Independent Researcher

Gradient-Based Attribution Training Data Attribution RL Fine-Tuning

Open paper AskPDF arXiv

When reinforcement learning teaches a language model a new behavior, can we find the training rollouts that taught it? And when an attribution method says it can, how do we know the answer is real? We study both questions on online RL fine-tuning with GRPO, using a planted behavior with a known cause. We release BehaviorTrace, an open evaluation harness that combines full-gradient sketching, the planted-behavior setup, and controls for gradient magnitude, fluency, headroom, and variation across seeds and generation draws. Across three seeds on Qwen2.5-1.5B, much of the apparent attribution signal comes from confounds. A control that ranks training steps by gradient size alone, with no behavior target, reaches 4.2 to 4.5 times chance and matches or beats the best targeted estimator on two of three seeds. At saturated checkpoints, model fluency predicts the behavior label at least as well as every gradient method we compared it with. Once fluency is controlled, the per-rollout results change from seed to seed and from one generation draw to the next, so a single run cannot settle the question. One signal does hold on all three seeds. The gradient of the trigger tokens aligns with a target built where the behavior actually occurs. We turn these findings into a checklist for evaluating attribution in RL. We test existing estimators, including GAS (renormalized TracInCP) and a TRAK-style estimator, and do not propose a new one.

Momentum

Related topics

Latest papers 162