RL Fine-Tuning

RL: Reinforcement Learning

Reinforcement learning fine-tuning (RFT) adapts pretrained language, vision-language, and control models by updating them from feedback on generated answers or actions rather than relying only on demonstrations. Common approaches include PPO and GRPO for outcome-based learning, RL with verifiable rewards for math and code, and preference or rubric-based methods for open-ended tasks; curricula, reward models, and step-level credit assignment help make feedback more useful. Work also extends to tool-using agents, robots, diffusion policies, and offline-to-online control, where methods such as value learning, residual policies, and flow-matching optimization address costly or sparse environment feedback. Key challenges are designing reliable rewards, balancing exploration with stable updates, reducing rollout costs, and ensuring gains transfer across tasks without forgetting prior capabilities or exploiting the evaluator.

Latest papers

233 papers17 papers in the last four weeks, down 23% on the four weeks before. 0.1% of all new papers.

When Does Closed-Loop Curriculum Adaptation Help GRPO? An Analysis on Countdown
Oct 28, 2026Yuv Raj Pant, Moshiur Rahman Tonmoy, Kaiqun FuRL for Language Model Reasoning Curriculum RL
Department of Computer Science, Texas Christian University · Department of Electrical Engineering and Computer Science, Texas A&M University - Kingsville
Freeze, Don't Clone: Fine-Tuning with Expressive Policies Without Retaining Offline Data
Oct 26, 2026Sami Nur Islam, Johan Obando-Ceron, Doina Precup +2RL Fine-Tuning Offline-To-Online RL
Mila – Québec AI Institute · McGill University · Université de Montréal +1
End-to-End MXFP8 Reinforcement Learning on Blackwell: What Matching Buys, When It Pays Off, and How It Fails
Oct 26, 2026Wengee ZhangRL Fine-Tuning Mixed-Precision Quantization
Is it RL or Data? A Controlled Comparison of GRPO and Rejection-Sampling Fine-Tuning
Oct 19, 2026Lakshya ChaudhryFine-Tuning RL Fine-Tuning
Purdue University
Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning
Oct 15, 2026Ruizhe Shi, Zaiwen Yang, Qiubo Wang +2Continual Learning for LLMs Sparse Fine-Tuning
University of Washington · Tsinghua University · University of Hong Kong +1
Post-Training, Finetuning, and Steering:A Field Guide for Curating Training Data
Oct 13, 2026Maria Attarian, Yifan Ruan, Claas Voelcker +5VLA Model Adaptation RL Fine-Tuning
University of Toronto · University of Texas at Austin
Spectral-Driven Adaptive-Rank Optimizer for Large Language Model Reinforcement Learning with Verifiable Rewards
Oct 12, 2026Zhipeng Chen, Wayne Xin Zhao, Ji-Rong WenRL Fine-Tuning RL for Language Model Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China
Growing and Transferring Pathways for LLM Reinforcement Learning
Oct 12, 2026Haolin Tian, Yikang Li, Tonghan WangTransfer Learning in RL RL for Language Model Reasoning
Tsinghua University
PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation
Oct 8, 2026Heng Li, Yong Zhang, Ning Cheng +5Few-Shot Text Classification Adaptive Knowledge Distillation
Ping An Technology (Shenzhen) Co., Ltd., China · University of Science and Technology of China
Which Rollout Taught It That? BehaviorTrace and the Limits of Training-Data Attribution in Online RL
Oct 7, 2026Amit NautiyalGradient-Based Attribution Training Data Attribution
Independent Researcher
Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization
Oct 7, 2026Haoru Li, Jinmei Liu, Zhiyong Wang +5Robot Policy Generalization RL Fine-Tuning
Nanjing University · Harbin Institute of Technology (Shenzhen) · Australian National University +1
VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents
Oct 6, 2026Jiaju Chen, Min Yang, Jinghua Piao +4Token-Level Credit Assignment Credit Assignment in RL
University of Science and Technology of China · Zhongguancun Academy · Shandong University +2
Answer with Evidence: Consistency-Aware Grounded Visual Question Answering for Roadside Traffic Scenes
Oct 4, 2026Runwei Guan, Rongsheng Hu, Shangshu Chen +10Autonomous Driving Benchmarks Visual Question Answering
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Jiangnan University, Wuxi, China. · Central China Normal University, Wuhan, China. +6
Playing social deduction games with reinforcement fine-tuned large language models
Oct 3, 2026Lingzhe Zhang, Yunpeng Zhai, Tong Jia +7RL for LLM Agents RL Fine-Tuning
Peking University, Beijing, China. · Alibaba Group, China. · University of Illinois Chicago, Chicago, Illinois, United States.
Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT
Oct 2, 2026Joery Ariën de Vries, Neil David Lawrence, Zhenwen DaiCritic-Free RL RL Fine-Tuning
Trent AI Limited, London, United Kingdom · Computer Laboratory, University of Cambridge, Cambridge, United Kingdom
Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks
Oct 1, 2026Haodong Liang, Yanhao Jin, Krishnakumar Balasubramanian +1Supervised Fine-Tuning RL Fine-Tuning
University of California, Davis
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Sep 30, 2026Bo-Wen Zhang, Junwei He, Maoqi Liu +6Agentic RL Reward Shaping
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · ByteDance
GRPO Training Dynamics for Small Language Models
Sep 30, 2026Rajat Ghosh, Vaishnavi Bhargava, Henry Wong +2RL for Language Model Reasoning Group Relative Policy Optimization
Nutanix
On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training
Sep 29, 2026Shufan Shen, Zhongni Hou, Junshu Sun +5LLM Post-Training Fine-Tuning
State Key Lab. of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy Sciences · Meituan
Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine Tuning
Sep 29, 2026Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu +12Vision-Language-Action Models VLMs for Robotics
Beihang University · The Chinese University of Hong Kong · PKU-Psibot Lab +4
PF-RL: Progress Field Reinforcement Learning via Goal-Conditioned Value Geometry for Vision-Language-Action Models
Sep 26, 2026Yunpeng Qing, Yilun Kong, Sixu Lin +9Reinforcement Learning RL for VLMs
Zhejiang University · Nanyang Technological University · CUHK-Shenzhen +7
Train4Merge: RL or SFT---Which Yields Better Teachers for OPD-Based Model Merging?
Sep 26, 2026Jingyuan Huang, Zuming Huang, Yucheng Shi +4Language Model Merging Language Model Distillation
University of Georgia · INF Tech · Tencent +1
RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process
Sep 26, 2026Yu Li, Shenghe Hu, Yuhan Wang +4Robot Policy Learning RL for Robotics
Institute for Artificial Intelligence, Peking University · Zhongguancun Academy · PKU-PsiBot Joint Lab +1
Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy
Sep 23, 2026Mehmet Turan Yardımcı, Yunus Emre ÇoğurcuRobot Policy Learning RL Exploration
Karlsruhe Institute of Technology · Çukurova University
WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps
Sep 22, 2026Abbas Mammadov, Jerry Y. Huang, Justin Lin +5Reinforcement Learning Generative Flow Networks
University of Oxford · Carnegie Mellon University
Towards Full Pipeline FP8 Reinforcement Learning for LLMs
Sep 19, 2026Fanchao Chen, Ziheng Jiang, Ziyun Wei +5RL for Language Models RL Fine-Tuning
University of Wisconsin–Madison · Work done during internship at ByteDance Seed · ByteDance Seed
From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention
Sep 18, 2026Sichang Su, Benjamin Yang, Zhiyun Deng +4Robot Policy Adaptation Reinforcement Learning
UT Austin · Autel US · UC Berkeley
SynthDemo-RL: Breaking the Zero-Reward Barrier in VLA Adaptation with LLM-Guided Synthetic Demonstrations
Sep 18, 2026Hiroaki Kingetsu, Hiroaki Kurihara, Kaoru Yokoo +2Vision-Language-Action Models RL Fine-Tuning
Fujitsu Limited, Kawasaki, Japan · The Institute of Statistical Mathematics, Tokyo, Japan
OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teacher
Sep 17, 2026Damiano Da Col, Maximilian Igl, Peter Karkus +5End-to-End Autonomous Driving RL Post-Training
KE:SAI · ETH Zürich · NVIDIA Research +2
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
Sep 17, 2026Juzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan +2Supervised Fine-Tuning Reinforcement Learning
University of Maryland · AWS AI Labs

Oct 28, 2026J/K move · Enter open · S save

When Does Closed-Loop Curriculum Adaptation Help GRPO? An Analysis on Countdown

Yuv Raj Pant, Moshiur Rahman Tonmoy, Kaiqun Fu

Department of Computer Science, Texas Christian University · Department of Electrical Engineering and Computer Science, Texas A&M University - Kingsville

RL for Language Model Reasoning Curriculum RL RL Fine-Tuning Group Relative Policy Optimization

Open paper AskPDF arXiv

Reinforcement fine-tuning has become an effective post-training paradigm for improving the reasoning ability of large language models, but it remains sample-inefficient when the training distribution is mismatched to the model's current capability. In group-relative objectives such as Group Relative Policy Optimization (GRPO), problems that are too hard often produce uniformly failed rollouts, while problems that are too easy provide little useful advantage signal. In this work, we introduce CLOSEDLOOPGAUSSIAN, a lightweight adaptive curriculum method for efficient reinforcement fine-tuning on ordered reasoning tasks. Instead of following a fixed easy-to-hard schedule, CLOSEDLOOPGAUSSIAN preserves the structure of an Easy-to-Hard Gaussian sampler while dynamically adjusting its pace and spread using periodic probe estimates of per-difficulty accuracy. The controller accelerates training when the model is ahead of a reference curriculum, slows down when the model falls behind, and broadens sampling when earlier difficulties regress. This feedback mechanism keeps training focused on tasks that are challenging but still learnable, without changing the reward function, model architecture, or underlying GRPO optimizer. Experiments on symbolic arithmetic reasoning task show that CLOSEDLOOPGAUSSIAN improves exact-match accuracy over vanilla GRPO and fixed open-loop curricula, with the strongest gains in intermediate-difficulty regimes where group-relative learning signals are most informative. Codes are available at https://github.com/yuvrajpant56/Closed_Loop_Curriculum.