RL Fine-Tuning

RL: Reinforcement Learning

Reinforcement learning fine-tuning (RFT) adapts pretrained language, vision-language, and control models by updating them from feedback on generated answers or actions rather than relying only on demonstrations. Common approaches include PPO and GRPO for outcome-based learning, RL with verifiable rewards for math and code, and preference or rubric-based methods for open-ended tasks; curricula, reward models, and step-level credit assignment help make feedback more useful. Work also extends to tool-using agents, robots, diffusion policies, and offline-to-online control, where methods such as value learning, residual policies, and flow-matching optimization address costly or sparse environment feedback. Key challenges are designing reliable rewards, balancing exploration with stable updates, reducing rollout costs, and ensuring gains transfer across tasks without forgetting prior capabilities or exploiting the evaluator.

Latest papers

233 papers17 papers in the last four weeks, down 23% on the four weeks before. 0.1% of all new papers.

Direct Reward Fine-Tuning on Poses for Single Image to 3D Human in the Wild
Mar 3, 2026Seunguk Do, Minwoo Huh, Joonghyuk Shin +1Multi-View Diffusion Models 3D Human Reconstruction
Seoul National University
Harmonizing Dense and Sparse Signals in Multi-turn RL: Dual-Horizon Credit Assignment for Industrial Sales Agents
Mar 2, 2026Haojin Yang, Ai Jian, Xinyue Huang +5RL for LLM Agents Reinforcement Learning
Peking University · Beijing University of Posts and Telecommunications · Sun Yat-sen University +2
Learn Hard Problems During RL with Reference Guided Fine-tuning
Mar 1, 2026Yangzhen Wu, Shanda Li, Zixin Wen +5LLM Mathematical Reasoning RL for Language Model Reasoning
UC Berkeley · Carnegie Mellon University · ByteDance Seed
Stabilizing Policy Optimization via Logits Convexity
Mar 1, 2026Hongzhan Chen, Tao Yang, Yuhua Zhu +3Policy Optimization Reinforcement Learning
School of Computer Science and Engineering, Sun Yat-sen University, China · Shanghai Innovation Institute · Wechat Search, Tencent Inc, China +1
RT-RMOT: A Dataset and Framework for RGB-Thermal Referring Multi-Object Tracking
Feb 25, 2026Yanqiu Yu, Zhifan Jin, Sijia Chen +4Referring MOT Visual Object Tracking
Huazhong University of Science and Technology, Wuhan, Hubei, China · South-Central Minzu University, Wuhan, Hubei, China
IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic Manipulation
Feb 24, 2026Zhian Su, Weijie Kong, Haonan Dong +1RL Fine-Tuning VLMs for Robotics
Grasp Laboratory, Mechanical Engineering Department, Zhejiang University, Hangzhou 310027, China. · Torch Kernel Co., Ltd., Hangzhou 310000, China.
Janus-Q: End-to-End Event-Driven Trading via Hierarchical-Gated Reward Modeling
Feb 23, 2026Xiang Li, Zikai Wei, Yiyan Qi +6Algorithmic Trading Reward Modeling
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · International Digital Economy Academy, Shenzhen, China
STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens
Feb 17, 2026Shiqi Liu, Zeyu He, Guojian Zhan +10Reinforcement Learning RL for Language Model Reasoning
School of Vehicle and Mobility & College of AI, Tsinghua University · Didi Voyager Labs, DiDi Autonomous Driving
TAROT: Test-driven and Capability-adaptive Curriculum Reinforcement Fine-tuning for Code Generation with Large Language Models
Feb 17, 2026Chansung Park, Juyong Jiang, Fan Wang +4LLM Code Generation RL for Code Generation
Electronics and Telecommunications Research Institute · The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology +2
Goldilocks RL: Tuning Task Difficulty to Escape Sparse Rewards for Reasoning
Feb 16, 2026Ilia Mahrooghi, Aryo Lotfi, Emmanuel AbbeRL for Language Model Reasoning Curriculum RL
EPFL · Apple
Constrained Group Relative Policy Optimization
Feb 5, 2026Roger Girgis, Rodrigue de Schaetzen, Luke Rowe +3Group Relative Policy Optimization Constrained RL
Mila – Quebec AI Institute Université de Montréal
QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuning
Feb 4, 2026Doyeon Lee, Eunyi Lyou, Hyunsoo Cho +3Trust-Region Optimization Policy Learning
Seoul National University · Department of Artificial Intelligence / Institute for Multiscale Matter and Systems, Ewha Woman University · Geogia Institute of Technology
OpenTinker: Separating Concerns in Agentic Reinforcement Learning
Jan 12, 2026Siqi Zhu, Jiaxuan YouAgentic RL LLM Agent Training
University of Illinois Urbana-Champaign
Understanding Negative Learning in Large Language Model Fine-tuning
Nov 30, 2025Yue Wang, Qizhou Wang, Zizhuo Zhang +3LLM Fine-Tuning RL Fine-Tuning
TMLR Group, Department of Computer Science, Hong Kong Baptist University · RIKEN AIP · The University of Tokyo
Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models
Oct 11, 2025Mingyang Lyu, Yinqian Sun, Erliang Lin +4Flow Matching Robotic Manipulation
Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China. · Beijing Institute of AI Safety and Governance, China. · University of Chinese Academy of Sciences (UCAS), Beijing, China. +3
Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
Oct 9, 2025Yuchen Zhu, Wei Guo, Jaemoo Choi +4RL for Language Model Reasoning Policy Optimization
Georgia Institute of Technology.
Oracle-RLAIF: An Improved Fine-Tuning Framework for Multi-modal Video Models using Reinforcement Learning from Ranking Feedback
Oct 2, 2025Derek Shi, Ruben Glatt, Christine Klymko +5Video-Language Models RL Fine-Tuning
Stanford University, CA, USA · Lawrence Livermore National Laboratory, CA, USA · Microsoft, WA, USA
The Three Regimes of Offline-to-Online Reinforcement Learning
Oct 1, 2025Lu Li, Tianwei Ni, Yihao Sun +1Offline RL RL Fine-Tuning
Mila – Québec AI Institute · Université de Montréal · CIFAR AI Chair
Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning
Sep 29, 2025Xin Qiu, Yulu Gan, Conor F. Hayes +6Fine-Tuning Evolutionary Optimization
Cognizant AI Lab, San Francisco, CA, USA · Massachusetts Institute of Technology, Cambridge, MA, USA · University of California, Los Angeles, Los Angeles, CA, USA +1
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Aug 13, 2025Maxime Heuillet, Yufei Cui, Boxing Chen +2RL for Language Model Reasoning Efficient Language Model Training
Universit´e Laval (IID), Canada · Mila - Qu´ebec AI Institute, Canada · Huawei Noah’s Ark Lab (Montreal Research Center), Canada +1
Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
Jul 7, 2025Song Lai, Haohan Zhao, Rong Feng +9Continual Learning for LLMs LLM Post-Training
Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong, China · City University of Hong Kong, Hong Kong, China · School of Artificial Intelligence, Jilin University, Changchun, China +3
MARFT: Multi-Agent Reinforcement Fine-Tuning
Apr 21, 2025Junwei Liao, Muning Wen, Jun Wang +1Multi-Agent LLM Systems RL Fine-Tuning
Shanghai Jiao Tong University · Shanghai Innovation Institute · OPPO Research Institute
Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
Apr 7, 2025Taiwei Shi, Yiyang Wu, Linxin Song +2RL for Language Model Reasoning Mathematical Reasoning
University of Southern California · Carnegie Mellon University · Mohamed Bin Zayed University of Artificial Intelligence

Mar 3, 2026J/K move · Enter open · S save

Direct Reward Fine-Tuning on Poses for Single Image to 3D Human in the Wild

Seunguk Do, Minwoo Huh, Joonghyuk Shin, Jaesik Park

Seoul National University

Multi-View Diffusion Models 3D Human Reconstruction Single-View 3D Reconstruction Diffusion Models RL Fine-Tuning

Open paper AskPDF arXiv

Single-view 3D human reconstruction has achieved remarkable progress through the adoption of multi-view diffusion models, yet the recovered 3D humans often exhibit unnatural poses. This phenomenon becomes pronounced when reconstructing 3D humans with dynamic or challenging poses, which we attribute to the limited scale of available 3D human datasets with diverse poses. To address this limitation, we introduce DrPose, Direct Reward fine-tuning algorithm on Poses, which enables post-training of a multi-view diffusion model on diverse poses without requiring expensive 3D human assets. DrPose trains a model using only human poses paired with single-view images, employing a direct reward fine-tuning to maximize PoseScore, which is our proposed differentiable reward that quantifies consistency between a generated multi-view latent image and a ground-truth human pose. This optimization is conducted on DrPose15K, a novel dataset that was constructed from an existing human motion dataset and a pose-conditioned video generative model. Constructed from abundant human pose sequence data, DrPose15K exhibits a broader pose distribution compared to existing 3D human datasets. We validate our approach through evaluation on conventional benchmark datasets, in-the-wild images, and a newly constructed benchmark, with a particular focus on assessing performance on challenging human poses. Our results demonstrate consistent qualitative and quantitative improvements across all benchmarks. Project page: https://seunguk-do.github.io/drpose.