RL Fine-Tuning

RL: Reinforcement Learning

Overview

Reinforcement learning fine-tuning (RFT) adapts pretrained language, vision-language, and control models by updating them from feedback on generated answers or actions rather than relying only on demonstrations. Common approaches include PPO and GRPO for outcome-based learning, RL with verifiable rewards for math and code, and preference or rubric-based methods for open-ended tasks; curricula, reward models, and step-level credit assignment help make feedback more useful. Work also extends to tool-using agents, robots, diffusion policies, and offline-to-online control, where methods such as value learning, residual policies, and flow-matching optimization address costly or sparse environment feedback. Key challenges are designing reliable rewards, balancing exploration with stable updates, reducing rollout costs, and ensuring gains transfer across tasks without forgetting prior capabilities or exploiting the evaluator.

Latest papers

CardsList

242 papers10 papers in the last four weeks, down 52% on the four weeks before. 0.1% of all new papers.

  1. MARFT: Multi-Agent Reinforcement Fine-Tuning

    Apr 21, 2025Junwei Liao, Muning Wen, Jun Wang +1Multi-Agent LLM SystemsRL Fine-Tuning

  2. Efficient Reinforcement Finetuning via Adaptive Curriculum Learning

    Apr 7, 2025Taiwei Shi, Yiyang Wu, Linxin Song +2RL for Language Model ReasoningMathematical Reasoning