RL for Language Model Reasoning

RL: Reinforcement Learning

Overview

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

CardsList

1,171 papers80 papers in the last four weeks, down 55% on the four weeks before. 0.6% of all new papers.

  1. Amortizing intractable inference in large language models

    Oct 6, 2023Edward J. Hu, Moksh Jain, Eric Elmoznino +4Amortized InferenceLLM Fine-Tuning