RL for Language Models

This field studies how feedback can shape a language model’s behavior after pretraining, including how it answers, reasons, uses tools, and handles uncertainty. Common approaches include preference-based reinforcement learning from human or AI feedback, reinforcement learning with verifiable rewards for tasks such as math and coding, and policy methods such as PPO and GRPO; rubrics and step- or turn-level rewards provide finer guidance. Researchers evaluate these methods on reasoning benchmarks as well as dialogue, tool use, multilingual tasks, and domain-specific applications. A central tension is that sparse or imperfect rewards can be costly to obtain and easy to exploit, while reliable credit assignment, exploration, and generalization beyond the training setting remain difficult.