RL for Generative Models

This field studies how reward and preference feedback can improve models that generate text, images, video, molecules, proteins, and robot actions. Common approaches include policy-gradient methods such as PPO and GRPO, reward-weighted fine-tuning, preference optimization, and reinforcement learning for diffusion and flow-matching models. Rewards may come from verifiable rules, learned evaluators, human comparisons, physical simulators, or downstream task performance. A central challenge is making feedback informative and reliable without reward hacking, unstable training, excessive compute, or loss of output diversity.