Self-Training for Language Models

Self-training studies how language models can improve by learning from text, solutions, critiques, or trajectories they generate, often using unlabeled prompts or user and environment feedback. Common approaches include confidence- or consensus-based pseudo-labeling, self-distillation, preference and reinforcement learning, verifier-guided training, and curricula that target model weaknesses; some adapt model weights at test time. Evaluations often focus on mathematical and code reasoning, tool use, long-context tasks, translation, and safety, using exact-answer, execution, or task-success benchmarks. The central challenge is distinguishing useful learning from reinforced errors, reduced diversity, bias, or gains caused by weak evaluation, so independent verification and stable stopping criteria remain important.