Self-Training for Language Models

Overview

Self-training studies how language models can improve by learning from text, solutions, critiques, or trajectories they generate, often using unlabeled prompts or user and environment feedback. Common approaches include confidence- or consensus-based pseudo-labeling, self-distillation, preference and reinforcement learning, verifier-guided training, and curricula that target model weaknesses; some adapt model weights at test time. Evaluations often focus on mathematical and code reasoning, tool use, long-context tasks, translation, and safety, using exact-answer, execution, or task-success benchmarks. The central challenge is distinguishing useful learning from reinforced errors, reduced diversity, bias, or gains caused by weak evaluation, so independent verification and stable stopping criteria remain important.

Latest papers

CardsList

95 papers12 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.

  1. Learning through Discernment: when a small language model learns from user feedback on the job, and when it decays

    Nov 18, 2026Sirisilp KongsilpLanguage Model Self-ImprovementOnline Language Model Adaptation

  2. Representational Change Under Self-Consuming Training: A Sparse Autoencoder Analysis

    Nov 16, 2026Suchit Gupte, Zhongteng Cai, Xueru Zhang +1Self-Training for Language ModelsSparse Autoencoders

  3. Compositional Self-Improvement: Promises and Pitfalls

    Oct 30, 2026Changho Shin, Daiwei Chen, Brenden Lake +2Language Model Self-ImprovementSelf-Training for Language Models

  4. ReMedI: Inference-Time Refinement and Self-Distillation for Controllable Molecular Optimization

    Oct 23, 2026Jia Yi Zhang, Shivanshu Shekhar, Allison McLevige +1Molecular OptimizationInference-Time Language Model Alignment

  5. Mitigating Model Collapse through Dreaming Learning

    Oct 21, 2026Matteo Benati, Alessandro Londei, Denise Lanzieri +2Model CollapseSelf-Training for Language Models

  6. Selection Is Not Learning: Choosing Model Futures

    Oct 15, 2026Haoyi Yang, Kshitij Mishra, Nils Lukas +1Language Model Self-ImprovementSelf-Training for Language Models

  7. DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents

    Oct 14, 2026Tianle Chen, Pushkar Singh, Ketan Todi +8Long-Horizon LLM AgentsSelf-Training for Language Models

  8. Heterogeneous Agentic Coevolution

    Oct 12, 2026Wenqian Ye, Ziwei Guan, Zi Yang +14LLM Agent Self-ImprovementSelf-Training for Language Models

  9. Environmental Feedback Modeling Matters: Rethinking Feedback Treatment in Agentic Hindsight Self-Distillation

    Oct 8, 2026Hangxi Guo, Fengyuan Liu, Yue Wang +4LLM Agent TrainingSelf-Distillation

  10. SIGMA: Self-Improving Alignment Generalization from a Model Spec

    Oct 6, 2026Jingyu Zhang, Shruti Palaskar, Daniel Khashabi +3LLM Safety AlignmentLLM Agent Safety

  11. Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation

    Oct 4, 2026Cheng Luo, Bing Li, Bernard GhanemTest-Time AdaptationContinual Test-Time Adaptation

  12. Questioning the Questions: Sustaining Self-Evolution in Reasoning Models

    Oct 3, 2026Jinyuan Li, Chengsong Huang, Langlin Huang +4Language Model Self-ImprovementSelf-Training for Language Models

  13. Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite

    Oct 2, 2026Zongxia Li, Yucheng Shi, Zhongzhi Li +7LLM Agent Trajectory SynthesisSelf-Training for Language Models

  14. SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision

    Sep 30, 2026Rong Wan, Wei Xie, Jiaxi Li +4Audio Deepfake DetectionSelf-Training for Language Models

  15. When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain

    Sep 29, 2026Chenxu Wang, Chaozhuo Li, Xinze Shi +6Language Model Self-ImprovementLLM Training

  16. Shockingly Simple Self-retrospection Improves Agentic Models Without RL

    Sep 28, 2026Jonathan Light, Christopher Zhang Cui, Jeonghye Kim +7Self-Training for Language ModelsLLM Agent Self-Improvement

  17. Teach to Learn: Hint Annealing for Self-improving LLM Reasoning

    Sep 28, 2026Zile Wang, Zijian Li, Haodong Wang +5Reinforcement LearningSelf-Training for Language Models

  18. Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training

    Sep 28, 2026Yuyang Deng, Yu Wang, Jiayun WangSelf-Training for Language ModelsLarge Language Model-Guided Optimization