Self-Training for Language Models

Follow this topic Ask about this topic

Self-training studies how language models can improve by learning from text, solutions, critiques, or trajectories they generate, often using unlabeled prompts or user and environment feedback. Common approaches include confidence- or consensus-based pseudo-labeling, self-distillation, preference and reinforcement learning, verifier-guided training, and curricula that target model weaknesses; some adapt model weights at test time. Evaluations often focus on mathematical and code reasoning, tool use, long-context tasks, translation, and safety, using exact-answer, execution, or task-success benchmarks. The central challenge is distinguishing useful learning from reinforced errors, reduced diversity, bias, or gains caused by weak evaluation, so independent verification and stable stopping criteria remain important.

Latest papers

84 papers20 papers in the last four weeks, up 82% on the four weeks before. 0.1% of all new papers.

A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement
Oct 28, 2026Yu Liu, Youxing Li, Yuedong Tan +17Language Model Self-Improvement Self-Training for Language Models
StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue
Oct 24, 2026Samarth Varshney, Ritu YadavInstruction Following Multi-Turn Dialogue Management
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
Oct 23, 2026Leon Sick, Lukas Hoyer, Yannick Strümpler +3Language Model Self-Improvement Self-Training for Language Models
Google · Ulm University
ReMedI: Inference-Time Refinement and Self-Distillation for Controllable Molecular Optimization
Oct 23, 2026Jia Yi Zhang, Shivanshu Shekhar, Allison McLevige +1Molecular Optimization Inference-Time Language Model Alignment
University of Illinois Urbana-Champaign
Mitigating Model Collapse through Dreaming Learning
Oct 21, 2026Matteo Benati, Alessandro Londei, Denise Lanzieri +2Model Collapse Self-Training for Language Models
DIAG, Sapienza University of Rome, Italy · Sony CSL Rome, CREF, Rome, Italy · Copenhagen Center for Social Data Science (SODAS), University of Copenhagen, Denmark +2
Learning from Unverified Peers via Shared Residual Distillation
Oct 15, 2026Feiyang Chen, Runze Ma, Zichen Wang +4Self-Training for Language Models Language Model Distillation
Shanghai Jiao Tong University
Selection Is Not Learning: Choosing Model Futures
Oct 15, 2026Haoyi Yang, Kshitij Mishra, Nils Lukas +1Language Model Self-Improvement Self-Training for Language Models
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents
Oct 14, 2026Tianle Chen, Pushkar Singh, Ketan Todi +8Long-Horizon LLM Agents Self-Training for Language Models
Google · Boston University · University of Washington +2
Back-Translation Still Matters: Leveraging Monolingual Data for Low-Resource Machine Translation with LLMs
Oct 13, 2026Divyanshu Bathla, Parag Singla, Jitin SinglaBack-Translation Low-Resource MT
IIT Roorkee · IIT Delhi
Heterogeneous Agentic Coevolution
Oct 12, 2026Wenqian Ye, Ziwei Guan, Zi Yang +14LLM Agent Self-Improvement Self-Training for Language Models
Meta AI · University of Virginia
Environmental Feedback Modeling Matters: Rethinking Feedback Treatment in Agentic Hindsight Self-Distillation
Oct 8, 2026Hangxi Guo, Fengyuan Liu, Yue Wang +4LLM Agent Training Self-Distillation
The Chinese University of Hong Kong, Shenzhen · Shanghai AI Laboratory · University of Science and Technology of China +2
SIGMA: Self-Improving Alignment Generalization from a Model Spec
Oct 6, 2026Jingyu Zhang, Shruti Palaskar, Daniel Khashabi +3LLM Safety Alignment LLM Agent Safety
Apple · Johns Hopkins University
Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation
Oct 4, 2026Cheng Luo, Bing Li, Bernard GhanemTest-Time Adaptation Continual Test-Time Adaptation
King Abdullah University of Science and Technology (KAUST)
Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
Oct 3, 2026Jinyuan Li, Chengsong Huang, Langlin Huang +4Language Model Self-Improvement Self-Training for Language Models
Washington University in St. Louis · University of Michigan, Ann Arbor
Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite
Oct 2, 2026Zongxia Li, Yucheng Shi, Zhongzhi Li +7LLM Agent Trajectory Synthesis Self-Training for Language Models
Tencent HY LLM Frontier · University of Maryland, College Park · University of Georgia +4
SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision
Sep 30, 2026Rong Wan, Wei Xie, Jiaxi Li +4Audio Deepfake Detection Self-Training for Language Models
University of Surrey · Guangxi University · Shenzhen University of Advanced Technology +1
When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain
Sep 29, 2026Chenxu Wang, Chaozhuo Li, Xinze Shi +6Language Model Self-Improvement LLM Training
Beijing University of Posts and Telecommunications · Beijing Academy of Artificial Intelligence · Central South University +2
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Sep 28, 2026Zhiwei Zhang, Huayu Deng, Fei Zhao +4Supervised Fine-Tuning Self-Training for Language Models
AllSpark Team
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
Sep 28, 2026Jonathan Light, Christopher Zhang Cui, Jeonghye Kim +7Self-Training for Language Models LLM Agent Self-Improvement
RPI · Microsoft Research · UC San Diego +2
Teach to Learn: Hint Annealing for Self-improving LLM Reasoning
Sep 28, 2026Zile Wang, Zijian Li, Haodong Wang +5Reinforcement Learning Self-Training for Language Models
The Hong Kong University of Science and Technology · Yuanbao Team, Tencent
Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training
Sep 28, 2026Yuyang Deng, Yu Wang, Jiayun WangSelf-Training for Language Models Large Language Model-Guided Optimization
Accenture, Center for Advanced AI · Georgia Institute of Technology
SeOPD: Self-Evolving LLMs via Online Policy Distillation from Self-Generated Chain-of-Thought
Sep 27, 2026Xiaoshu Chen, Sihang Zhou, Ke Liang +1Language Model Self-Improvement Self-Training for Language Models
National University of Defense Technology
Not Too Hard, Not Too Easy: Learning from Intermediate States for LLM Structured Reasoning
Sep 27, 2026Hongbo Chen, Guohua Lu, Ting Dang +1Structured Reasoning in Language Models Self-Training for Language Models
University of Auckland, Auckland, New Zealand · University of Melbourne, Melbourne, Australia
KernelZero: Co-Evolving Proposer and Coder for Continuously Improved GPU Kernel Generation
Sep 27, 2026Changxin Ke, Rui Zhang, Zixiang Fang +8GPU Kernel Generation LLM-Based Program Synthesis
State Key Lab of Processors, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Intelligent Software Research Center, Institute of Software, CAS +1
Strategically Diverse Sampling for Self-Training
Sep 25, 2026Alexander Gurung, Esmeralda S. Whitammer, Mirella LapataSelf-Training for Language Models Diversity-Aware Sampling
School of Informatics, University of Edinburgh
Don't Repeat Yourself: Self-Supervised Fine-Tuning for Coverage
Sep 16, 2026Eric Fithian, Kirill Skobelev, X. Y. HanSupervised Fine-Tuning Diverse Text Generation
University of Chicago · Northwestern University
STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution
Sep 16, 2026Yajie Yu, Mark Lee, Yue FengSelf-Training for Language Models RL for Language Model Reasoning
School of Computer Science University of Birmingham Birmingham United Kingdom
Break Step: Recursive Training Resonates with Replayed Sampling Noise
Sep 10, 2026Yangze Liu, Zhongyi HanSelf-Training for Language Models Model Collapse
Shandong University
From Rollouts to Recipes: Self-Contained Post-Training for LLMs
Sep 1, 2026Yifei Li, Lingling Zhang, Muye Huang +3Self-Training for Language Models RL for Language Model Reasoning
School of Computer Science and Technology, Xi’an Jiaotong University, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · Zhongguancun Academy, Beijing, China +1
Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Models
Sep 1, 2026Xincheng Wei, Yifan Ding, Fucheng Xiong +6Self-Training for Language Models Language Model Self-Improvement
The Chinese University of Hong Kong, Shenzhen · Meituan, LongCat Team · Peking University +2

Oct 28, 2026J/K move · Enter open · S save

A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement

Yu Liu, Youxing Li, Yuedong Tan, Zhiwei Yang+16

Language Model Self-Improvement Self-Training for Language Models Multimodal SSL

Open paper AskPDF arXiv

Recursive self-improvement requires models not only to acquire useful training experience, but also to decide when their current experience has ceased to be useful. Existing methods often focus on what experience to generate next, while paying less attention to when fresh experience should be introduced. Many also construct training targets through self-consistency or majority voting, making supervision depend on the model's own uncertain predictions. We address these two questions separately: training dynamics determine when to acquire new experience, while the answers to new tasks remain independently checkable. A-RSI monitors reward variation and accuracy in training logs to refresh a task pool when its learning signal declines, without consulting validation or test scores. A static library of model-authored programs produces multimodal questions and derives their reference answers; we specify the task-generation contract, automatically verify chosen properties of the reward and refresh rules, and separately audit the generated answers. The training loop uses no external training examples or human-written answer labels. On the primary eleven-benchmark suite, A-RSI raises the performance average from 57.77 to 61.05 with 800 fresh tasks.