Self-Training for Language Models

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 83% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 41

All topics

Environmental Feedback Modeling Matters: Rethinking Feedback Treatment in Agentic Hindsight Self-Distillation
Oct 8, 2026Hangxi Guo, Fengyuan Liu, Yue Wang +4LLM Agent Training Self-Distillation
The Chinese University of Hong Kong, Shenzhen · Shanghai AI Laboratory · University of Science and Technology of China +2
Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation
Oct 4, 2026Cheng Luo, Bing Li, Bernard GhanemTest-Time Adaptation Continual Test-Time Adaptation
King Abdullah University of Science and Technology (KAUST)
Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
Oct 3, 2026Jinyuan Li, Chengsong Huang, Langlin Huang +4Language Model Self-Improvement Self-Training for Language Models
Washington University in St. Louis · University of Michigan, Ann Arbor
SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision
Sep 30, 2026Rong Wan, Wei Xie, Jiaxi Li +4Audio Deepfake Detection Self-Training for Language Models
University of Surrey · Guangxi University · Shenzhen University of Advanced Technology +1
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Sep 28, 2026Zhiwei Zhang, Huayu Deng, Fei Zhao +4Supervised Fine-Tuning Self-Training for Language Models
AllSpark Team
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
Sep 28, 2026Jonathan Light, Christopher Zhang Cui, Jeonghye Kim +7Self-Training for Language Models LLM Agent Self-Improvement
RPI · Microsoft Research · UC San Diego +2
Teach to Learn: Hint Annealing for Self-improving LLM Reasoning
Sep 28, 2026Zile Wang, Zijian Li, Haodong Wang +5Reinforcement Learning Self-Training for Language Models
The Hong Kong University of Science and Technology · Yuanbao Team, Tencent
Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training
Sep 28, 2026Yuyang Deng, Yu Wang, Jiayun WangSelf-Training for Language Models Large Language Model-Guided Optimization
Accenture, Center for Advanced AI · Georgia Institute of Technology
SeOPD: Self-Evolving LLMs via Online Policy Distillation from Self-Generated Chain-of-Thought
Sep 27, 2026Xiaoshu Chen, Sihang Zhou, Ke Liang +1Language Model Self-Improvement Self-Training for Language Models
National University of Defense Technology
Not Too Hard, Not Too Easy: Learning from Intermediate States for LLM Structured Reasoning
Sep 27, 2026Hongbo Chen, Guohua Lu, Ting Dang +1Self-Training for Language Models Structured Reasoning
University of Auckland, Auckland, New Zealand · University of Melbourne, Melbourne, Australia
STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution
Sep 16, 2026Yajie Yu, Mark Lee, Yue FengSelf-Training for Language Models RL for Language Model Reasoning
School of Computer Science University of Birmingham Birmingham United Kingdom
Break Step: Recursive Training Resonates with Replayed Sampling Noise
Sep 10, 2026Yangze Liu, Zhongyi HanSelf-Training for Language Models Model Collapse
Shandong University
From Rollouts to Recipes: Self-Contained Post-Training for LLMs
Sep 1, 2026Yifei Li, Lingling Zhang, Muye Huang +3Self-Training for Language Models RL for Language Model Reasoning
School of Computer Science and Technology, Xi’an Jiaotong University, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · Zhongguancun Academy, Beijing, China +1
Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Models
Sep 1, 2026Xincheng Wei, Yifan Ding, Fucheng Xiong +6Self-Training for Language Models Language Model Self-Improvement
The Chinese University of Hong Kong, Shenzhen · Meituan, LongCat Team · Peking University +2
WebWorld: The Browser as a World Model for Self-Improving Web Code
Aug 31, 2026Jiajun Wu, Jian Yang, Yaxin Du +7Self-Training for Language Models Code Generation
Beihang University · Shanghai Jiao Tong University · IQuest Research +1
Learning to Reason and Use Tools through Unsupervised Fine-Tuning in Task-Oriented Dialog Systems
Aug 31, 2026Markel Ferro, Oier Lopez de LacalleSelf-Training for Language Models LLM Fine-Tuning
HiTZ Center - Ixa, University of the Basque Country UPV/EHU
J-Zero: Unified Challenger--Solver--Judge Self-Evolution from Zero Data
Aug 27, 2026Gyouk Chu, Myeongho Jeon, Teresa Yeo +1LLM-as-a-Judge Self-Training for Language Models
KAIST
Different Feedback, Different Updates: Selective Self-Learning from User Interactions for Large Language Models
Aug 10, 2026Xuanchen Li, Haitao Li, Yujia Zhou +5Continual Learning for LLMs Self-Training for Language Models
DCST, Tsinghua University · Tencent
Learning from Consensus and Disagreement: Unsupervised On-Policy Self-Distillation with Minority-Trajectory Contrast
Aug 9, 2026Jiaxin Guo, Yanwei Yue, Xuanbo Fan +2Self-Training for Language Models Language Model Distillation
School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, Peking University, Beijing, China · Ucap Cloud
The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data
Aug 4, 2026Irina Proskurina, Antoine Gourru, Julien VelcinSynthetic Data Pretraining Self-Training for Language Models
Laboratoire Hubert Curien, UMR CNRS 5516, Saint-Étienne, France · Université Claude Bernard Lyon 1, Université Lumière Lyon 2, ERIC · École Centrale de Lyon, LIRIS, CNRS UMR 5205
Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance
Aug 1, 2026Zhuowen Han, Jinwei Xiao, Zhengxi Lu +9Self-Training for Language Models RL for Language Model Reasoning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University · Meituan Longcat Team
EvoThink: Evolving Thinking in Large Reasoning Models via Self-Pruning and Aha-Moment Preference Optimization
Jul 22, 2026Xinbang Dai, Zheyu Xin, Huikang Hu +7Overthinking in Language Models Self-Training for Language Models
Southeast University · Noah’s Ark Lab
Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuning
Jul 19, 2026Xiaonan Luo, Yue Huang, Kehan Guo +4Self-Training for Language Models Model Collapse
University of Notre Dame · Vanderbilt University · University of Pennsylvania
Consensus as Privileged Context for Label-Free Self-Distillation
Jul 15, 2026John Gkountouras, Josip Jukić, Ivan TitovSelf-Training for Language Models Language Model Distillation
1ILLC, University of Amsterdam · 2ILCC, University of Edinburgh
Revisiting Chain-of-Thought Reasoning under Limited Supervision: Semi-supervised Chain-of-Thought Learning
Jul 1, 2026Hongyang He, Jiuming Liu, Victor SanchezSelf-Training for Language Models CoT Reasoning
University of Warwick · University of Cambridge
DRIFT: Difficulty Routing Self-DIstillation with Rhythm-Gated Exploration and Success BuFfer Training
Jun 29, 2026Haisen Luo, Yiwei Liu, Haoning Wang +13Self-Training for Language Models RL for Language Model Reasoning
3Beike Language and Intelligence · 2ENS Paris-Saclay · 1Tsinghua University
Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-Training
Jun 17, 2026Jianzhe LinRL for Code Generation Reinforcement Learning
MetaAI · *Work done at Meta
Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning
Jun 14, 2026Feng Lyu, Jinfeng Cen, Sijing Duan +4Self-Training for Language Models LLM Fine-Tuning
Central South University · Tsinghua University · Nanjing University +1
Teacher-Free Self-Training Amplifies but Does Not Compound: A Pass@ $K$ Crossover on a Free-Verifier Domain
Jun 5, 2026Igor Lima StrozziSelf-Training for Language Models Self-Training
Applied Mathematics Department Federal University of Rio de Janeiro
Not All Synthetic Data Is Yours to Learn From
May 29, 2026Sina Alemohammad, Li Chen, Richard G. Baraniuk +1Synthetic Data Pretraining Memorization in Language Models
ECE Department The University of Texas at Austin · Apple · ECE Department Rice University

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

Environmental Feedback Modeling Matters: Rethinking Feedback Treatment in Agentic Hindsight Self-Distillation

Hangxi Guo, Fengyuan Liu, Yue Wang, Yuhua Qi+3

The Chinese University of Hong Kong, Shenzhen · Shanghai AI Laboratory · University of Science and Technology of China+2

LLM Agent Training Self-Distillation Self-Training for Language Models

Open paper AskPDF arXiv

Reinforcement learning is commonly used to train language agents in interactive environments, but cannot be directly applied when rewards are unavailable. Recent methods use environmental feedback as privileged context for hindsight self-distillation, but our analysis suggests that simply conditioning the teacher on feedback is insufficient, motivating us to rethink how environmental feedback is used in agentic self-distillation. Given that environmental feedback contains rich supervision for modeling how the environment responds to agent actions, we introduce \textit{agentic SElf-distilLation with environmental Feedback modeling} (SELF), a framework that jointly optimizes environmental feedback modeling and hindsight self-distillation. SELF learns to predict environmental responses while distilling guidance from a feedback-conditioned self-teacher into the policy. Our analysis reveals a mutually reinforcing mechanism: environmental feedback modeling strengthens hindsight supervision and policy learning, while self-distillation enhances the model's ability to model environmental feedback. With Qwen3-8B, SELF outperforms SDPO and GRPO by 6.4 and 4.1 percentage points in

τ

-bench success rate, and by 10.71 and 3.57 percentage points in AppWorld task goal completion, respectively. These results show that SELF uses environmental feedback more effectively within agentic self-distillation, improving agent capabilities.

Momentum

Related topics

Latest papers 41