Language Model Self-Improvement

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 17% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 119

All topics

Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Jun 2, 2026Ali Behrouz, Farnoosh Hashemi, Adel Javanmard +1Continual Learning for LLMs Persistent Memory for Language Models
Google
A Theoretical Framework for Self-Play Theorem Proving Algorithms
Jun 1, 2026Thomas Chen, Zhiyuan LiAutomated Theorem Proving Language Model Self-Improvement
Learning from Saturated Data: Signals Beyond Correctness for LLM Training
May 31, 2026Hanno Hiss, Jasper Dekoninck, Martin VechevLanguage Model Self-Improvement LLM Training
ETH Zurich
On the Generalization Gap in Self-Evolving Language Model Reasoning
May 31, 2026Zhenting Qi, Susanna Maria Baby, Stefanie Anna Baby +5LLM Self-Refinement Language Modeling
Google Research · Harvard University · Google +1
SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Tasks
May 29, 2026Wai-Chung Kwan, Aryo Pradipta Gema, Joshua Ong Jun Leang +1Open-Ended Generation Self-Play RL
University of Edinburgh · Imperial College London
Not All Synthetic Data Is Yours to Learn From
May 29, 2026Sina Alemohammad, Li Chen, Richard G. Baraniuk +1Synthetic Data Pretraining Memorization in Language Models
ECE Department The University of Texas at Austin · Apple · ECE Department Rice University
Self-Trained Verification for Training- and Test-Time Self-Improvement
May 28, 2026Chen Henry Wu, Aditi RaghunathanLLM Self-Refinement Language Model Self-Improvement
Carnegie Mellon University
Self-Improving Language Models with Bidirectional Evolutionary Search
May 27, 2026Guowei Xu, Zhenting Qi, Huangyuan Su +4Evolutionary Optimization LLM Planning
1Harvard University · 2MIT
CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning
May 27, 2026Linas Nasvytis, Simon Jerome Han, Ben Prystawski +3LLM Self-Refinement Efficient Language Model Reasoning
Stanford University
Confidence-Orchestrated Self-Evolution against Uncertain LLM Feedback
May 27, 2026Bowen Wei, Nan Wang, Yuqing Zhou +2Self-Training for Language Models RL for Language Model Reasoning
George Mason University · Amazon
SIA: Self Improving AI with Harness & Weight Updates
May 26, 2026Prannay Hebbar, Yogendra Manawat, Samuel Verboomen +4Agent Harness Optimization LLM Agent Self-Improvement
Hexo Labs Palo Alto · Hexo Labs Brussels · University of Oxford +1
Better, Faster: Harnessing Self-Improvement in Large Reasoning Models
May 24, 2026Qihuang Zhong, Liang Ding, Juhua Liu +3Overthinking in Language Models RL for Language Model Reasoning
1Nanyang Technological University, Singapore · 2Alibaba Group · School of Computer Science, Wuhan University, China +1
Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap
May 23, 2026Tianlang Chen, Shirley Wu, Jure LeskovecLanguage Model Distillation Teacher-Student Learning
Stanford
Model Collapse as Cultural Evolution
May 21, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuCultural Evolution Self-Training for Language Models
The University of Hong Kong · Hong Kong, China · Stellaris AI Limited
Self-Policy Distillation via Capability-Selective Subspace Projection
May 21, 2026Guangya Hao, Yitong Shang, Yunbo Long +2Self-Training for Language Models Language Model Distillation
University of Cambridge · 2HKUST · University of Chicago
One-Way Policy Optimization for Self-Evolving LLMs
May 21, 2026Shuo Yang, Jinda Lu, Kexin Huang +6Policy Optimization Reinforcement Learning with Verifiable Rewards
1Shenzhen Graduate School, Peking Uni · 2Qwen Pilot Team · 2Qwen Pilot Team 3Dartmouth College +1
Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
May 20, 2026Tony Lee, Percy LiangSelf-Training for Language Models Language Model Self-Assessment
Stanford University
RISE: Reliable Improvement in Self-Evolving Vision-Language Models
May 20, 2026Chaoran Xu, Yingmao Miao, Pengfei Zhang +3Vision-Language Models VLM Adaptation
AMAP, Alibaba Group
Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies
May 20, 2026Ming LiuSelf-Training for Language Models Model Collapse
Amazon
LC-ERD: Mining Latent Logic for Self-Evolving Reasoning via Consistency-Regulated Reward Decomposition
May 19, 2026Yanyu Chen, Jiyue Jiang, Dianzhi Yu +8RL for Language Model Reasoning Process Supervision
The Chinese University of Hong Kong · Shanghai Jiaotong University · Fudan University
Test-Time Learning with an Evolving Library
May 14, 2026Weijia Xu, Alessandro Sordoni, Chandan Singh +4Memory-Augmented Language Models Continual Test-Time Adaptation
Microsoft Research
Learning to Build the Environment: Self-Evolving Reasoning RL via Verifiable Environment Synthesis
May 14, 2026Yucheng Shi, Zhenwen Liang, Kishan Panaganti +3Reinforcement Learning RL for Language Model Reasoning
Tencent HY LLM
Query-Conditioned Test-Time Self-Training for Large Language Models
May 13, 2026Chaehee Song, Minseok Seo, Yeeun Seong +2Test-Time Adaptation Test-Time Optimization
School of Electrical Engineering, KAIST · 2Graduate School of Green Growth and Sustainability, KAIST
STRIDE: Learnable Stepwise Language Feedback for LLM Reasoning
May 13, 2026Junjie Zhang, Guozheng Ma, Shunyu Liu +5RL for Language Model Reasoning Process Supervision
Generative AI Lab, College of Computing and Data Science, Nanyang Technological University, Singapore 639798 · Tongyi Lab, Alibaba Group
ICRL: Learning to Internalize Self-Critique with Reinforcement Learning
May 13, 2026Jianbo Lin, Xiaomin Yu, Yi Xin +7RL for Language Models LLM Self-Refinement
Hong Kong University of Science and Technology (Guangzhou) · Nanjing University · Sun Yat-sen University +3
AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions
May 13, 2026Ishika Agarwal, Sofia Stoica, Emre Can Acikgoz +4Synthetic Data Generation Synthetic Data Generation for Language Models
University of Illinois Urbana-Champaign1 · Amazon2
NOVA: Fundamental Limits of Knowledge Discovery Through AI
May 12, 2026Salman Avestimehr, Ken Duffy, Muriel MédardAdaptive Sampling Autonomous Scientific Discovery
University of Southern California · Northeastern University · Massachusetts Institute of Technology
Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation
May 12, 2026Yuwei Zhang, Sha Li, Changlong Yu +9Continual Learning for LLMs Self-Distillation
UC San Diego1 · Amazon2 · Georgia Institute of Technology3
Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
May 12, 2026Qihuang Zhong, Liang Ding, Wenjie Xuan +3Multimodal Large Language Models Multimodal Reasoning
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China · The University of Sydney, Australia · Nanyang Technological University, Singapore
Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling
May 11, 2026Yizhu Jiao, Ruixiang Zhang, Richard Bai +3RL for Code Generation Test-Time Scaling
Apple · University of Illinois Urbana-Champaign

Jun 2, 2026, cs.LGJ/K move · Enter open · S save

Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

Ali Behrouz, Farnoosh Hashemi, Adel Javanmard, Vahab Mirrokni

Google

Continual Learning for LLMs Persistent Memory for Language Models Replay-Based Continual Learning Language Model Self-Improvement Knowledge Distillation

Open paper AskPDF arXiv

The past few decades have witnessed significant advances in the design of machine learning algorithms, from early studies on task-specific shallow models to more general deep Large Language Models (LLMs). Despite showing promising results in tasks that require instant prediction or in-context learning, existing models lack the ability to continually learn and effectively transfer their temporal in-context knowledge to their long-term parameters. Inspired by human learning process, we introduce a ''Sleep'' paradigm that allows the models to continually learn, distill their short-term fragile memories into stable long-term knowledge with replay, and recursively improve themselves with ''Dreaming'' process. In more detail, sleep consists of two stages: (1) Memory Consolidation: an upward distillation process, called Knowledge Seeding, where the memories of a smaller-self are distilled into a larger network to provide more capacity while preserving the knowledge. As a proof of concept, we present a new Generalized Distillation process for {Knowledge Seeding} (i.e., the combination of on-policy distillation with Reinforcement Learning (RL)-based imitation learning); (2) Dreaming: a self-improvement phase, where the model uses RL to generate a curriculum of synthetic data to rehearse new knowledge and refine existing capabilities without human supervision. Our experiments on long-horizon, continual learning, knowledge incorporation, and few-shot generalization tasks support the importance of the sleep stage.

Momentum

Related topics

Latest papers 119