Language Model Self-Improvement

Follow this topic Ask about this topic

The field studies how language models can improve their reasoning, instruction following, coding, and other skills by learning from their own attempts and interactions. Common approaches include self-training and self-distillation on generated responses, reinforcement learning and challenger–solver self-play, and test-time adaptation through memory, skills, or evolving prompts. Feedback ranges from exact checks such as code execution and formal proof to model-generated critiques, preferences, and confidence signals, with evaluation spanning math and coding benchmarks as well as agent and multimodal tasks. The central tension is whether these internal signals are reliable enough to produce transferable gains: noisy feedback, narrow curricula, and repeated selection can cause regressions or collapse, while oracle-supervised performance often remains difficult to match.

Latest papers

174 papers26 papers in the last four weeks, up 30% on the four weeks before. 0.2% of all new papers.

Learning to Self-Evolve
Mar 19, 2026Xiaoyin Chen, Canwen Xu, Yite Wang +3Language Model Self-Improvement RL for Language Models
Mila – Quebec AI Institute · University of Montreal · Snowflake
Continually self-improving AI
Mar 18, 2026Zitong YangLanguage Model Self-Improvement Self-Training for Language Models
Department of Statistics, Stanford University
Online Experiential Learning for Language Models
Mar 17, 2026Tianzhu Ye, Li Dong, Qingxiu Dong +3Continual Learning for LLMs Language Model Self-Improvement
Microsoft Research
GASP: Guided Asymmetric Self-Play For Coding LLMs
Mar 16, 2026Swadesh Jana, Cansu Sancaktar, Tomáš Daniš +3Language Model Self-Improvement LLM Code Generation
University of Tübingen · Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen +1
Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Data
Mar 10, 2026Hyunji Nam, Haoran Li, Natasha JaquesLLM Personalization Mutual Information Maximization
Stanford University · Character AI · University of Washington
A Dynamic Self-Evolving Extraction System
Mar 6, 2026Moin Amin-Naseri, Hannah Kim, Estevam HruschkaDocument Information Extraction LLM Information Extraction
Megagon Labs
Preventing Curriculum Collapse in Self-Evolving Reasoning Systems
Mar 3, 2026Vaibhav MishraMathematical Problem Generation Language Model Self-Improvement
Test-Time Meta-Adaptation with Self-Synthesis
Mar 3, 2026Zeyneb N. Kaya, Nick RuiInference-Time Adaptation of Language Models Test-Time Adaptation
Stanford University
CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production
Mar 2, 2026Yixin Nie, Lin Guan, Zhongyao Ma +19LLM Post-Training Conversational Agents
Meta Superintelligence Labs · OpenAI · xAI +1
Provable and Practical In-Context Policy Optimization for Self-Improvement
Mar 2, 2026Tianrun Yu, Yuxiao Yang, Zhaoyang Wang +6Language Model Self-Improvement Test-Time Scaling
Brigham Young University · University of North Carolina at Chapel Hill · Microsoft
Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences
Feb 25, 2026Sweta Karlekar, Carolina Zheng, Magnus Saebo +5Large Language Model-Guided Optimization Test-Time Scaling
Columbia University · Los Alamos National Laboratory
Improving Interactive In-Context Learning from Natural Language Feedback
Feb 17, 2026Martin Klissarov, Jonathan Cook, Diego Antognini +5In-Context Learning Language Model Self-Improvement
Google DeepMind
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
Feb 16, 2026Lunjun Zhang, Ryan Chen, Bradly C. StadieLanguage Model Self-Improvement RL for Language Models
Department of Computer Science, University of Toronto · Department of Statistics and Data Science, Northwestern University · Bridgewater AIA Labs
TTSR: Test-Time Self-Evolving via Reflection
Feb 6, 2026Haoyang He, Zihua Rong, Yunjia Zhao +3LLM Self-Refinement Test-Time Optimization
Beijing University of Posts and Telecommunications · Southwestern University of Finance and Economics · China Unicom Online Information Technology Co., Ltd.
TTCS: Test-Time Curriculum Synthesis for Self-Evolving
Jan 30, 2026Chengyi Yang, Zhishang Xiang, Yunbo Tang +5Test-Time Training Language Model Self-Improvement
Xiamen University · Washington University in St. Louis · Renmin University of China
Self-Improvement as Coherence Optimization: A Theoretical Account
Jan 20, 2026Tianyi Qiu, Ahmed Hani Ismail, Zhonghao He +1Language Model Self-Improvement
Peking University · University of Oxford · UC Berkeley +1
OpenSIR: Open-Ended Self-Improving Reasoner
Nov 1, 2025Wai-Chung Kwan, Joshua Ong Jun Leang, Pavlos Vougiouklis +3Self-Play RL RL for Language Model Reasoning
University of Edinburgh · Imperial College London · Huawei Technologies Research & Development (UK) Limited +1
A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
Oct 21, 2025Mengqi Li, Lei Zhao, Anthony Man-Cho So +2Self-Training for Language Models Mathematical Reasoning
The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University · The Chinese University of Hong Kong
RFG: Self-Improving Diffusion Large Language Models with Reward-Free Guidance
Sep 29, 2025Tianlang Chen, Minkai Xu, Jure Leskovec +1Diffusion Guidance Diffusion Language Model Inference
Stanford University
Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation
Sep 18, 2025Yujun Zhou, Zhenwen Liang, Haolin Liu +7Language Model Self-Improvement RL for Language Models
Tencent AI Lab · University of Notre Dame · University of Virginia
Discovering New Theorems via LLMs with In-Context Proof Learning in Lean
Sep 16, 2025Kazumi Kasaura, Naoto Onda, Yuta Oriike +3Automated Theorem Proving In-Context Learning
OMRON SINIC X Corporation · RIKEN AIP · AutoRes +7
ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search
Aug 27, 2025Sining Zhoubian, Dan Zhang, Jie TangRL for Code Generation RL for Language Model Reasoning
Department of Computer Science and Technology, Tsinghua University · Work done when interned at Z.ai. · School of Electronics and Computer Scinence, Univeristy of Southampton
SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning
Jul 22, 2025Xiuwei Chen, Wentao Hu, Hanhui Li +9Synthetic Data Generation Multimodal Large Language Models
Sun Yat-sen University · The Hong Kong Polytechnic University · MBZUAI +2
RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
Oct 31, 2024Fu-Chieh Chang, Yu-Ting Lee, Hui-Ying Shih +2Reinforcement Learning RL for Language Model Reasoning
MediaTek Research, Taipei, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Department of Mathematical Sciences, National Chengchi University, Taipei, Taiwan +2

Mar 19, 2026J/K move · Enter open · S save

Learning to Self-Evolve

Xiaoyin Chen, Canwen Xu, Yite Wang, Boyi Liu+2

Mila – Quebec AI Institute · University of Montreal · Snowflake

Language Model Self-Improvement RL for Language Models

Open paper AskPDF arXiv

We introduce Learning to Self-Evolve (LSE), a reinforcement learning framework that trains large language models (LLMs) to improve their own contexts at test time. We situate LSE in the setting of test-time self-evolution, where a model iteratively refines its context from feedback on seen problems to perform better on new ones. Existing approaches rely entirely on the inherent reasoning ability of the model and never explicitly train it for this task. LSE reduces the multi-step evolution problem to a single-step RL objective, where each context edit is rewarded by the improvement in downstream performance. We pair this objective with a tree-guided evolution loop. On Text-to-SQL generation (BIRD) and general question answering (MMLU-Redux), a 4B-parameter model trained with LSE outperforms self-evolving policies powered by GPT-5 and Claude Sonnet 4.5, as well as prompt optimization methods including GEPA and TextGrad, and transfers to guide other models without additional training. Our results highlight the effectiveness of treating self-evolution as a learnable skill.