Language Model Self-Improvement

Follow this topic Ask about this topic

The field studies how language models can improve their reasoning, instruction following, coding, and other skills by learning from their own attempts and interactions. Common approaches include self-training and self-distillation on generated responses, reinforcement learning and challenger–solver self-play, and test-time adaptation through memory, skills, or evolving prompts. Feedback ranges from exact checks such as code execution and formal proof to model-generated critiques, preferences, and confidence signals, with evaluation spanning math and coding benchmarks as well as agent and multimodal tasks. The central tension is whether these internal signals are reliable enough to produce transferable gains: noisy feedback, narrow curricula, and repeated selection can cause regressions or collapse, while oracle-supervised performance often remains difficult to match.

Latest papers

174 papers26 papers in the last four weeks, up 30% on the four weeks before. 0.2% of all new papers.

Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
Oct 29, 2026Lingkai Kong, Yuzhe Gu, Haiteng Zhao +6Language Model Self-Improvement RL for Language Model Reasoning
Shanghai AI Laboratory · Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong +2
A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement
Oct 28, 2026Yu Liu, Youxing Li, Yuedong Tan +17Language Model Self-Improvement Self-Training for Language Models
AudioRSI: Recursively Self-Improving Audio Reasoners
Oct 27, 2026Jiajun Fan, Roger Ren, Jingyuan Li +6Audio Reasoning Audio-Language Models
UIUC
Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges
Oct 26, 2026Jiayue Pu, Ang Gao, Jinyi Bai +9Recursive Self-Improvement Language Model Self-Improvement
Gaoling School of Artificial Intelligence, Renmin University of China · University of Chinese Academy of Sciences
Self-Evolution Reasoner: Continuous Optimization via Policy-Intrinsic Exploration and Exploitation
Oct 25, 2026Wenhang Shi, Yiren Chen, Zhe Zhao +3Language Model Self-Improvement RL for Language Model Reasoning
Renmin University of China · Peking University · Tencent
StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue
Oct 24, 2026Samarth Varshney, Ritu YadavInstruction Following Multi-Turn Dialogue Management
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
Oct 23, 2026Leon Sick, Lukas Hoyer, Yannick Strümpler +3Language Model Self-Improvement Self-Training for Language Models
Google · Ulm University
Recursive Code Evolution for Task-Oriented Semantic Communications
Oct 21, 2026Yan Gong, Zheng Chu, Wei Koong Chai +5Semantic Communication Large Language Model-Guided Optimization
School of Computing and Engineering, Bournemouth University, Fern Barrow, Poole, Dorset, BH12 5BB, United Kingdom · Department of Electrical and Electronic Engineering, University of Nottingham Ningbo China, Ningbo, China · School of Information Management, Wuhan University, Wuhan, China +3
MaD-RSI: Model-as-Data for Recursive Self-Improvement of Agent Harnesses
Oct 20, 2026Yu Cui, Hong He, Ruiqing Yue +8Agent Harness Optimization LLM Agent Harnesses
Beijing Institute of Technology · Tsinghua University · Chengdu Institute of Computer Applications, Chinese Academy of Sciences +4
Language Models Learn to Improve Language Models
Oct 16, 2026Minhua Lin, Zijun Wang, Zewen Liu +7Language Model Self-Improvement Large Language Model-Guided Optimization
The Pennsylvania State University · UC Santa Cruz · Emory University +2
Selection Is Not Learning: Choosing Model Futures
Oct 15, 2026Haoyi Yang, Kshitij Mishra, Nils Lukas +1Language Model Self-Improvement Self-Training for Language Models
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
ECG-RSI: Recursive Self-Improvement through Targeted Electrocardiogram Waveform Evolution
Oct 13, 2026Xiaoliang Wang, Zhuo Chen, Chuanyang Zheng +9Language Model Self-Improvement Electrocardiogram Classification
Zhejiang University · Ant Group · Shanghai Artificial Intelligence Laboratory +1
Self-Distillation with Verbal Regularization
Oct 12, 2026Jiazheng Li, Yawei Li, Junru Lu +7On-Policy Self-Distillation Language Model Self-Improvement
King’s College London · LMU Munich · Munich Center for Machine Learning +1
From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement
Oct 10, 2026Leijun Zhou, Yanke Yu, Zhihao Liu +10Recursive Self-Improvement LLM Agent Self-Improvement
PrismShadow · Peking University · Zhejiang University +1
Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization
Oct 9, 2026Jiaqi Tian, Ting LongLanguage Model Self-Improvement On-Policy Self-Distillation
School of Artificial Intelligence Jilin University China
Recursive Self-Improvement through Multi-Agent Self-Supervision
Oct 8, 2026Hyunin Lee, Jinglue Xu, Jeffrey Seely +4Language Model Self-Improvement LLM Agent Workflow Optimization
UC Berkeley · SakanaAI · Sakana AI
Learning to Plan by Looking Back: Hindsight Hierarchies for Training Reasoning Models
Oct 8, 2026Lars Simon, Holger Eble, Manuel RadonsLanguage Model Self-Improvement Large Reasoning Models
Bundesdruckerei GmbH Berlin, Germany
MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
Oct 8, 2026Xiaomi LLM-Core Team, :, Zongming Qiao +147Reinforcement Learning Language Model Self-Improvement
Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models
Oct 7, 2026Yi Wang, Rui Qian, Yu Li +2Language Model Self-Improvement On-Policy Distillation
ShanghaiTech University · Fudan University · Southeast University +1
The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules
Oct 7, 2026Litao Hu, Yutong TangLLM Self-Refinement Language Model Self-Improvement
Meta · Microsoft
SIGMA: Self-Improving Alignment Generalization from a Model Spec
Oct 6, 2026Jingyu Zhang, Shruti Palaskar, Daniel Khashabi +3LLM Safety Alignment LLM Agent Safety
Apple · Johns Hopkins University
ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement
Oct 5, 2026Xingbo Yao, Xiaoman Wang, Zhengwu Lei +11LLM Fine-Tuning LLM Post-Training
HKUST(GZ) · Xiaohongshu Inc. · ECNU +3
When Is Enough Enough in Self-Evolving LLM Systems?
Oct 3, 2026Enoch Yin, Bin Liu, Zhengling QiEarly Stopping Language Model Self-Improvement
Department of Statistics, The George Washington University · Department of Statistics and Data Science, School of Management, Fudan University · Department of Decision Sciences, School of Business, The George Washington University
Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
Oct 3, 2026Jinyuan Li, Chengsong Huang, Langlin Huang +4Language Model Self-Improvement Self-Training for Language Models
Washington University in St. Louis · University of Michigan, Ann Arbor
PAPER2LLM++: Continual Self-Evolution of LLMs from Research Papers
Oct 2, 2026Hongji Pu, Yilun Zhao, Wenpeng YinLanguage Model Self-Improvement Continual Learning for LLMs
UIUC · Yale University · Penn State
Prompt2Skill: Unsupervised Skill Optimization From Natural Language Instructions
Sep 29, 2026Bo Ni, Li Li, Ryan A. Rossi +2LLM Agent Skill Learning Language Model Self-Improvement
Vanderbilt University · University of Southern California · Adobe Systems
RLTLDR: Self-improvement by Internalizing Self-generated Feedback
Sep 29, 2026Michael Kirchhof, Eleonora Gualdoni, Andrew Szot +6RL for Code Generation Reinforcement Learning with Verifiable Rewards
Apple
Gödel Forest: Balancing Search Depth and Breadth for Data-Centric Recursive Self-Improvement
Sep 29, 2026Ziqi Zhao, Fanqing Meng, Haocheng Lu +4Language Model Self-Improvement Recursive Self-Improvement
Evolvent AI · The Hong Kong Polytechnic University · National University of Singapore +1
When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain
Sep 29, 2026Chenxu Wang, Chaozhuo Li, Xinze Shi +6Language Model Self-Improvement LLM Training
Beijing University of Posts and Telecommunications · Beijing Academy of Artificial Intelligence · Central South University +2
Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training
Sep 28, 2026Yuyang Deng, Yu Wang, Jiayun WangSelf-Training for Language Models Large Language Model-Guided Optimization
Accenture, Center for Advanced AI · Georgia Institute of Technology

Oct 29, 2026J/K move · Enter open · S save

Incentivizing Self-Learning in Large Language Models via Reinforcement Learning

Lingkai Kong, Yuzhe Gu, Haiteng Zhao, Zijian Wu+5

Shanghai AI Laboratory · Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong+2

Language Model Self-Improvement RL for Language Model Reasoning Long-Context Reasoning in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) draw on parametric knowledge and retrieval-augmented generation (RAG) to solve knowledge-intensive tasks. Yet problem solving in unfamiliar domains calls for tightly coupled knowledge acquisition and reasoning, beyond factual retrieval and aggregation or learning solely from a preassembled context. We study self-learning -- autonomously retrieving, studying, and applying knowledge from long-form reference materials on the fly. To investigate this capability, we formulate a textbook-based mathematical problem-solving task. LLMs must navigate textbook exposition, learn and apply relevant principles, and construct proofs for exercises, with reference solutions withheld. We introduce a BookNavigator interface to support this human-inspired study process and incentivize self-learning through reinforcement learning (RL) with fine-grained rewards. Experiments show that our method improves performance across in-domain and out-of-domain textbooks and subjects, raising Intern-S2-Preview-35B's overall proof accuracy from 51.5% to 79.5% and outperforming RL-trained textbook-free and fixed-retrieval baselines. These gains extend to complex long-context benchmarks involving different task types, with improvements over the base model of 1.7 and 2.3 percentage points on BrowseComp-Plus and CL-bench, respectively. Together, these findings suggest that RL can foster a domain-agnostic self-learning strategy, highlighting the potential of self-learning from long-form corpora to strengthen broader problem-solving capabilities on complex, long-context tasks.