Language Model Self-Improvement

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 17% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 119

All topics

The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules
Oct 7, 2026Litao Hu, Yutong TangLLM Self-Refinement Language Model Self-Improvement
Meta · Microsoft
ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement
Oct 5, 2026Xingbo Yao, Xiaoman Wang, Zhengwu Lei +11LLM Fine-Tuning LLM Post-Training
HKUST(GZ) · Xiaohongshu Inc. · ECNU +3
Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
Oct 3, 2026Jinyuan Li, Chengsong Huang, Langlin Huang +4Language Model Self-Improvement Self-Training for Language Models
Washington University in St. Louis · University of Michigan, Ann Arbor
Prompt2Skill: Unsupervised Skill Optimization From Natural Language Instructions
Sep 29, 2026Bo Ni, Li Li, Ryan A. Rossi +2LLM Agent Skill Learning Language Model Self-Improvement
Vanderbilt University · University of Southern California · Adobe Systems
RLTL;DR: Self-improvement by Internalizing Self-generated Feedback
Sep 29, 2026Michael Kirchhof, Eleonora Gualdoni, Andrew Szot +6RL for Code Generation Reinforcement Learning with Verifiable Rewards
Apple
Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training
Sep 28, 2026Yuyang Deng, Yu Wang, Jiayun WangSelf-Training for Language Models Large Language Model-Guided Optimization
Accenture, Center for Advanced AI · Georgia Institute of Technology
COEVO: Co-Evolving Context and Parameters for Recursive Self-Improvement
Sep 27, 2026Siwei Chen, Xinping Bao, Xinyu Cai +3RL for Language Models Prompt Optimization
Peking University · Emotional Machine
NLPG: Natural-Language Policy Gradients for Self-Evolving Language Agents
Sep 27, 2026Xu Liu, WenZhang Wei, Jun Cao +4Continual Learning for LLM Agents Policy Optimization
School of Electronic Information, Wuhan University, Wuhan 430079, China · School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
SeOPD: Self-Evolving LLMs via Online Policy Distillation from Self-Generated Chain-of-Thought
Sep 27, 2026Xiaoshu Chen, Sihang Zhou, Ke Liang +1Language Model Self-Improvement Self-Training for Language Models
National University of Defense Technology
Exact Feedback Is Not Control: Evaluating Text-based Closed-Loop Revision in LLMs
Sep 23, 2026Haitong Jiang, Chunlin Liu, Yile Wang +1LLM Self-Correction Controllable Text Generation
College of Computer Science and Software Engineering, Shenzhen University
STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution
Sep 16, 2026Yajie Yu, Mark Lee, Yue FengSelf-Training for Language Models RL for Language Model Reasoning
School of Computer Science University of Birmingham Birmingham United Kingdom
Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement
Sep 15, 2026Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang +1Continual Learning for LLMs Memory-Augmented Language Models
Department of Computer Science and Information Engineering, National Taiwan University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · AI Research Center (AINTU), National Taiwan University, Taiwan
Data-Efficient Language Modeling: From Frontier Advancement to Principle-Guided Model Improvement
Sep 12, 2026Shuxing Yang, Kaihao Zhu, Junjie Yang +13Efficient Language Model Training Language Modeling
College of Information Science and Electronic Engineering, Zhejiang University · Qiushi Engine Team
The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement
Sep 10, 2026Yi Duan, Ying Liu, Zirui Tang +32Self-Improving Agents Language Model Self-Improvement
Shanghai Jiao Tong University · Theseus Labs · Agent-Native Research Lab +6
Negative Self-Distillation: Learning to Reason by Avoiding Flaws
Sep 10, 2026Rongcan Pei, Zhepei Wei, Shuyao Xu +3Language Model Distillation Self-Distillation
Department of Computer Science, University of Virginia · Stanford University
Harness-agnostic detection and immunization of reward hacking in self-evolving language models
Sep 7, 2026Rongxin Yang, Yang Liu, Shang Luo +10Reward Hacking Language Model Self-Improvement
1Fullive-AI · 2Peking University · 4Nanyang Technological University +2
MetaRSI / RSI2: A Meta-Recursive Self-Improving System for Recursive Self-Improving Systems Themselves
Sep 6, 2026Zihan Tan, Leixin Sun, Zitong Shi +28Meta-Learning Agent Harness Optimization
Hugging Face
FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience
Sep 3, 2026Zixun Huang, Kishan Panaganti, Haitao Mi +1RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Tencent HY LLM Frontier · University of Pennsylvania
StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?
Sep 1, 2026Yinghao Chen, Zixi Chen, Bingxiang He +7Benchmark Design Language Model Self-Improvement
Tsinghua University · Zhejiang University
Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Models
Sep 1, 2026Xincheng Wei, Yifan Ding, Fucheng Xiong +6Self-Training for Language Models Language Model Self-Improvement
The Chinese University of Hong Kong, Shenzhen · Meituan, LongCat Team · Peking University +2
Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence
Aug 31, 2026Zhiqin Yang, Jingwen Fu, Yuhan Liu +16Reinforcement Learning RL for Language Model Reasoning
The Hong Kong University of Science and Technology · Zhongguancun Academy · Xi’an Jiaotong University +6
Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement
Aug 31, 2026Yi Ding, Ruqi ZhangRL for Language Models Language Model Distillation
Department of Computer Science, Purdue University, USA
WebWorld: The Browser as a World Model for Self-Improving Web Code
Aug 31, 2026Jiajun Wu, Jian Yang, Yaxin Du +7Self-Training for Language Models Code Generation
Beihang University · Shanghai Jiao Tong University · IQuest Research +1
DataFoundry: Evolving Data Preparators via Recursive Self-Improvement
Aug 30, 2026Cehao Yang, Xiaojun Wu, Xueyuan Lin +4Training Data Curation Domain Adaptation for LLMs
1IDEA Research, International Digital Economy Academy · 2Hong Kong University of Science and Technology (Guangzhou) · 3DataArc Tech Ltd.
J-Zero: Unified Challenger--Solver--Judge Self-Evolution from Zero Data
Aug 27, 2026Gyouk Chu, Myeongho Jeon, Teresa Yeo +1LLM-as-a-Judge Self-Training for Language Models
KAIST
DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution
Aug 12, 2026Siheng Xiong, Ali Payani, Oguzhan Gungordu +1Language Model Self-Improvement
Georgia Institute of Technology · Cisco Research
Dual-Loop Self-Evolution via Verifiable Emotion Feedback for Multi-Turn Empathetic Dialogue
Aug 11, 2026Yi Wei, Shuo Jiang, Huaixia Dou +5Reinforcement Learning with Verifiable Rewards Curriculum Learning
Qwen DianJin Team, Alibaba Cloud Computing · Beihang University · School of Computer Science and Technology, Soochow University
Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA
Aug 10, 2026Mind Lab, :, Vin Bo +74Continual Learning for LLM Agents Agentic RL
On-Policy Self-Distillation without Any Supervision
Aug 6, 2026Yijiang Li, Bingyang Wang, Yijun Liang +3On-Policy Self-Distillation Language Model Self-Improvement
UC San Diego · Georgia Institute of Technology · University of Maryland, College Park +1
Evaluating and Improving Pedagogical Fit in LLM-Based AI Tutors with the Pedagogical Suitability Index
Aug 5, 2026Benjamin Barlog, Hudson Craig, Zedong PengLLM Evaluation LLM Alignment
Department of Computer Science University of Montana Missoula, MT, USA

Oct 7, 2026, cs.AIJ/K move · Enter open · S save

The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules

Litao Hu, Yutong Tang

Meta · Microsoft

LLM Self-Refinement Language Model Self-Improvement LLM Evaluation

Open paper AskPDF arXiv

Self-improving LLM systems propose changes to themselves and keep those that score better on a small evaluation set. We treat this keep-if-better step as selection under measurement noise, model the correlated errors of the candidates in a single decision, and study empirically what happens when the evaluation set is reused. In runs where Qwen models rewrite their own instructions and every candidate is also scored on 600 held-out items, most proposals after the first are harmful, and the model gives the size of the winner's curse of a generation's best candidate. With a prior from a separate pilot, it matches the average overstatement of first-generation commits in native loops, though not setting by setting. In a pre-registered study, the final selection-set score of greedy loops exceeded held-out accuracy by 13 to 20 points with 16 selection items and by 1 to 5 points with 256. Held-out gains grew with the selection set on TREC but not on GSM8K, and the tested acceptance rules did not beat greedy acceptance over whole runs. Gains measured on the selection set also exceeded held-out gains when a current model refined a competent instruction, and in the validation scores of GEPA and MIPROv2. Scoring the starting and the current instruction on 64 items never used for selection removes the average bias of a loop's reported gain, but single estimates remain off by about 6 points. Self-improvement studies should report held-out gains with their uncertainty.

Momentum

Related topics

Latest papers 119