Language Model Self-Improvement

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 17% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 119

All topics

Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agents
May 11, 2026Zhiyuan Fan, Wenwei Jin, Feng Zhang +4Reinforcement Learning Language Model Self-Improvement
1Xiaohongshu Inc. · School of Computer Science, Peking University
G-Zero: Self-Play for Open-Ended Generation from Zero Data
May 11, 2026Chengsong Huang, Haolin Liu, Tong Zheng +7Open-Ended Generation Self-Play RL
Washington University in St. Louis · University of Virginia · University of Maryland +1
Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning
May 8, 2026Seohyun Lee, Wenzhi Fang, Dong-Jun Han +2LLM Fine-Tuning Language Model Self-Improvement
Purdue University · Yonsei University · University at Buffalo-SUNY
SEIF: Self-Evolving Reinforcement Learning for Instruction Following
May 8, 2026Qingyu Ren, Qianyu He, Jiajie Zhu +7RL for Language Models Reinforcement Learning
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University · School of Data Science, Fudan University · Ant Group
EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics
May 5, 2026Shuyue Stella Li, Rui Xin, Teng Xiao +8RL for Language Models LLM Evaluation
University of Washington · 2Allen Institute for AI · University of Pennsylvania
Iterative Finetuning is Mostly Idempotent
May 1, 2026Zephaniah Roe, Jack Sanderson, Dang Nguyen +5Supervised Fine-Tuning LLM Post-Training
University of Chicago
From Context to Skills: Can Language Models Learn from Context Skillfully?
Apr 30, 2026Shuzheng Si, Haozhe Zhao, Yu Lei +10In-Context Learning Agent Skill Learning
THU · UIUC · DeepLang AI +2
IRIS: Interpolative Rényi Iterative Self-play for Large Language Model Fine-Tuning
Apr 22, 2026Wenjie Liao, Like Wu, Liangjie Zhao +2Fine-Tuning LLM Fine-Tuning
Graduate School of Information, Production and Systems, Waseda University, Fukuoka, Japan · Institute of Computing Technology, University of the Chinese Academy of Sciences
Scaling Self-Play with Self-Guidance
Apr 22, 2026Luke Bailey, Kaiyue Wen, Kefan Dong +2Self-Play RL Automated Theorem Proving
Stanford University
Self-Improving Tabular Language Models via Iterative Reward-Guided Post-Training
Apr 21, 2026Yunbo Long, Tejumade Afonja, Guangya Hao +2Tabular Foundation Models Group Relative Policy Optimization
Department of Engineering, University of Cambridge · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany · The Alan Turing Institute, London
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Apr 20, 2026Yongrui Heng, Chaoya Jiang, Han Yang +2Synthetic Data Generation Multimodal Large Language Models
School of Control Science and Engineering, Shandong University Jinan, China · Zeekr, Geely Auto Shanghai, China
SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expression
Apr 20, 2026Shaowei Zhang, Faqiang Qian, Yan Chen +5RL for Language Models Emotion Recognition in Conversations
1SenseTime · 2Tsinghua University · 3Shanghai Jiao Tong University +1
Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning
Apr 19, 2026Ziqing Zhuang, Linhai Zhang, Jiasheng Si +2Continual Learning for LLMs Meta-Learning
Southeast University · King’s College London · Qilu University of Technology (Shandong Academy of Sciences) +1
Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
Apr 19, 2026Zhiyin Yu, Bo Zhang, Qibin Hou +3Self-Training for Language Models RL for Language Model Reasoning
Peking University · Shanghai Artificial Intelligence Laboratory · Nankai University +1
ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization
Apr 17, 2026Yixu Huang, Xinglei Yu, Zhongyu WeiExecution-Guided Code Generation Language Model Self-Improvement
School of Data Science Fudan University Shanghai, China
Peer-Predictive Self-Training for Language Model Reasoning
Apr 14, 2026Shi Feng, Hanlin Zhang, Fan Nie +2Self-Training for Language Models Language Model Post-Training
Harvard University · Stanford University
Embarrassingly Simple Self-Distillation Improves Code Generation
Apr 1, 2026Ruixiang Zhang, Richard He Bai, Huangjie Zheng +3Language Model Decoding Language Model Distillation
Apple
Online Experiential Learning for Language Models
Mar 17, 2026Tianzhu Ye, Li Dong, Qingxiu Dong +3Continual Learning for LLMs Language Model Self-Improvement
Microsoft Research
Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Data
Mar 10, 2026Hyunji Nam, Haoran Li, Natasha JaquesLLM Personalization Mutual Information Maximization
Stanford University · Character AI · University of Washington
TTSR: Test-Time Self-Evolving via Reflection
Feb 6, 2026Haoyang He, Zihua Rong, Yunjia Zhao +3LLM Self-Refinement Test-Time Optimization
Beijing University of Posts and Telecommunications · Southwestern University of Finance and Economics · China Unicom Online Information Technology Co., Ltd.
Self-Improvement as Coherence Optimization: A Theoretical Account
Jan 20, 2026Tianyi Qiu, Ahmed Hani Ismail, Zhonghao He +1Language Model Self-Improvement
Peking University · University of Oxford · UC Berkeley +1
OpenSIR: Open-Ended Self-Improving Reasoner
Nov 1, 2025Wai-Chung Kwan, Joshua Ong Jun Leang, Pavlos Vougiouklis +3Self-Play RL RL for Language Model Reasoning
University of Edinburgh · Imperial College London · Huawei Technologies Research & Development (UK) Limited +1
A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
Oct 21, 2025Mengqi Li, Lei Zhao, Anthony Man-Cho So +2Self-Training for Language Models Mathematical Reasoning
The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University · The Chinese University of Hong Kong
RFG: Self-Improving Diffusion Large Language Models with Reward-Free Guidance
Sep 29, 2025Tianlang Chen, Minkai Xu, Jure Leskovec +1Diffusion Guidance Diffusion Language Model Inference
Stanford University
Discovering New Theorems via LLMs with In-Context Proof Learning in Lean
Sep 16, 2025Kazumi Kasaura, Naoto Onda, Yuta Oriike +3Automated Theorem Proving In-Context Learning
OMRON SINIC X Corporation · RIKEN AIP · AutoRes +7
ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search
Aug 27, 2025Sining Zhoubian, Dan Zhang, Jie TangRL for Code Generation RL for Language Model Reasoning
Department of Computer Science and Technology, Tsinghua University · Work done when interned at Z.ai. · School of Electronics and Computer Scinence, Univeristy of Southampton
SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning
Jul 22, 2025Xiuwei Chen, Wentao Hu, Hanhui Li +9Synthetic Data Generation Multimodal Large Language Models
Sun Yat-sen University · The Hong Kong Polytechnic University · MBZUAI +2
RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
Oct 31, 2024Fu-Chieh Chang, Yu-Ting Lee, Hui-Ying Shih +2Reinforcement Learning RL for Language Model Reasoning
MediaTek Research, Taipei, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Department of Mathematical Sciences, National Chengchi University, Taipei, Taiwan +2
EVOQUANT: Self-Evolving Verifier-Guided Strategy Optimization for Robust Quantitative Trading
Date pendingJie Mao, Changlun Li, Xiang Li +6Large Language Model-Guided Optimization Language Model Self-Improvement
The Hong Kong University of Science and Technology (Guangzhou) · Paradoox AI Research

May 11, 2026, cs.AIJ/K move · Enter open · S save

Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agents

Zhiyuan Fan, Wenwei Jin, Feng Zhang, Bin Li+3

1Xiaohongshu Inc. · School of Computer Science, Peking University

Reinforcement Learning Language Model Self-Improvement RL for LLM Agents Self-Evolving Agents

Open paper AskPDF arXiv

Experience-driven self-evolving agents aim to overcome the static nature of large language models by distilling reusable experience from past interactions, thus enabling adaptation to novel tasks at deployment time. This process places substantial demands on the foundation model's capacities for abstraction, generalization, and in-context learning. However, most existing studies focus primarily on system-level design choices, such as how experience is represented and managed, neglecting the inherent capabilities of the underlying model. While some recent works have started to optimize the experience utilization stage via reinforcement learning, they still fail to treat self-evolution as a unified process to be jointly optimized. To this end, we propose Evolving-RL, an efficient algorithmic framework that jointly improves the experience extraction and utilization capabilities required for self-evolution. Specifically, we center the learning process on experience extraction and evaluation, using the two supervisory signals derived from evaluation to optimize the extractor and solver separately and thus enable their coordinated co-evolution. Experiments on ALFWorld and Mind2Web show that Evolving-RL effectively enhances LLMs' ability to extract and reuse experience, leading to strong performance gains on out-of-distribution tasks (up to 98.7% relative improvement over the GRPO baseline on ALFWorld unseen tasks and 35.8% on Mind2Web), and these gains are fully unlocked only through the coordinated co-evolution of experience extraction and utilization. Furthermore, Evolving-RL inherently functions as an experience-augmented RL algorithm. By internalizing reusable experience patterns directly into model parameters, it achieves remarkable performance gains over standard baselines on both seen and unseen tasks, even in the absence of test-time experience accumulation.

Momentum

Related topics

Latest papers 119