From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement

Leijun Zhou; Yanke Yu; Zhihao Liu; Qu Xiang; Tiancheng Hu; Yiru Dong; Yihan Wu; Yangxi Xiang

cs.AIOct 10, 2026

From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement

Authors: Leijun Zhou, Yanke Yu, Zhihao Liu, Qu Xiang, Tiancheng Hu, Yiru Dong, Yihan Wu, Yangxi Xiang,

Organizations: PrismShadow · Peking University · Zhejiang University · New York University

Abstract

Recursive self-improvement (RSI) offers a route to AI systems that participate in their own continued development. Recent methods allow language agents to retain changes to their parameters, memory, skills, and code for subsequent execution and improvement. We examine how these methods can inform new research ideas through two complementary perspectives. First, we introduce a unified taxonomy with nine design dimensions organized into four groups: update targets, update organization, update schedule and scope, feedback and selection. This taxonomy places existing methods in a shared design space, where examining individual dimensions and their interactions can inspire new research ideas. Second, we examine how existing RSI methods draw on traditional deep learning, for example through optimization, regularization, data augmentation, continual learning, and adversarial learning. We develop further ideas by examining more specific learning mechanisms and drawing on evolutionary computation. We also review how existing benchmarks assess self-improvement under online, offline, and offline-to-online settings. Together, these perspectives help researchers develop new RSI research ideas from existing designs and learning mechanisms. RSI papers and resources are available at https://prism-shadow.github.io/awesome-rsi/.

Figures & tables

Figure 1: Overview of the survey. Agent resources form the trunk, four groups of design dimensions organize the canopy, and traditional deep learning supplies the roots. Execution traces and feedback inform self-improvement. New growth represents research opportunities arising from the taxonomy and learning mechanisms.

Figure 2: The unified taxonomy and its use as a design space. Left: four groups organize nine design dimensions. Right: an illustrative slice locates existing designs and a candidate direction within the same space.

Figure 3: Methods grouped by RSI artifact. Papers with multiple labels appear on multiple branches.

Figure 4: Connections between traditional deep learning and RSI, illustrated by representative methods. Each panel links a learning mechanism to its use in RSI; the regularization panel includes both complexity control and temporary masking. Network sketches illustrate the learning principles; SkillEvoReg masks skill instructions, while SkillProx simplifies skill contents.

Figure 5: Broadening the sources of ideas for RSI. Left: the lottery ticket hypothesis, adversarial dropout, and maintaining plasticity in deep learning. Center: Bayesian optimization. Right: evolution of modularity, illustrated by revising production or retrieval modules to adapt to new tasks. Bottom arrows indicate how these sources can inspire RSI research directions.

Study	Domain	Evaluation setup	Feedback for improvement	Evaluation focus
Online
Evo-Memory [ 68 ]	Reasoning, coding, and interactive tasks.	Streams with varying task difficulty and order.	Correctness signals after execution.	Performance gains from using earlier task records.
SkillFlow [ 69 ]	Finance, healthcare, documents.	Related tasks in increasing difficulty.	Execution traces and verifier feedback.	Skill acquisition, repair, and reuse.
AgentStream [ 70 ]	Web research, coding, and tool use.	Isolated, sequential, or interleaved domains.	Execution results and self-evaluation.	Transfer and interference across domains.
PATH-Bench [ 71 ]	Code generation and tool use.	Probes repeated after helpful or interfering examples.	Execution records between probes.	Transfer and forgetting.
Offline

Table 1: Comparison of representative RSI benchmarks and evaluation settings.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

P: parameters; H: harness code; C: context; M: memory; S: skills; O: other artifacts. Off/On: offline/online.
Method	Targets	Organization Updater / topology	Schedule and scope Mode / frequency / scope	Feedback Source / type	Acceptance	Update mechanism
A-MEM [ Xu et al., 2025 ]	M	Teacher Sequential	On Event General	LLM feedback Other	No validation	Turns each new memory into a structured note and lets it revise summaries, attributes, and links of related historical memories in a Zettelkasten-style graph.
ACE [ Zhang et al., 2025b ]	C, M, S	Teacher Sequential	Off, On Trajectory, Batch General, Specialized	Train/dev set, Benchmark, Environment, Executable verifier, LLM feedback Binary, Non-binary, Ground truth, LLM-as-a-judge, Other	No validation	Turns execution feedback into incremental updates to a structured playbook, preserving useful strategies while revising and deduplicating experience for later tasks.
ACE (adversarial) [ Huang et al., 2026b ]	P	Joint Sequential	Off Batch Specialized	Train/dev set, Executable verifier Binary, Non-binary	No validation	Updates solver and adversary adapters across rounds, using selected programs to train the solver and execution-derived preferences to train adversarial test generation.
ACON [ Kang et al., 2025 ]	C	Teacher Sequential	Off Batch Specialized	Train/dev set, Environment, Executable verifier, LLM feedback Binary, Non-binary, LLM-as-a-judge, Other	Benchmark score, Combined metrics	Contrasts successful full-context runs with failed compressed-context runs to revise compression guidelines, selecting candidates by task success and context cost.
AHE [ Lin et al., 2026 ]	H, C, M, S	Teacher Sequential	Off Batch General	Benchmark, Executable verifier, LLM feedback Binary, Non-binary, LLM-as-a-judge, Other	Artifact validation, Benchmark score	Distills batches of coding-agent trajectories into layered diagnostics, edits file-level harness components, and uses next-round task outcomes to verify or roll back each predicted change.

Appendix

Table 2: Method papers and their classifications under the unified taxonomy.

P: parameters; H: harness code; C: context; M: memory; S: skills; O: other artifacts. Off/On: offline/online.
Benchmark	Resources	Mode	Task setup	Feedback for updates	Reported measures
LifelongAgentBench [ Zheng et al., 2025 ]	C, M	On	Skill-dependent task sequences in interactive environments	Environment observations and terminal success/failure	Task performance, Gain, Cost
StuLife [ Cai et al., 2025 ]	P, C, S, M	On	Stateful campus task sequence; includes a fine-tuning baseline	Environment observations, tool results, and task outcomes	Task performance, Gain, Interaction turns
Evo-Memory [ Wei et al., 2025 ]	C, M	On	Task streams with varied order and difficulty	Task trajectories and correctness signals	Task performance, Gain, Interaction steps
SkillFlow [ Zhang et al., 2026c ]	C, S	On	Task families ordered by within-family difficulty	Execution traces and rubric-based verifier feedback	Task performance, Gain, Cost
CL-Bench [ Asawa et al., 2026 ]	C, M, O	On	Related instances in shared environments, including concept drift	Environment observations and task-dependent feedback	Task performance, Gain, Cost

Appendix

Table 3: Benchmarks and their evaluation settings.

System	Main functionality	Repository
Proteus [ Proteus Authors, 2026 ]	Evolves agent harnesses, records version snapshots, and measures changes.	https://github.com/proteus-evolve/Proteus
Reef [ Human-Agent-Society, 2026 ]	Connects execution records and feedback to updates, evaluation, and version deployment.	https://github.com/Human-Agent-Society/reef
ReMe [ AgentScope Team, 2026 ]	Stores, retrieves, and maintains agent memories for reuse across tasks.	https://github.com/agentscope-ai/ReMe

Appendix

Table 4: Software systems for agent self-improvement.

Recursive Self-Improvement

Recursive self-improvement studies AI systems that use feedback from their own work to modify and evaluate future versions.

See all 83 papers

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement
Sep 10, 2026Yi Duan, Ying Liu, Zirui Tang +32Self-Improving Agents Language Model Self-Improvement
Shanghai Jiao Tong University · Theseus Labs · Agent-Native Research Lab +6
AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement
Aug 20, 2026Yizhe Chi, Wenyi Li, Deyao Hong +7Recursive Self-Improvement Automated Algorithm Discovery
Navers Lab, Einsia.AI · Tsinghua University
Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
Jul 8, 2026Mingguang Chen, Licheng Wang, Bo QuLLM Self-Refinement Human-in-the-Loop Evaluation
University of California, Riverside (UCR) · AlphaAvatar · Illinois Institute of Technology (IIT)

Sep 10, 2026J/K move · Enter open · S save

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement

Yi Duan, Ying Liu, Zirui Tang, Haodong Chen+31

Shanghai Jiao Tong University · Theseus Labs · Agent-Native Research Lab+6

Self-Improving Agents Language Model Self-Improvement Recursive Self-Improvement

Open paper AskPDF arXiv

Recursive self-improvement (RSI) enables AI systems to turn experience and feedback into persistent changes that improve both their capabilities and the process of future improvement. We first use the Headroom-Closed Index (HCI) to reveal the problems of existing LLMs, then introduce the RSI concept and its development roadmap: from improvement-execution autonomy, improvement-strategy autonomy, experience-acquisition autonomy, and environment-adaptation autonomy, to recursive meta-improvement. Next we examine RSI across scenarios (e.g., scientific discovery, embodied intelligence, software engineering), highlighting their distinct requirements and development speeds. Drawing on diverse industry practices and preliminary empirical evidence, we connect RSI research with practical systems and identify key challenges to achieving genuine RSI.

Abstract

Recursive Self-Improvement

Explore similar work

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement

AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement

Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops