RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent

Huan Zhang; Mingju Chen; Dongxu Zhou; Can Lv; Sen Cui; Faguo Wu; Shiji Zhou

cs.AIOct 15, 2026

RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent

Authors: Huan Zhang, Mingju Chen, Dongxu Zhou, Can Lv, Sen Cui, Faguo Wu, Shiji Zhou

Organizations: Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Tsinghua University · Beijing Academy of Artificial Intelligence (BAAI)

Abstract

Large language model (LLM) agents have demonstrated strong capabilities in complex tasks, including code generation, tool use, and multi-hop question answering. Existing self-improvement methods enhance agent performance through model optimization, harness evolution, and executable artifact refinement, while recursive self-improvement (RSI) further extends optimization to the improvement mechanisms themselves. However, adapting self-improvement workflows to evolving performance bottlenecks while improving workflow orchestration through execution feedback remains a key challenge. We propose RSIFlow, a Recursive Self-Improvement framework that integrates adaptive workflow orchestration with meta-agent capability evolution. RSIFlow diagnoses performance bottlenecks from task-agent trajectories and environmental feedback to dynamically select component-specific improvement workflows. The meta-agent then distills workflow outcomes into component-specific skills and cross-component principles, evolving its own harness to improve subsequent workflow orchestration. Experiments across six benchmarks and two backbone models show that RSIFlow achieves the SOTA average performance on all benchmarks. Ablation studies further demonstrate the contributions of adaptive workflow orchestration and meta-harness evolution. Code is available at https://github.com/1HUAN1/RSI-Flow.

Figures & tables

Figure 1: Motivation of RSIFlow. (A) Two coupled challenges in RSI. (B) The execution loop of RSIFlow.

Figure 2: Overview of RSIFlow. (A) Definition of task domains and agent workflows. (B) The Task-Agent executes tasks to collect trajectories and feedback. (C) The Meta-Agent retrieves skills to adaptively orchestrate workflows. (D) Workflow results are distilled into skills, principles and Meta-Harness. The updated Meta-Harness guides subsequent orchestration, closing the recursive self-improvement loop.

Method	Code		Tool Use		SearchQA		Overall
Method	HumanEval+	MBPP+	BFCL-v3	ACEBench	Hotpot	2Wiki
Qwen3-4B
AFlow	39.63	53.37	26.32	34.00	52.59	38.45	40.73
MermaidFlow	48.78	57.87	28.19	31.58	56.32	38.79	43.59
SFT	39.63	55.62	26.88	29.82	53.52	41.24	41.12
GRPO	43.90	55.06	22.30	37.25	54.39	39.32	42.04

Table 1: Main results across six benchmarks with Qwen3-4B and Qwen3-8B backbones. Best results are highlighted in bold.

Figure 3: Budget–performance Pareto analysis across all six benchmarks

Meta	Task	Code	Tool use	SearchQA
Evo.	Rou.
Vanilla		48.25	29.70	34.52
$\times$	$\checkmark$	56.73 ( $\uparrow$ 8.48)	29.70 ( $\rightarrow$ 0.00)	49.98 ( $\uparrow$ 15.46)
$\checkmark$	$\times(H)$	63.42 ( $\uparrow$ 15.17)	32.93 ( $\uparrow$ 3.23)	44.89 ( $\uparrow$ 10.37)
$\checkmark$	$\times(M)$	50.31 ( $\uparrow$ 2.06)	32.15 ( $\uparrow$ 2.45)	44.73 ( $\uparrow$ 10.21)
$\checkmark$	$\times(A)$	52.34 ( $\uparrow$ 4.09)	30.69 ( $\uparrow$ 0.99)	52.57 ( $\uparrow$ 18.05)

Table 2: Module ablation of RSIFlow using Qwen3- 4B under

R=5

evolution rounds. With meta evolution enabled, we compare fixed task-evolution routines targeting the harness (H), model (M), or artifacts (A).

Figure 4: Sensitivity of RSIFlow to different Meta-Agent backbones on HumanEval+.

Meta	Component	Code	Tool Use	SearchQA
Vanilla		48.25	29.70	34.52
Meta0	HARNESS	48.30	30.40	44.62
Meta1	HARNESS	46.32	31.16	45.04
Meta2	MODEL	67.90	31.51	43.99
Meta3	ARTIFACTS	67.08	31.83	50.58
Meta4	HARNESS	68.86	30.84	54.13

Table 3: Domain-level performance of selected Meta candidates. Scores are macro-averages (%) across two benchmarks per domain. Component refers to the workflow selected for orchestration by meta agent.

Figure 5: Case study of five rounds of recursive self-improvement in RSIFlow.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Dataset	Training total	Training per round	Reporting total
Code	TACO (DeepCoder subset)	1,000	200	—
	HumanEval+	—	—	164
	MBPP+	—	—	178
	Code subtotal	1,000	200	342
Tool use	EnvScaler	1,000	200	—
	BFCL-v3	—	—	152

Appendix

Table 4: Distribution of training and reporting tasks. Training counts denote distinct tasks; the per-round column describes each of the five evolution batches. A dash indicates that a dataset is not used in that role.

Benchmark	Tasks	Metric
HumanEval+	164	System pass@1
MBPP+	178	System pass@1
BFCL-v3	152	Weighted accuracy
ACEBench	50	Native weighted score
HotpotQA	100	Answer F1
2WikiMultiHopQA	100	Answer F1

Appendix

Table 5: Benchmarks and reporting metrics. ACEBench has 50 tasks and 57 native scoring records. QA exact match is retained separately.

Method	Task backbone	Model training	Harness updates	Editable Task components	Adaptive selection	Meta evolution
SFT	Qwen3-4B / 8B	Yes	No	M	No	No
GRPO	Qwen3-4B / 8B	Yes	No	M	No	No
AFlow	Qwen3-4B / 8B	No	Yes	H	No	No
MermaidFlow	Qwen3-4B / 8B	No	Yes	H	No	No
HarnessForge	Qwen3-4B / 8B	Yes	Yes	M, H	No	No
SIA	Qwen3-4B / 8B	Yes	Yes	M, H	Yes	No

Appendix

Table 6: Baseline update configurations. M, H, and A denote Model, Harness, and Artifacts. Adaptive selection refers to choosing among Task-component intervention routes.

Parameter	Setting
Task backbone	Qwen3-4B / Qwen3-8B
Meta backbone	GPT-5.6 Sol
Evolution rounds	5
Attempts per round	At most 2
Default Model update	SFT
Candidate initialization	Frozen round parent

Appendix

Table 7: Main evolution settings. The separate 4B SFT/GRPO comparison appears in Appendix C.1 .

Parameter	4B SFT	4B GRPO
Learning rate	$2\times 10^{-6}$	$2\times 10^{-6}$
LoRA rank / alpha	8 / 16	8 / 16
LoRA dropout	0.05	0
Precision / ranks	BF16 / 8	BF16 / 8
Maximum length	16,384	32,768
Global batch	8 calls	8 trajectories

Appendix

Table 8: Recorded 4B training settings. † Evaluation temperature refers to the B1-training/B2-evaluation comparison in Table 11 . SFT lengths and batches are measured per eligible call; GRPO batches contain complete rollouts.

Round	Route	Average (%)	Task intervention	Skill update and reuse
R1	H	32.23	Add output-contract verification.	H1: distinguish interface errors from semantic failures.
R2	H	38.04	Add receipt-grounded checks for unsupported completions.	H2: reuse H1’s contract-checking criterion for execution evidence.
R3	M	45.46	Apply LoRA replay with 3,798 eligible calls.	M1: consolidate recurring behavior through successful-call replay.
R4	M	47.11	Extend replay to 5,397 eligible calls.	Reuse M1 and supplement it with new evidence.
R5	A	51.21	Task-Agent edits, tests, and resubmits its own code within budget; Model and Harness remain fixed.	A1: output repair using prior guidance and current feedback.

Appendix

Table 9: Five-round HHMMA case: Task interventions, accumulated skills, and reported average scores. Skill entries summarize the reported procedures; A1 denotes feedback-guided output revision at R5.

Component	Editable object	Retained across rounds	Use on a new task	Preserved scope
Model	Task-model parameters or adapters.	Accepted checkpoint or adapter.	Generate fresh outputs with the updated model.	Harness unchanged; no direct output revision.
Harness	Action, Skill, or Planning modules.	Accepted execution logic and configuration.	Execute the new task using the revised Harness.	Model unchanged; no direct output revision.
Artifacts	Current code, structured submission, or answer.	Revised output for the same task; reusable experience enters Meta-Skills separately.	Generate a fresh output and reuse applicable revision guidance.	Model parameters and Harness unchanged.

Appendix

Table 10: Task-component intervention scope and persistence. Meta-Skill maintenance follows the intervention and is recorded separately. Task contracts, evaluators, and registered execution limits are preserved across all routes.

Figure 6: Model-update comparison. (a) GRPO loss with a five-step mean and min–max band. (b) Benchmark scores, rollout tokens, and success on 744 tasks. Labels show GRPO–SFT gains (%). † Token lower bound; 40 calls lack usage.

Figure 7: Rollout-budget–performance trade-offs on ACEBench, HotpotQA, and BFCL-v3, complementing the main-text budget analysis. Stars denote RSIFlow; dashed lines trace the empirical Pareto boundary. K denotes 1,000 rollouts.

Metric	SFT-1200	GRPO-3000
HumanEval+ pass@1	68.29	70.12
MBPP+ pass@1	64.04	64.04
BFCL-v3 weighted accuracy	26.32	27.63
ACEBench weighted score	31.57	33.33
HotpotQA answer F1	50.06	52.91
2Wiki answer F1	37.79	40.45

Appendix

Table 11: Measured 4B B1-training/B2-evaluation results. Scores are percentages; complete-task success uses QA EM and complete ACE task success.

Figure 8: HHMMA evolution across five rounds. H, A, and M denote the selected intervention; points show the reported average scores.

Meta	Route	Code	Tool	QA
Initial Task	—	48.25	29.70	34.52
Meta0	H	48.30	30.40	44.62
Meta1	H	46.32	31.16	45.04
Meta2	M	67.90	31.51	43.99
Meta3	A	67.08	31.83	50.58
Meta4	H	68.86	30.84	54.13

Appendix

Table 12: Selected Meta versions acting on a common initial Task-Agent. Each score is a macro-average over the two benchmarks in that domain. Scores are independent of the sequential HHMMA lineage.

Figure 9: Meta-Agent backbone sensitivity on HumanEval+ over five evolution rounds. Bars show pass rates using the bottom axes, while lines show cumulative Meta-Agent token consumption using the top axes. Cumulative counts sum token usage through the corresponding round; M denotes one million tokens.

Operation	Evidence-driven change
Add	Create an entry for an uncovered strategy, with conditions and source cases.
Supplement	Attach support or counterexamples to an existing strategy.
Revise	Update a condition, step, or invariant in a linked successor version.
Merge	Consolidate duplicate or superseded entries in the active view.

Appendix

Table 13: Maintenance operations for component skills and general principles. Original cases remain traceable after active content changes.

LLM Agent Self-Improvement

Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.

See all 277 papers

R $^2$ Flow: Recursive Self-Improvement via Recursive Skill Evolution
Sep 27, 2026Mingda Zhang, Qiang Huang, Yanjin Li +4LLM Agent Self-Improvement Generative Flow Networks
The Chinese University of Hong Kong, Shenzhen · Fudan University · University of Illinois at Urbana-Champaign +2
It Takes Workflows to Evolve Better Workflows
Oct 1, 2026Xuehang Guo, Haoyu Wang, Haifeng Chen +3LLM Agent Workflow Optimization Multi-Agent Collaboration
William & Mary · NEC Corporation of America · University of Illinois Urbana-Champaign
Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses
Aug 9, 2026Tailin ZhouAgent Harness Optimization LLM Agent Self-Improvement
HKUST

Sep 27, 2026J/K move · Enter open · S save

R $^2$ Flow: Recursive Self-Improvement via Recursive Skill Evolution

Mingda Zhang, Qiang Huang, Yanjin Li, Zijia Wang+3

The Chinese University of Hong Kong, Shenzhen · Fudan University · University of Illinois at Urbana-Champaign+2

LLM Agent Self-Improvement Generative Flow Networks Agent Skill Learning

Open paper AskPDF arXiv

LLM-based agents can improve themselves across tasks by reusing and revising the skills they orchestrate into executable procedures. Flow-based training fits this loop: it samples procedures in proportion to reward, and the flow through each skill credits it for the next library revision. Three obstacles stand in the way of making this self-improvement reliable: flow training suffers strategy collapse over tree-structured histories; nonnegative flow-based credit rewards frequent use as if it were benefit; and library edits rest on the task reward the policy optimizes. We introduce R

^2

Flow, a recursive self-improvement framework that alternates policy learning, independent verification, and versioned skill-library updates on a shared-state orchestration graph. The graph merges histories that differ only in the order of independent steps, allowing flow training to pool evidence across equivalent executions. A flow-share readout of the trained flow, invariant to the backward policy, and a separate signed utility rank which skills to change, verifier evidence decides whether an edit is warranted, and a residual-variance plateau sets when to update. Committed edits reshape the graph the next policy learns on, realizing recursive skill evolution. Across question answering, mathematical reasoning, interactive decision making, and code generation, R

^2

Flow improves task accuracy and library-edit precision over heuristic orchestration, reinforcement learning, and skill-evolution baselines, and transfers across executors. Code is available at https://github.com/beita6969/r2flow.

Abstract

LLM Agent Self-Improvement

Explore similar work

R2^22 Flow: Recursive Self-Improvement via Recursive Skill Evolution

It Takes Workflows to Evolve Better Workflows

Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses

R $^2$ Flow: Recursive Self-Improvement via Recursive Skill Evolution