MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling

Renhao Li; Xinjie Shen; Yang Su; Jianhong Tu; Derek F. Wong; Dayiheng Liu; Min Yang

cs.AIOct 15, 2026

MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling

Authors: Renhao Li, Xinjie Shen, Yang Su, Jianhong Tu, Derek F. Wong, Dayiheng Liu, Min Yang

Organizations: University of Macau · Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · Georgia Institute of Technology · Alibaba Token Hub, Alibaba Group

Abstract

Language-model agents can accumulate interaction feedback, but reliably turning that feedback into reusable experience remains challenging. Existing methods either optimize external experience scaffolds or train experience-related behavior on specific task distributions, leaving open whether experience learning itself can become a transferable model capability. Training such a capability calls for supervision beyond isolated tasks, which mainly support in-episode problem solving rather than revealing what knowledge remains useful across episodes. In this work, we introduce MetaExp, a scalable framework that trains this capability through verifiable task families with controlled cross-task structure. Each family preserves a reusable factor while varying task-specific realizations, enabling transfer without direct solution reuse. We propose two complementary designs: Shared-Tool families keep black-box tools fixed across tasks to elicit operational experience, while Shared-Workflow families preserve latent procedures across changing realizations to elicit procedural experience. We scale environments under both designs, collect and filter sequential interaction trajectories, and fine-tune Qwen3.5-122B-A10B on 4,956 trajectories from 1,000 task families. With parameters frozen during downstream evolution, MetaExp improves by +8.5 points on SpreadsheetBench, peaks at +2.3 on APEX-Agents, and improves by +14.8 on AutomationBench. The gains persist under domain-excluded training, supporting experience learning as a transferable capability for continued non-parametric improvement after deployment.

Figures & tables

Figure 1: Overview of MetaExp . Heterogeneous seeds are expanded into verifiable task families with complementary Shared-Tool and Shared-Workflow structure. Family-centered task sequences are used to collect execution and feedback-driven reflection trajectories, which train the model to acquire, update, and reuse experience. At evaluation, model parameters are frozen and experience is accumulated from an empty bank on unseen task distributions.

Task success rate (%, $\uparrow$ )
Model	Protocol	$k=0$	$k=5$	$k=10$	$k=20$	$k=40$
GPT-4.1	EvoSkill	44.2	–	–	–	$37.2\pm 2.6$ $(-7.0)$
GPT-5.4	EvoSkill	62.5	–	–	–	$68.1\pm 0.5$ $(+5.6)$
GPT-4.1	SkillGrad	44.2	–	–	–	$54.2\pm 3.5$ $(+10.0)$
GPT-5.4	SkillGrad	62.5	–	–	–	$71.1\pm 1.7$ $(+8.6)$
GLM-5.2	SeqEvo	$88.3\pm 1.2$	$88.9\pm 0.5$ $(+0.6)$	$89.5\pm 0.7$ $(+1.2)$	$89.6\pm 0.4$ $(+1.3)$	$89.4\pm 0.4$ $(+1.1)$

Table 1: Performance and efficiency on SpreadsheetBench. Values are mean

\pm

population standard deviation, with

\Delta_{k}=S(k)-S(0)

shown in parentheses. Green/red denotes improvement/degradation relative to

k=0

. EvoSkill and SkillGrad results are taken from Wang et al. (2026a) and follow their originally reported evolution protocols; all other results use our cold-start SeqEvo protocol. “–” denotes an unreported checkpoint.

Figure 2: Performance on APEX-Agents and AutomationBench. Left: aggregate success rate over adaptation steps, with shaded population standard deviations over three runs. Right: domain-wise success rate and changes

\Delta_{k}

on each adaptation step for MetaExp -Qwen3.5-122B-A10B and Qwen3.5-122B-A10B; green/red denotes improvement/degradation.

Model	$k=0$	$k=5$	$k=10$	$k=20$	$k=40$
Task success rate (%, $\uparrow$ )
Qwen3.5-122B-A10B	$68.9\pm 1.8$	$67.3\pm 1.8$ $(-1.6)$	$67.7\pm 0.5$ $(-1.2)$	$67.0\pm 0.9$ $(-1.9)$	$66.7\pm 1.3$ $(-2.2)$
- MetaExp	$65.0\pm 2.2$	$69.5\pm 3.3$ $(+4.5)$	$72.9\pm 3.7$ $(+7.9)$	$73.1\pm 2.8$ $(+8.1)$	$73.5\pm 3.4$ $(+8.5)$
- Strict-OOD MetaExp	$62.7\pm 1.4$	$66.8\pm 5.0$ $(+4.1)$	$66.2\pm 4.0$ $(+3.5)$	$72.5\pm 3.0$ $(+9.8)$	$72.0\pm 0.8$ $(+9.3)$

Table 2: SpreadsheetBench task success under domain-excluded training. Values are mean

\pm

population standard deviation over three runs, with

\Delta_{k}=S(k)-S(0)

shown in parentheses.

Model	Score (%)	Pass (%)
GLM-5.2	65.1	74.7
Baseline	33.3	47.1
MetaExp	41.4 $(+8.1)$	62.1 $(+15.0)$

Table 3: Model performance on SkillsBench. Values are averaged across three runs.

Task success rate (%, $\uparrow$ )
Benchmark	Shared-Tool Only		Shared-Workflow Only		Full MetaExp
Benchmark	Initial	Final	Initial	Final	Initial	Final
SpreadsheetBench	$62.7\pm 1.0$	$72.4\pm 7.3$ $(+9.7)$	$63.6\pm 1.5$	$69.7\pm 2.7$ $(+6.1)$	$65.0\pm 2.2$	$73.5\pm 3.4$ $(+8.5)$
APEX-Agents	$13.2\pm 1.3$	$12.7\pm 1.3$ $(-0.5)$	$12.2\pm 0.6$	$12.6\pm 0.7$ $(+0.4)$	$12.2\pm 1.7$	$14.0\pm 1.4$ $(+1.8)$
AutomationBench	$7.2\pm 0.5$	$14.8\pm 3.8$ $(+7.6)$	$8.5\pm 0.5$	$15.6\pm 3.6$ $(+7.1)$	$14.5\pm 0.9$	$29.3\pm 3.3$ $(+14.8)$
Execution turns ( $\downarrow$ )

Table 4: Ablation of cross-family task designs on models. Values are mean

\pm

population standard deviation over three runs, with

\Delta=S_{\mathrm{final}}-S_{\mathrm{initial}}

shown in parentheses.

Model	Condition	Skills in bank	Any invocation (%)	Skill as first tool (%)	Calls/task
Baseline-Qwen3.5	$k=5$	$3.0\pm 0.0$	$9.3\pm 2.0$	$2.4\pm 1.3$	$0.094\pm 0.021$
	$k=10$	$5.3\pm 0.5$	$11.3\pm 2.5$	$4.2\pm 1.5$	$0.113\pm 0.025$
	$k=20$	$10.0\pm 0.0$	$18.6\pm 4.7$	$7.8\pm 0.1$	$0.188\pm 0.049$
	$k=40$	$10.0\pm 0.0$	$17.0\pm 8.2$	$5.0\pm 2.6$	$0.172\pm 0.084$
MetaExp -Qwen3.5	$k=5$	$2.0\pm 0.8$	$76.4\pm 16.4$	$63.1\pm 23.2$	$0.765\pm 0.162$
	$k=10$	$3.3\pm 2.6$	$79.5\pm 16.9$	$67.0\pm 23.0$	$0.800\pm 0.164$

Table 5: SpreadsheetBench experience-access behavior. Values are means

\pm

population standard deviations over three seeds.

Model	Conditions	Invoked skill	Did not invoke
Baseline-Qwen3.5	All $k$ pooled	$-4.2$	$-1.2$
MetaExp -Qwen3.5	All $k$ pooled	$+9.5$	$-1.9$
MetaExp -Qwen3.5	$k=20$	$+10.8\pm 4.7$	$-0.3\pm 4.8$
MetaExp -Qwen3.5	$k=40$	$+10.2\pm 6.4$	$-4.7\pm 6.7$

Table 6: Paired success-rate change relative to the same task under an empty-bank condition (%).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Baseline-Qwen3.5	MetaExp -Qwen3.5
Skills	30	14
Lexical tokens	29,679	31,155
Tokens per skill	989.3	2,225.4
Failure terms / 1K tokens	1.42	5.42
Verification terms / 1K tokens	3.13	4.56
Opaque IDs / 1K tokens	2.26	0.00

Appendix

Table 7: Structure of the final SpreadsheetBench skill banks at

k=40

, pooled over three seeds.

Case	Baseline-Qwen3.5	MetaExp -Qwen3.5
Cross-sheet lookup	A local VLOOKUP/INDEX–MATCH recipe with several formula variants and a recommendation to write static values.	A symptom–cause–repair–verification taxonomy covering truncated ranges, wrong-sheet references, hidden whitespace, cached formula values, and cell-level validation.
Boundary-sensitive editing	A generic checklist: preserve values, use absolute references, handle missing keys, and verify the output.	Conditional diagnostic rules for inflated worksheet bounds and locale-dependent formula separators, tied to observable failure signals and corrective checks.

Appendix

Table 8: Case study on how different models encode experiences on SpreadsheetBench.

Panel of Figure 3 — Figure 3: Domain distribution of corpus seeds under Shared-Tool and Shared-Workflow construction.

Panel of Figure 4 — Figure 4: Primary-intent distribution of the task-example seeds used in MetaExp .

Panel of Figure 5 — Figure 5: Capability distribution of tool examples and their coverage across seed records.

LLM Agent Self-Improvement

Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.

See all 277 papers

XferAgent: Cross-Agent Experience Transfer for Self-Improving LLM Agents
Oct 14, 2026Weihao Li, Jun Bai, Shijie Chen +3LLM Agent Self-Improvement Memory-Augmented Language Model Agents
McCormick School of Engineering, Northwestern University · School of Computer Science, McGill University · Mila – Quebec AI Institute +1
Training Language Agents to Learn from Experience
May 19, 2026Yuval Shalev, Zifeng Ding, Mateja JamnikContinual Learning for LLM Agents Self-Improving Agents
University of Cambridge
Experiential Reflective Learning for Self-Improving LLM Agents
Mar 25, 2026Marc-Antoine Allard, Arnaud Teinturier, Victor Xing +1Self-Improving Agents LLM Agent Training
Illuin Technology

Oct 14, 2026J/K move · Enter open · S save

XferAgent: Cross-Agent Experience Transfer for Self-Improving LLM Agents

Weihao Li, Jun Bai, Shijie Chen, Qincheng Lu+2

McCormick School of Engineering, Northwestern University · School of Computer Science, McGill University · Mila – Quebec AI Institute+1

LLM Agent Self-Improvement Memory-Augmented Language Model Agents Graph-Based Agent Memory Transfer Learning Continual Learning for LLM Agents

Open paper AskPDF arXiv

Large language model (LLM) agents can reuse experience from prior task executions to improve performance on subsequent tasks. Although existing methods enable cross-agent experience sharing, differences in agents' execution histories and task requirements make shared experiences difficult to reuse effectively. We propose XferAgent, a framework for cross-agent experience transfer that organizes reusable experience in local and shared experience graphs. XferAgent reconciles compatible contributions from heterogeneous agents while avoiding unreliable merges. For each new task, it selects shared experiences and integrates them with the target agent's local experience to support task execution. Experiments across diverse task domains show improvements over representative memory-based methods and effective experience transfer between heterogeneous agent frameworks. Cumulative task success also increases over successive task passes as agents update and reuse their experiences.

Abstract

LLM Agent Self-Improvement

Explore similar work

XferAgent: Cross-Agent Experience Transfer for Self-Improving LLM Agents

Training Language Agents to Learn from Experience

Experiential Reflective Learning for Self-Improving LLM Agents