MineEvolve: Self-Evolution with Accumulated Knowledge for Long-Horizon Embodied Minecraft Agents

Zhengwei Xie; Zhisheng Chen; Ziyan Weng; Jinhan Li; Chenglong Li; Zikai Xiao; Jingwei Song; Jinhao Jing

cs.AIMar 13, 2026

MineEvolve: Self-Evolution with Accumulated Knowledge for Long-Horizon Embodied Minecraft Agents

Authors: Zhengwei Xie, Zhisheng Chen, Ziyan Weng, Jinhan Li, Chenglong Li, Zikai Xiao, Jingwei Song, Jinhao Jing,

Organizations: USTC · NTU · CityU-DG · THU ·

Abstract

Long-horizon embodied intelligence requires agents to improve through interaction, not merely to execute plans generated from static goals. A central challenge is therefore to transform past executions into knowledge that can shape future decisions. Minecraft provides a representative testbed for this problem, where tasks such as crafting tools, building redstone components, and obtaining diamond equipment involve long prerequisite chains and are frequently disrupted by missing tools, blocked paths, GUI failures, or stagnant execution. To this end, we propose MineEvolve, a knowledge-driven self-evolution framework that converts execution feedback into actionable behavioral knowledge. MineEvolve first uses \underline{\emph{\ding{182}Monitor}} to convert each subgoal execution into typed feedback, including state changes, inventory changes, failure types, progress signals, and stagnation indicators. \underline{\emph{\ding{183}Inducer}} then derives reusable skills from successful executions and remedies from failed or stagnant executions. \underline{\emph{\ding{184}Curator}} validates, merges, filters, and retrieves these knowledge entries, while \underline{\emph{\ding{185}Adaptor}} uses them to repair the unfinished part of the plan under repeated failures or stagnation. Experiments on the Minecraft MCU long-horizon task suite show that MineEvolve consistently improves performance across multiple language-model planners, with larger gains on high-dependency task groups. Ablation and knowledge-accumulation studies further demonstrate that converting execution signals into structured behavioral knowledge is an effective path toward self-evolving embodied agents in long-horizon environments. Our code is available at https://github.com/xzw-ustc/MC-MineEvolve.

Figures & tables

Figure 1: Prior methods either retrieve raw trajectories, reuse static successful cases, or rely on generic text reflections, which often provide weak guidance for repairing failures. In contrast, MineEvolve collects typed execution signals, induces actionable behavioral knowledge, maintains it in a curated knowledge bank, and uses it to repair the plan, turning past experience into explicit guidance for future decisions.

Figure 2: Progression of Minecraft from wooden to diamond tiers, illustrating the hierarchy of tools, materials, and equipment used to construct long-horizon tasks for embodied agent evaluation.

Figure 3: MineEvolve models self-evolution as the continual update and use of an external behavioral knowledge system, and consists of four sequential operations. First, Monitor converts each subgoal execution into typed execution feedback. Second, Inducer generates skills and remedies from this feedback. Third, Curator validates, merges, filters, and retrieves candidate knowledge. Finally, Adaptor injects the retrieved knowledge into the planner and repairs the unfinished part of the plan when repeated failures or stagnation occur.

Method	Qwen3.5-Flash	Qwen3.5-Plus	GLM-4.7	Gemini-3-Flash	GPT-5.5
Weighted Easy Task Average
DEPS	57.69	58.65	56.60	59.07	61.34
JARVIS-1	70.47	71.04	69.30	70.62	72.41
Optimus-1	73.76	73.66	72.55	73.26	75.29
Optimus-2 (GOAP)	73.93	74.50	73.28	74.86	75.60
MineEvolve	74.14	75.61	72.97	75.47	76.98

Table 1: Compact main results on the MCU task suite. Values denote success rate SR (%). Easy Avg. is the task-count-weighted average over Wooden, Stone, and Gold tasks; Hard Avg. is the task-count-weighted average over Iron, Redstone, Diamond, and Armor tasks; Overall is the task-count-weighted average over all 70 tasks. Best scores are highlighted in orange, and second-best scores are highlighted in teal.

Figure 4: Targeted ablation results on the four difficult task groups. Bars denote SR (%). MineEvolve consistently outperforms ablated variants across Iron, Redstone, Diamond, and Armor tasks.

Source of Remedies	Iron	Redstone	Diamond	Armor	Hard Avg.
Binary Feedback	42.36	24.72	10.28	18.36	27.06
Trajectory-level Reflection	46.88	26.79	11.94	20.36	29.98
Typed Execution Feedback	55.83	31.24	17.06	27.63	37.13

Table 2: Ablation on feedback granularity. All variants share the same low-level execution policy, knowledge-base size, retrieval budget, replanning trigger conditions, and evaluation-time LLM-call budget. Values denote SR (%).

Figure 5: Knowledge accumulation curves on the four difficult task groups. At each checkpoint, the external knowledge base is frozen before evaluation. Compared with Static Store and Text Reflection, MineEvolve shows more pronounced and sustained improvement on Iron, Redstone, Diamond, and Armor, indicating that structured skills and remedies can be accumulated and reused across task groups.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: (a) Diverse block types form an explorable, harvestable, and modifiable world. (b) The crafting interface illustrates recipe-based item composition and tool progression. (c) Survival mode introduces continuous survival pressure from health, hunger, and environmental hazards. (d) The day–night cycle changes visibility and threats, making exploration and shelter building more strategic.

Group	Task Num.	Representative Task Types
Wooden	11	Wooden tools and weapons; wood, planks, sticks, and saplings gathering
Stone	10	Stone tools and weapons; stone and cobblestone mining; basic tool upgrades
Iron	16	Iron ore mining; smelting; iron tools and weapons; villager trading
Gold	7	Gold-related tools; golden foods; repairing golden equipment
Redstone	6	Redstone components; simple mechanisms; basic circuits and clocks
Diamond	7	Diamond mining; diamond tools; repairing and trading for high-tier items

Appendix

Table 3: MCU tech-tree evaluation suite with seven task groups.

Planner backbone	Method	Wo.	St.	Ir.	Go.	Re.	Di.	Ar.	Overall
Qwen3.5-Flash	DEPS	82.91	68.37	16.48	2.79	5.84	2.11	7.25	28.90
	JARVIS-1	92.37	88.14	34.76	10.83	23.41	8.39	14.92	41.75
	Optimus-1	98.30	92.75	45.24	8.07	28.17	8.21	14.80	45.83
	Optimus-2 (GOAP)	98.60	92.45	51.30	8.70	28.10	12.40	20.95	48.84
	MineEvolve	98.09	92.61	52.43	10.11	30.77	14.58	23.47	50.09
Qwen3.5-Plus	DEPS	84.05	69.34	17.39	3.47	8.94	2.43	4.36	29.25

Appendix

Table 4: Full results on the MCU task suite. Values denote success rate SR (%). All system-level methods are evaluated under the same task set, evaluation metric, and interaction horizon. Wo., St., Ir., Go., Re., Di., and Ar. denote Wooden, Stone, Iron, Gold, Redstone, Diamond, and Armor, respectively. Overall is task-weighted over all 70 tasks.

Method	Low-level policy	Use of failure feedback	Typed feedback	Continual knowledge update	Same-policy comparison
DEPS	STEVE-1	Feedback interpretation and subgoal selection	No	No	Yes
JARVIS-1	STEVE-1	Weak; mainly successful-case retrieval	No	No	Yes
Optimus-1	STEVE-1	Retrieved as part of the experience pool	No	Partial	Yes
Optimus-2	GOAP	Not a core mechanism	No	No	No
MineEvolve	STEVE-1	Remedy generation	Yes	Yes	Yes

Appendix

Table 5: Mechanistic positioning of recent Minecraft agents.

Method	Task group	0 eps.	50 eps.	100 eps.	200 eps.	400 eps.
Static Store	Iron	39.8	40.8	41.9	42.9	43.6
	Redstone	24.2	24.9	25.6	26.2	26.5
	Diamond	9.0	9.5	9.9	10.4	10.9
	Armor	16.2	17.2	18.2	19.3	19.4
	Hard Avg.	25.13	26.01	26.90	27.79	28.22
Text Reflection	Iron	39.8	42.3	44.1	44.9	45.4

Appendix

Table 6: Group-wise results for the knowledge accumulation curves. Values denote SR (%) on held-out hard tasks. Hard Avg. is the task-count-weighted average over Iron, Redstone, Diamond, and Armor, using group sizes 16, 6, 7, and 13, respectively. At each checkpoint, the external knowledge base is frozen before evaluation.

Checkpoint	#Skills	#Remedies	Retrieved tokens	Retrieval latency	Eval. LLM calls
0 eps.	0	0	0	0.00 s	7.82
50 eps.	31	44	628	0.10 s	7.96
100 eps.	56	78	846	0.14 s	8.07
200 eps.	94	128	1087	0.19 s	8.18
400 eps.	143	191	1216	0.25 s	8.31

Appendix

Table 7: Knowledge-base size and runtime overhead. Values are averaged over held-out hard-task episodes.

Strategy	Knowledge initialization	Training-stage update	Purpose
Cold Start (Empty KB)	Empty $K_{\mathrm{skill}}/K_{\mathrm{remedy}}$	No cross-episode knowledge writing	Estimate the lower bound without external behavioral knowledge
Diamond-only Self-learning	Empty $K_{\mathrm{skill}}/K_{\mathrm{remedy}}$	Write Diamond-only experience	Test whether self-learning only on the target hard tasks is sufficient
Curriculum Pretrain $\rightarrow$ Freeze	Knowledge pretrained on lower-tier tasks	Freeze lower-tier knowledge during the Diamond stage	Test whether lower-tier knowledge transfers to Diamond tasks
Mixed Sampling (1:1)	Unified online knowledge base	Jointly update with lower-tier and Diamond experience	Test whether mixed updates yield stronger continual improvement

Appendix

Table 8: Curriculum-style knowledge construction strategies for Diamond tasks. All settings share the same planner, low-level execution policy, interaction horizon, and retrieval budget.

Strategy	SR@20%	SR@40%	SR@60%	SR@80%	SR@100%
Cold Start (Empty KB)	2.6	2.8	2.7	2.8	2.9
Diamond-only Self-learning	3.0	6.1	9.6	12.2	13.9
Curriculum Pretrain $\rightarrow$ Freeze	5.7	8.9	11.8	14.3	16.4
Mixed Sampling (1:1)	6.2	9.8	13.1	15.8	17.6

Appendix

Table 9: Success rates of different curriculum-style knowledge construction strategies on Diamond tasks. Values denote SR (%), averaged over seven Diamond tasks and repeated across multiple world seeds.

Strategy	Checkpoint Avg.	Final SR	Final gain vs. Diamond-only
Cold Start (Empty KB)	2.76	2.9	-11.0
Diamond-only Self-learning	8.96	13.9	0.0
Curriculum Pretrain $\rightarrow$ Freeze	11.42	16.4	+2.5
Mixed Sampling (1:1)	12.50	17.6	+3.7

Appendix

Table 10: Summary of gains from curriculum-style knowledge construction on Diamond tasks. Checkpoint Avg. denotes the average success rate over the five checkpoints. Final gain is computed relative to Diamond-only Self-learning.

Figure 7: Failure-conditioned local replanning. When the agent repeatedly moves near a terrain obstacle without making effective progress and without any inventory change, the Monitor identifies the behavior as a low-progress navigation failure. The resulting remedy prompts the Adaptor to repair the remaining plan by inserting a corrective subgoal: first break the dirt block blocking the path, and then continue collecting the target wood.

Figure 8: Feedback-conditioned generation of skills and remedies. A failed crafting attempt is converted into a remedy, indicating that recipes requiring a larger crafting grid must first obtain and place a crafting table. Once the corrected execution succeeds, the successful process is further stored as a reusable skill.

Self-Evolving Agents

Self-evolving agents improve their skills, memory, tools, policies, or environments by learning from interaction and evaluation feedback.

See all 416 papers

MC-EvoBench: Auditing Experience-Driven Self-Improvement in Minecraft Agents
Oct 9, 2026Jingwei Song, Zhisheng Chen, Bingfan Zeng +5AI Agent Benchmarks Self-Improving Agents
Shanghai Jiao Tong University · University of the Chinese Academy of Sciences · South China University of Technology +4
PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft
May 26, 2026Yuchen Guo, Junli Gong, Weicheng Wang +3Continual Learning for LLM Agents LLM Agent Memory
Northwestern University · Northeastern University · South China University of Technology +2
EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds
Oct 14, 2026Runxuan Liu, Jingqi Shi, Baoxin Wang +10Self-Evolving Agents Agent Skill Learning
Harbin Institute of Technology · iFLYTEK · Meituan +1

Oct 9, 2026J/K move · Enter open · S save

MC-EvoBench: Auditing Experience-Driven Self-Improvement in Minecraft Agents

Jingwei Song, Zhisheng Chen, Bingfan Zeng, Jing Tang+4

Shanghai Jiao Tong University · University of the Chinese Academy of Sciences · South China University of Technology+4

AI Agent Benchmarks Self-Improving Agents Long-Horizon Agent Evaluation Transfer Learning

Open paper AskPDF arXiv

A central challenge in embodied recursive self-improvement is enabling agents to convert interaction history into capabilities that persist beyond one world and transfer to later tasks. A saved memory or successful final action does not by itself establish that a reusable capability was acquired; evaluation must separate capability change from carried-over resources, unequal source exposure, and early execution failures. We construct MC-EvoBench as a controlled Minecraft benchmark with a fixed catalog of 19 registered evolution chains, including 12 primary evaluation chains and seven calibration or probe chains, together with three graph-aligned paired probe families for transfer. The benchmark represents each task as a task-state graph, defines stage-level environment verifiers, assigns matched source and target roles, resets the target world, and permits only a declared agent artifact to cross the source-target boundary. For each probe, no-prior, related, unrelated, and shuffled source conditions are evaluated under matched environment-step budgets and five seeds; we report TSR, WGP, and VMP and retain provenance records. The benchmark reveals a consistent direct-capability gap between controller and planner-backed variants, while related experience yields no stable transfer: it does not improve either tool probe, its small wood-species gain hinges on one seed, and all intervals include zero. Stage audits further show that early wood-collection and crafting bottlenecks often leave later capabilities unobserved. MC-EvoBench thus offers a controlled testbed for durable, transferable embodied capabilities. Code and data are available at https://github.com/xzw-ustc/mc-bench-open-source

Abstract

Self-Evolving Agents

Explore similar work

MC-EvoBench: Auditing Experience-Driven Self-Improvement in Minecraft Agents

PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft

EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds