Long-horizon embodied intelligence requires agents to improve through interaction, not merely to execute plans generated from static goals. A central challenge is therefore to transform past executions into knowledge that can shape future decisions. Minecraft provides a representative testbed for this problem, where tasks such as crafting tools, building redstone components, and obtaining diamond equipment involve long prerequisite chains and are frequently disrupted by missing tools, blocked paths, GUI failures, or stagnant execution. To this end, we propose MineEvolve, a knowledge-driven self-evolution framework that converts execution feedback into actionable behavioral knowledge. MineEvolve first uses \underline{\emph{\ding{182}Monitor}} to convert each subgoal execution into typed feedback, including state changes, inventory changes, failure types, progress signals, and stagnation indicators. \underline{\emph{\ding{183}Inducer}} then derives reusable skills from successful executions and remedies from failed or stagnant executions. \underline{\emph{\ding{184}Curator}} validates, merges, filters, and retrieves these knowledge entries, while \underline{\emph{\ding{185}Adaptor}} uses them to repair the unfinished part of the plan under repeated failures or stagnation. Experiments on the Minecraft MCU long-horizon task suite show that MineEvolve consistently improves performance across multiple language-model planners, with larger gains on high-dependency task groups. Ablation and knowledge-accumulation studies further demonstrate that converting execution signals into structured behavioral knowledge is an effective path toward self-evolving embodied agents in long-horizon environments. Our code is available at https://github.com/xzw-ustc/MC-MineEvolve.
Figures & tables
Figure 1: Prior methods either retrieve raw trajectories, reuse static successful cases, or rely on generic text reflections, which often provide weak guidance for repairing failures. In contrast, MineEvolve collects typed execution signals, induces actionable behavioral knowledge, maintains it in a curated knowledge bank, and uses it to repair the plan, turning past experience into explicit guidance for future decisions.
Figure 2: Progression of Minecraft from wooden to diamond tiers, illustrating the hierarchy of tools, materials, and equipment used to construct long-horizon tasks for embodied agent evaluation.
Figure 3: MineEvolve models self-evolution as the continual update and use of an external behavioral knowledge system, and consists of four sequential operations. First, Monitor converts each subgoal execution into typed execution feedback. Second, Inducer generates skills and remedies from this feedback. Third, Curator validates, merges, filters, and retrieves candidate knowledge. Finally, Adaptor injects the retrieved knowledge into the planner and repairs the unfinished part of the plan when repeated failures or stagnation occur.
Method
Qwen3.5-Flash
Qwen3.5-Plus
GLM-4.7
Gemini-3-Flash
GPT-5.5
Weighted Easy Task Average
DEPS
57.69
58.65
56.60
59.07
61.34
JARVIS-1
70.47
71.04
69.30
70.62
72.41
Optimus-1
73.76
73.66
72.55
73.26
75.29
Optimus-2 (GOAP)
73.93
74.50
73.28
74.86
75.60
MineEvolve
74.14
75.61
72.97
75.47
76.98
Table 1: Compact main results on the MCU task suite. Values denote success rate SR (%). Easy Avg. is the task-count-weighted average over Wooden, Stone, and Gold tasks; Hard Avg. is the task-count-weighted average over Iron, Redstone, Diamond, and Armor tasks; Overall is the task-count-weighted average over all 70 tasks. Best scores are highlighted in orange, and second-best scores are highlighted in teal.
Figure 4: Targeted ablation results on the four difficult task groups. Bars denote SR (%). MineEvolve consistently outperforms ablated variants across Iron, Redstone, Diamond, and Armor tasks.
Source of Remedies
Iron
Redstone
Diamond
Armor
Hard Avg.
Binary Feedback
42.36
24.72
10.28
18.36
27.06
Trajectory-level Reflection
46.88
26.79
11.94
20.36
29.98
Typed Execution Feedback
55.83
31.24
17.06
27.63
37.13
Table 2: Ablation on feedback granularity. All variants share the same low-level execution policy, knowledge-base size, retrieval budget, replanning trigger conditions, and evaluation-time LLM-call budget. Values denote SR (%).
Figure 5: Knowledge accumulation curves on the four difficult task groups. At each checkpoint, the external knowledge base is frozen before evaluation. Compared with Static Store and Text Reflection, MineEvolve shows more pronounced and sustained improvement on Iron, Redstone, Diamond, and Armor, indicating that structured skills and remedies can be accumulated and reused across task groups.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: (a) Diverse block types form an explorable, harvestable, and modifiable world. (b) The crafting interface illustrates recipe-based item composition and tool progression. (c) Survival mode introduces continuous survival pressure from health, hunger, and environmental hazards. (d) The day–night cycle changes visibility and threats, making exploration and shelter building more strategic.
Group
Task Num.
Representative Task Types
Wooden
11
Wooden tools and weapons; wood, planks, sticks, and saplings gathering
Stone
10
Stone tools and weapons; stone and cobblestone mining; basic tool upgrades
Iron
16
Iron ore mining; smelting; iron tools and weapons; villager trading
Gold
7
Gold-related tools; golden foods; repairing golden equipment
Redstone
6
Redstone components; simple mechanisms; basic circuits and clocks
Diamond
7
Diamond mining; diamond tools; repairing and trading for high-tier items
Appendix
Table 3: MCU tech-tree evaluation suite with seven task groups.
Planner backbone
Method
Wo.
St.
Ir.
Go.
Re.
Di.
Ar.
Overall
Qwen3.5-Flash
DEPS
82.91
68.37
16.48
2.79
5.84
2.11
7.25
28.90
JARVIS-1
92.37
88.14
34.76
10.83
23.41
8.39
14.92
41.75
Optimus-1
98.30
92.75
45.24
8.07
28.17
8.21
14.80
45.83
Optimus-2 (GOAP)
98.60
92.45
51.30
8.70
28.10
12.40
20.95
48.84
MineEvolve
98.09
92.61
52.43
10.11
30.77
14.58
23.47
50.09
Qwen3.5-Plus
DEPS
84.05
69.34
17.39
3.47
8.94
2.43
4.36
29.25
Appendix
Table 4: Full results on the MCU task suite. Values denote success rate SR (%). All system-level methods are evaluated under the same task set, evaluation metric, and interaction horizon. Wo., St., Ir., Go., Re., Di., and Ar. denote Wooden, Stone, Iron, Gold, Redstone, Diamond, and Armor, respectively. Overall is task-weighted over all 70 tasks.
Method
Low-level policy
Use of failure feedback
Typed feedback
Continual knowledge update
Same-policy comparison
DEPS
STEVE-1
Feedback interpretation and subgoal selection
No
No
Yes
JARVIS-1
STEVE-1
Weak; mainly successful-case retrieval
No
No
Yes
Optimus-1
STEVE-1
Retrieved as part of the experience pool
No
Partial
Yes
Optimus-2
GOAP
Not a core mechanism
No
No
No
MineEvolve
STEVE-1
Remedy generation
Yes
Yes
Yes
Appendix
Table 5: Mechanistic positioning of recent Minecraft agents.
Method
Task group
0 eps.
50 eps.
100 eps.
200 eps.
400 eps.
Static Store
Iron
39.8
40.8
41.9
42.9
43.6
Redstone
24.2
24.9
25.6
26.2
26.5
Diamond
9.0
9.5
9.9
10.4
10.9
Armor
16.2
17.2
18.2
19.3
19.4
Hard Avg.
25.13
26.01
26.90
27.79
28.22
Text Reflection
Iron
39.8
42.3
44.1
44.9
45.4
Appendix
Table 6: Group-wise results for the knowledge accumulation curves. Values denote SR (%) on held-out hard tasks. Hard Avg. is the task-count-weighted average over Iron, Redstone, Diamond, and Armor, using group sizes 16, 6, 7, and 13, respectively. At each checkpoint, the external knowledge base is frozen before evaluation.
Checkpoint
#Skills
#Remedies
Retrieved tokens
Retrieval latency
Eval. LLM calls
0 eps.
0
0
0
0.00 s
7.82
50 eps.
31
44
628
0.10 s
7.96
100 eps.
56
78
846
0.14 s
8.07
200 eps.
94
128
1087
0.19 s
8.18
400 eps.
143
191
1216
0.25 s
8.31
Appendix
Table 7: Knowledge-base size and runtime overhead. Values are averaged over held-out hard-task episodes.
Strategy
Knowledge initialization
Training-stage update
Purpose
Cold Start (Empty KB)
Empty Kskill/Kremedy
No cross-episode knowledge writing
Estimate the lower bound without external behavioral knowledge
Diamond-only Self-learning
Empty Kskill/Kremedy
Write Diamond-only experience
Test whether self-learning only on the target hard tasks is sufficient
Curriculum Pretrain → Freeze
Knowledge pretrained on lower-tier tasks
Freeze lower-tier knowledge during the Diamond stage
Test whether lower-tier knowledge transfers to Diamond tasks
Mixed Sampling (1:1)
Unified online knowledge base
Jointly update with lower-tier and Diamond experience
Test whether mixed updates yield stronger continual improvement
Appendix
Table 8: Curriculum-style knowledge construction strategies for Diamond tasks. All settings share the same planner, low-level execution policy, interaction horizon, and retrieval budget.
Strategy
SR@20%
SR@40%
SR@60%
SR@80%
SR@100%
Cold Start (Empty KB)
2.6
2.8
2.7
2.8
2.9
Diamond-only Self-learning
3.0
6.1
9.6
12.2
13.9
Curriculum Pretrain → Freeze
5.7
8.9
11.8
14.3
16.4
Mixed Sampling (1:1)
6.2
9.8
13.1
15.8
17.6
Appendix
Table 9: Success rates of different curriculum-style knowledge construction strategies on Diamond tasks. Values denote SR (%), averaged over seven Diamond tasks and repeated across multiple world seeds.
Strategy
Checkpoint Avg.
Final SR
Final gain vs. Diamond-only
Cold Start (Empty KB)
2.76
2.9
-11.0
Diamond-only Self-learning
8.96
13.9
0.0
Curriculum Pretrain → Freeze
11.42
16.4
+2.5
Mixed Sampling (1:1)
12.50
17.6
+3.7
Appendix
Table 10: Summary of gains from curriculum-style knowledge construction on Diamond tasks. Checkpoint Avg. denotes the average success rate over the five checkpoints. Final gain is computed relative to Diamond-only Self-learning.
Figure 7: Failure-conditioned local replanning. When the agent repeatedly moves near a terrain obstacle without making effective progress and without any inventory change, the Monitor identifies the behavior as a low-progress navigation failure. The resulting remedy prompts the Adaptor to repair the remaining plan by inserting a corrective subgoal: first break the dirt block blocking the path, and then continue collecting the target wood.
Figure 8: Feedback-conditioned generation of skills and remedies. A failed crafting attempt is converted into a remedy, indicating that recipes requiring a larger crafting grid must first obtain and place a crafting table. Once the corrected execution succeeds, the successful process is further stored as a reusable skill.
A central challenge in embodied recursive self-improvement is enabling agents to convert interaction history into capabilities that persist beyond one world and transfer to later tasks. A saved memory or successful final action does not by itself establish that a reusable capability was acquired; evaluation must separate capability change from carried-over resources, unequal source exposure, and early execution failures. We construct MC-EvoBench as a controlled Minecraft benchmark with a fixed catalog of 19 registered evolution chains, including 12 primary evaluation chains and seven calibration or probe chains, together with three graph-aligned paired probe families for transfer. The benchmark represents each task as a task-state graph, defines stage-level environment verifiers, assigns matched source and target roles, resets the target world, and permits only a declared agent artifact to cross the source-target boundary. For each probe, no-prior, related, unrelated, and shuffled source conditions are evaluated under matched environment-step budgets and five seeds; we report TSR, WGP, and VMP and retain provenance records. The benchmark reveals a consistent direct-capability gap between controller and planner-backed variants, while related experience yields no stable transfer: it does not improve either tool probe, its small wood-species gain hinges on one seed, and all intervals include zero. Stage audits further show that early wood-collection and crafting bottlenecks often leave later capabilities unobserved. MC-EvoBench thus offers a controlled testbed for durable, transferable embodied capabilities. Code and data are available at https://github.com/xzw-ustc/mc-bench-open-source
Jingwei Song, Zhisheng Chen, Bingfan Zeng +5
Shanghai Jiao Tong University · University of the Chinese Academy of Sciences · South China University of Technology +4
We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience. PEAM pairs a slow deliberative LLM for open-ended reasoning with a fast parametric module for reflexive execution of consolidated skills. The fast module is a multimodal Mixture-of-Experts LoRA architecture with per-category physically isolated adapters, enabling parameter-level continual learning without catastrophic forgetting. We treat failure as a first-class training signal: failure--correction trajectory pairs are internalized through a joint behavioral-cloning and contrastive objective, so the agent learns not only what succeeds but also how corrected actions differ from failed ones. To govern consolidation, PEAM introduces a parameterization-worthiness score for deciding which experience should be internalized, and a scale-free self-triggered consolidation mechanism for deciding when to internalize without task-specific hand-tuned thresholds, making the agent self-evolving as the trigger transfers across task distributions without re-tuning. Experiments in Minecraft show that PEAM improves long-horizon task performance, mitigates forgetting on previously consolidated skills, and improves parametric-versus-retrieval efficiency over retrieval-based embodied agents and parametric memory variants.
Yuchen Guo, Junli Gong, Weicheng Wang +3
Northwestern University · Northeastern University · South China University of Technology +2
Multi-modal agents have made significant progress in interacting with open worlds. However, their ability to self-evolve remains limited. Supervised fine-tuning enables imitation but not active experience organization, while outcome-based reinforcement learning promotes exploration yet underuses intermediate trajectory information. Therefore, we propose EvolvingBrain, which uses an adaptive skill system as the brain for agent self-evolution, consisting of two following components. Stratified Skill Self-Evolution: extracts Procedural Patterns and Problematic Patterns from model-specific trajectories, summarizes experience by generating skills, and evolves through multiple interactions with the open world, ultimately forming a Stratified Skill Architecture consisting of Domain, Plan, and Action/Trap Skills, which provides the agent with a multi-granularity knowledge repository and a novel evolution approach. State-Aware Adaptive Execution: summarizes long-horizon interactions into Structured Abstractive Memory and uses it to guide the State-Aware Skill Fusion Strategy, enabling the agent to utilize knowledge according to its current state. Experiments on VAB-Minecraft and DiscoveryWorld demonstrate that EvolvingBrain improves success rates on open-world tasks and exhibits generalization capabilities. Our code is provided in the supplementary materials and will be publicly released in the camera-ready version.
Runxuan Liu, Jingqi Shi, Baoxin Wang +10
Harbin Institute of Technology · iFLYTEK · Meituan +1