Multi-modal agents have made significant progress in interacting with open worlds. However, their ability to self-evolve remains limited. Supervised fine-tuning enables imitation but not active experience organization, while outcome-based reinforcement learning promotes exploration yet underuses intermediate trajectory information. Therefore, we propose EvolvingBrain, which uses an adaptive skill system as the brain for agent self-evolution, consisting of two following components. Stratified Skill Self-Evolution: extracts Procedural Patterns and Problematic Patterns from model-specific trajectories, summarizes experience by generating skills, and evolves through multiple interactions with the open world, ultimately forming a Stratified Skill Architecture consisting of Domain, Plan, and Action/Trap Skills, which provides the agent with a multi-granularity knowledge repository and a novel evolution approach. State-Aware Adaptive Execution: summarizes long-horizon interactions into Structured Abstractive Memory and uses it to guide the State-Aware Skill Fusion Strategy, enabling the agent to utilize knowledge according to its current state. Experiments on VAB-Minecraft and DiscoveryWorld demonstrate that EvolvingBrain improves success rates on open-world tasks and exhibits generalization capabilities. Our code is provided in the supplementary materials and will be publicly released in the camera-ready version.
Figures & tables
Figure 1: Inspired by human experiential learning, EvolvingBrain extracts, evolves, organizes, and adaptively utilizes knowledge from open-world interactions.
Figure 2: Overview of EvolvingBrain.
Procedural Pattern
Problematic Pattern
Success [2pt]Trajectory
Extract step-by-step guidance that effectively advances the task, forming a reusable experience scaffold.
Identify ineffective, redundant, or risky steps that may still occur in successful trajectories, enabling the agent to learn how to bypass traps.
Failure [2pt]Trajectory
Summarize problematic patterns to refine procedural patterns, thereby reducing detours and repeated mistakes.
Extract the conditions and behaviors that lead to failure or make recovery difficult, enabling the agent to recognize traps and recover after entering them.
Table 1: Relationship between trajectory types and pattern types.
Backbone
Method
VAB-Minecraft
DiscoveryWorld
Success (%)
Avg. score
Success (%)
Avg. score
Qwen3.8-27B
Direct
57.67
0.6561
50.00
0.7510
MMSkills
69.84
0.7580
–
–
EvolvingBrain
73.02
0.8071
58.02
0.7910
Kimi-K3
Direct
69.84
0.7756
63.58
0.8160
MMSkills
70.37
0.7889
–
–
Table 2: Main results on VAB-Minecraft and DiscoveryWorld, averaged over three runs.
Figure 3: Ablation results on skill components.
Figure 4: Success-rate loss of fixed-granularity retrieval relative to adaptive retrieval. Lower is better.
Figure 5: Self-evolution gains and out-of-domain generalization.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Split
Difficulties
Variants
Instances
Training
All
0–2
54
In-domain test
All
3–4
36
Out-of-domain test
All
0–4
30
Appendix
Table 3: Our DiscoveryWorld data split.
Figure 6: State-aware recovery on a held-out VAB-Minecraft piston task. Both agents use Kimi-K3 and start from the same state under a 60-round interaction budget. The direct agent repeatedly reformulates its search and terminates without a piston. EvolvingBrain first retrieves Plan guidance, retrieves and fuses frozen Trap guidance after reaching bedrock without iron, refreshes its guidance after the observed inventory changes, and then reuses that guidance to craft and verify the piston. The Skill Brain is frozen throughout evaluation; the trajectory therefore illustrates state-aware retrieval and recovery rather than test-time skill learning. Here, R k denotes the state observed after k actions, so R0 is the initial state and R27 is the terminal state after 27 actions.
Backbone
Variant
VAB-Minecraft
DiscoveryWorld
Success (%) ↑
Avg. score ↑
Success (%) ↑
Avg. score ↑
Qwen3.8-27B
Direct
57.67
0.6561
50.00
0.7510
w/o Trap Skills
70.90
0.7871
55.02
0.7760
EvolvingBrain
73.02
0.8071
58.02
0.7910
Kimi-K3
Direct
69.84
0.7756
63.58
0.8160
w/o Trap Skills
75.27
0.8198
69.77
0.8670
Appendix
Table 4: Ablation of Trap Skill retrieval on VAB-Minecraft and DiscoveryWorld, averaged over three runs.
Backbone
Retrieval policy
VAB-Minecraft
DiscoveryWorld
Success (%) ↑
Avg. score ↑
Success (%) ↑
Avg. score ↑
Qwen3.8-27B
Coarse procedural
66.38
0.7486
53.70
0.7692
Fine-grained procedural
72.06
0.7984
57.41
0.7867
Adaptive
73.02
0.8071
58.02
0.7910
Kimi-K3
Coarse procedural
78.21
0.8427
72.84
0.8896
Fine-grained procedural
74.18
0.8165
68.52
0.8574
Appendix
Table 5: Complete results for the adaptive retrieval analysis. Trap Skill retrieval is retained in all variants, while only procedural retrieval depth is varied.
Backbone
Max. iterations
VAB-Minecraft
DiscoveryWorld
Success (%) ↑
Avg. score ↑
Success (%) ↑
Avg. score ↑
Qwen3.8-27B
1
68.68
0.7714
54.63
0.7675
3
73.02
0.8071
58.02
0.7910
Kimi-K3
1
75.29
0.8204
69.44
0.8618
3
79.37
0.8518
74.07
0.9020
GPT-5.6-Sol
1
75.57
0.8176
65.74
0.8516
Appendix
Table 6: Effect of the maximum number of self-evolution iterations.
Backbone
Method
In-domain test
Out-of-domain test
Success (%) ↑
Avg. score ↑
Success (%) ↑
Avg. score ↑
Qwen3.8-27B
Direct
50.00
0.7510
36.67
0.6812
EvolvingBrain
58.02
0.7910
46.67
0.7410
Kimi-K3
Direct
63.58
0.8160
48.89
0.7584
EvolvingBrain
74.07
0.9020
60.00
0.8457
GPT-5.6-Sol
Direct
61.72
0.8280
46.67
0.7726
Appendix
Table 7: In-domain and out-of-domain results on DiscoveryWorld.
Figure 7: Domain Skill template. Domain Skills summarize environment-level mechanics, conventions, and reusable knowledge shared across tasks.
Figure 8: Plan Skill template. Plan Skills encode task-oriented procedures, including prerequisites, ordered stages, and completion criteria.
Figure 9: Action Skill template. Action Skills ground reusable operations in observable preconditions, executable actions, and expected outcomes.
Figure 11: Plan Skill generation prompt. It abstracts successful trajectories into reusable task procedures with explicit applicability conditions and ordered stages.
Figure 12: Action Skill generation prompt. It converts interaction evidence into grounded operations with observable preconditions and verifiable effects.
Figure 13: Trap Skill generation prompt. It extracts recurrent failure modes and records their triggers, diagnoses, avoidance rules, and recovery procedures.
Figure 14: Domain Skill synthesis prompt. It consolidates cross-task evidence into environment-level knowledge while removing task-specific details.
Figure 15: Structured memory update prompt. It compresses the latest observation and action outcome into a bounded state representation used by subsequent retrieval decisions.
Figure 16: State-aware gate prompt. It decides whether additional procedural or failure-aware knowledge is needed under the current task state.
Figure 17: Hierarchical Skill retrieval prompt. It selects the appropriate retrieval depth and returns the most relevant Domain, Plan, Action, and Trap Skills.
Figure 18: State-aware Skill fusion prompt. It reconciles retrieved procedural and failure-aware guidance into a concise, state-conditioned instruction for the acting agent.