Building autonomous agents that continually explore open worlds and acquire new capabilities through experience is a long-standing aspiration in artificial intelligence. Existing research often pursues lifelong learning within predefined tasks, curricula, or objectives, with self-improvement largely confined to individual agents. We introduce OpenKAGE, an agent-group evolution framework for open-ended lifelong learning. Within this framework, agents autonomously explore open worlds and continually acquire new capabilities by accumulating and reusing knowledge from their own interactions and the experience of others. OpenKAGE builds on two key mechanisms: (1) a two-layer knowledge representation, with a memory layer for situated memories and an experience layer for transferable experience and skills that retains the applicability conditions needed for reuse; and (2) a knowledge-sharing mechanism that semantically consolidates textual experience into a continuously evolving shared knowledge graph, while executable skills are shared verbatim. In Minecraft, OpenKAGE supports open-ended lifelong learning, enabling agent groups to sustain exploration and reach advanced technological milestones. Furthermore, the accumulated knowledge can be transferred to a newly initialized agent group on a different backbone model, improving its survival and exploration progress.
Figures & tables
Figure 1: Open-ended lifelong learning through shared knowledge in OpenKAGE. Agents explore separate Minecraft worlds and keep situated memories local; transferable experience, with its applicability conditions, is consolidated into a shared knowledge graph, and executable skills are shared verbatim. The bottom strip illustrates milestones, not a prescribed curriculum.
Figure 2: OpenKAGE overview. Each agent’s local graph has a private memory layer and a shareable experience layer; only experience-layer records leave it. Textual records are consolidated into the shared graph every K days, and skills enter it verbatim under their authors’ names. An intent-aware retriever serves the planner. Schematic; agents receive text observations, not images.
Figure 3: Operation in game-day cycles. Left: structured observation and execution feedback (schematic). Right: within each 24,000-tick day the planner alternates between thinking, acting, and observing; at dawn the world pauses for reflection, and aggregation runs every K days.
Figure 4: Intent-aware subgraph retrieval (schematic). The retriever selects anchors by relative similarity, then expands along relations scored by query-conditioned relevance and retrieval-use strength; blue solid arrows trace one sampled path, dashed arrows the other candidates. A structured subgraph is returned, not a ranked list.
Survival
Exploration
Method
Deaths ↓
Repeat ↓
XP/life ↑
Tool era
Obsidian
Nether
Voyager
228.9
203.1
1.0
dmd.(3.0day,1/30)
—
—
Echo
73.6
53.1
4.2
dmd.(15.8day,26/30)
—
—
Planner-only
27.9
15.1
14.0
dmd.(15.2day,16/30)
—
—
OpenKAGE
16.2
7.4
18.2
dmd.(13.7day,24/30)
22.8day(6/30)
23.8day(4/30)
Table 1: Comparison with baselines on DeepSeek-V4.1-Flash (30 game days, 10 runs of three agents per method). Deaths, Repeat (deaths repeating an earlier one of the same agent) and XP/life are per-agent means. Tool era (dmd. = diamond), Obsidian and Nether give the mean first-attainment day over the agents that reached it, with their count out of 30; — means none. Definitions: Appendix B.2 .
Survival
Exploration
Backbone
Method
Deaths ↓
Repeat ↓
XP/life ↑
Tool era
Obsidian
Nether
DS-V4.1F
Planner-only
27.9
15.1
14.0
dmd.(15.2day,16/30)
—
—
OpenKAGE
16.2
7.4
18.2
dmd.(13.7day,24/30)
22.8day(6/30)
23.8day(4/30)
GLM-5.3
Planner-only
33.4
17.4
8.1
dmd.(15.2day,21/30)
—
—
OpenKAGE
28.6
16.9
24.1
dmd.(13.8day,28/30)
23.9day(10/30)
27.0day(2/30)
hy4-preview
Planner-only
47.0
28.8
8.2
dmd.(16.3day,29/30)
—
—
Table 2: OpenKAGE and its planner-only control across backbones (30 game days, 10 runs of three agents per cell). All metrics and conventions are as in Table 1 ; — means none attained.
Figure 5: Deaths per agent over 30 days by cause (combat: hostile mobs, arrows, explosions); bar-end numbers are totals.
Survival
Exploration
Condition
Deaths ↓
Repeat ↓
XP/life ↑
Tool era
Obsidian
Nether
Full
16.2
7.4
18.2
dmd.(13.7day,24/30)
22.8day(6/30)
23.8day(4/30)
Raw
17.6
10.8
21.6
dmd.(13.0day,14/30)
—
—
−S
17.4
8.1
16.1
dmd.(13.3day,21/30)
21.5day(2/30)
21.0day(1/30)
−R
15.6
7.4
13.3
dmd.(13.3day,12/30)
—
—
−M
11.6
4.1
37.7
dmd.(17.9day,19/30)
—
—
Table 3: Ablation results (DeepSeek-V4.1-Flash, 30 game days, 10 runs per condition; metrics as in Table 1 ). Conditions are named by what they remove: −S sharing, −R subgraph retrieval, −M the memory layer, −E the experience layer, B all graphs; Raw keeps every component but shares experience records verbatim (Appendix B.5 ).
Figure 6: Knowledge transfer (transfer solid, non-transfer dashed; three runs per group, nine-agent means, bands ±1 sd across agents). Icons in (c) mark the median first-reach night of the iron pickaxe, first armor, first diamond and obsidian; their height is arbitrary.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Field
Content
Written by
slug
ASCII snake-case identifier, unique; the handle every operation addresses
R / A
name
free text; a proper name in the memory layer, a common name in the experience layer
R / A
summary
one-line statement of what the concept is
R / A
layer
mem or exp , declared when the concept is created
R
category
episodic , spatial , lesson or status ; empty for experience concepts
R (memory only)
strength
real, initialized to 1.0 ; monotone, and only the retrieval path writes it
Ret only
Appendix
Table 4: Concept fields. Who writes: R = the reflector (private graph), A = the aggregator (shared graph), Ret = the retrieval path of the acting agent. No other component may write any field.
Figure 7: Situated memory and transferable experience. Five examples from different agents and runs: left , private memory keeps episode-specific events and locations; right , the experience layer holds reusable textual knowledge records and executable skills. Arrows mark thematic correspondence, not graph edges.
Category
Holds
Goes stale
Budget
episodic
what is bound to a specific day: the day record, that day’s blockers and findings
within a few days
18
spatial
durable assets with a location: camps, chests, mines, workbenches
slowly, when exhausted
10
lesson
what an encounter left behind: deaths, misleading clues
not with time
4
status
the rolling self-model: current progress, backlog, equipment plan
overwritten daily
3
Appendix
Table 5: The four memory categories and their injection budgets. A bucket takes the concepts of that category in decreasing order of updated_at ; budgets are caps, not targets, and are not redistributed across buckets.
Group
Calls
Pathfinding
bot.pathfinder.goto with a goal object: GoalNear , GoalNearXZ (within a radius of a point or a column), GoalBlock , GoalXZ , GoalY (a cell, a column, a height)
Goals for objects
GoalGetToBlock , GoalLookAtBlock (reach a block to touch or see it), GoalPlaceBlock , GoalFollow , GoalInvert (retreat), GoalCompositeAny / GoalCompositeAll
Direct control
bot.setControlState for movement keys, bot.look and bot.lookAt for turning
Blocks
bot.findBlock , bot.findBlocks , bot.blockAt to read; bot.dig to mine one block (no tool choice, no approach, no pickup); bot.placeBlockSafe to place the held block on a face of a reference block, returning where it landed; bot.collectBlock.collect for a small batch
Crafting
bot.recipesFor , bot.recipesAll for the recipes of an item id; bot.craft to execute one, given the recipe, a count, and the crafting-table block where one is needed
Inventory, containers
bot.inventory.items() , .count() , .emptySlotCount() over 36 slots; bot.openContainer and bot.openFurnace return a window whose withdraw and deposit move items
Appendix
Table 6: The atomic action API, in the scope of every generated program.
Field
Radius
Content
Biome
own cell
biome of the block underfoot
Time
—
phase label ( sunrise , day , noon , sunset , night , midnight ) and the current tick out of 24,000
Light
own cell
block light and sky light, each 0–15, reported separately: the first is what the agent can change (torches), the second is exposure to sky and does not fall at night
Position
—
absolute coordinates, to one decimal, since block operations are sensitive to a one-block difference
Vitals
—
health and food out of 20 with saturation, XP level and points, and oxygen when it is not full
Nearby entities
32 blocks
per type: how many, the distance to the nearest, and the absolute-axis directions of the three nearest
Appendix
Table 7: Observation fields returned after each step.
Agent
How it tends to explore
What it underweights
Alice (ESTP)
Enters the situation first and trusts what the world feeds back now over anything reasoned out in advance; improvises quickly and dislikes standing still.
The longer-run cost that a repeated short-term gamble is quietly accumulating.
Bob (INFP)
Goes where the thing itself feels worth doing—new terrain, an untried method—and keeps options open rather than committing to a direction early.
Structure and commitment: will hold back and under-push rather than stake on a plan it cannot yet see through.
Carol (ISTJ)
Checks what is already known before spending a step on it: reuses what worked verbatim, skirts what only partly matches, and treats gaps as the only places worth new effort.
Open possibility; at night tends to audit and qualify what it already recorded rather than widen it.
Dave (ISTP)
Works in small pieces, one thing at a time, and wants a verdict on each step before starting the next; probes cheaply first and states the state before and after.
The shape of the whole: reports one step’s detail precisely, rarely what the day added up to.
Erin (ISFJ)
Refuses any bet whose loss would wipe out what has been accumulated; keeps a fixed home and works out and back from it in short trips instead of carrying everything onward.
Uncertain upside; as risk rises it drops productive work early to secure safety.
Frank (ENTJ)
Sets a target first, works backwards to what reaching it requires, then pushes through in one pass; re-targets outright when the target proves not worth it.
Whatever cannot be scored against the plan, and the option of waiting: standing still is the one thing it treats as always wrong.
Appendix
Table 8: The eight personas. An agent receives its own row as the first standing message; nothing else varies per agent. Each is written from one template, an MBTI cognitive-function stack (type in parentheses), naming what the agent is drawn to and what it underweights. Full text, and eight further personas, are released with the code.
Tool
What it does
createplan
Sets the goal for this day–night cycle and the ordered steps it decomposes into. Planning with a goal triggers retrieval, whose result comes back with this call.
updateplan
Marks a step running, succeeded or failed with a note, or replaces the remaining steps given a reason. Revising triggers retrieval keyed to that reason.
retrieve
Queries the graphs by intent: which layer (experience or memory), which scope (own, shared or both), and how many hops of relational expansion.
expand
Deepens from one concept along its relations, optionally along one relation only. This is the deliberate, high-precision form of use that scores strength (Appendix A.2 ).
retrieve_skills
Looks up skill signatures by intent, up to three. The code itself is already in scope, so a signature is all that is needed to call one.
write_code
Executes one program: the only tool that changes the world.
Appendix
Table 9: The planner’s tools. Only write_code changes the world.
Figure 8: The two channels of knowledge sharing. Left: textual experience from different agents consolidated into one shared graph. Right: skills travel verbatim; Carol’s staircaseClimbToSurface (day 2) was used by all three agents from day 3, with 114 successes and 81 cross-agent calls by day 30.
Symbol
Value
Role
k
4
vector candidates considered for anchoring
ρ
0.80
relative similarity gate: admit within ρ of the best match
h
2
hop cap; the agent chooses 1..h per query
p
0.95
nucleus pruning of low-probability edges, after softmax
τ
1.0
softmax temperature over edge scores
Δexpand
1.0
strength gain when the agent expands a concept
Appendix
Table 10: Retrieval hyperparameters, identical across all conditions. The temperature and h govern only edge sampling; anchoring is deterministic given the index.
System
Unit
Artifact
Situated / transferable split
Shared & consolidated
Store moved to a new group
Memory → context
Voyager
1
personal skill library (code)
— (skills only)
—
— (one agent)
embedding top- k over skill descriptions
GITM / JARVIS-1
1
structured external knowledge + text memory
—
—
n.e.
retrieval over knowledge base
Optimus-1
1
hybrid graph + experience pool
—
—
n.e.
graph query for planning
MrSteve
1
What-Where-When episodic trace
episodic only
—
n.e.
spatiotemporal retrieval
AriGraph
1
KG world model, episodic + semantic
✓ (two layers, one agent)
—
n.e.
local subgraph exploration
ADAM
1
causal graph
—
—
n.e.
graph-guided reasoning
Appendix
Table 11: Related agent-memory and experience-sharing systems. “Unit” is who accumulates; “artifact” is what accumulation produces; “memory → context” is how stored knowledge reaches the acting agent. Rows are compared on design, not on reported scores.
Group
Value
Backbone
DeepSeek-V4.1-Flash (DS-V4.1F, default; also the backbone of the ablations and of both external baselines), GLM-5.3, hy4-preview and Kimi-K3; transfer target GLM-5.3-Flash; one backbone serves all three roles of a run
Temperature
0.3 planner, 0.7 reflector, 0.7 aggregator
Top- p
not set
Reasoning
low for all three roles
Output cap
131,072 tokens per planner step, 65,536 per reflector and aggregator step, 8,192 for the night note
Step cap
299 tool calls per day, 99 per reflector session, 99 per aggregation
Appendix
Table 12: Reproducibility configuration. Values are identical in every reported condition unless a row says otherwise; the configuration file released with the code carries every value, and each run records the configuration it used in its own summary.
Backbone
Run
Deaths ↓
XP/life ↑
Tool era
Obsidian
Nether
DS-V4.1F
Voyager-1
325.0
0.7
iron(2.3day,3/3)
—
—
Voyager-2
412.0
0.5
iron(2.5day,2/3)
—
—
Voyager-3
167.7
1.7
dmd.(3.0day,1/3)
—
—
Voyager-4
280.7
0.6
iron(2.3day,3/3)
—
—
Voyager-5
205.3
0.9
iron(2.3day,3/3)
—
—
Voyager-6
208.3
1.1
iron(3.7day,3/3)
—
—
Appendix
Table 13: Per-run values behind Tables 1 and 2 : one row per run of three agents (ten per cell, in launch order; seed 7), with metrics as in Table 1 computed over the run’s agents; Tool era gives the highest tier reached in the run. A dash means no agent reached the milestone within 30 game days. Names following a milestone count are the personas of Table 8 .
Figure 9: Run-level values behind Table 2 : one dot per run (per-agent mean over its three agents), with the cell mean as a horizontal bar. (a) Deaths. (b) XP per life. (c) Milestones unlocked.
Figure 10: Deaths per agent per day (three-day moving average) for OpenKAGE and its planner-only control on each backbone: mean over 10 runs of three agents, with a band of ±1 standard error across runs. Each panel has its own vertical scale.
Figure 11: Growth and reuse of the shared skill library (10 runs of three agents per curve). (a) Skills available to an agent on DeepSeek-V4.1-Flash: under Full the whole group library, of which the agent wrote only the self-written part and the shaded gap comes from teammates; under −S only the skills it wrote. (b) Share of skill calls that raised an error, for own and teammates’ skills on each backbone (bars pool ten runs, dots are single runs). (c) DS-V4.1F: each thin line is one of the 151 revisions after which the skill was called again, joining its failure share before and after; the thick line pools all.
Skill
Error before
Change
Before
After
build_box_hut_with_door
doorCell is not iterable
Reads the door cell as an array and guards a missing door
1/1
1/0
seal_side_shelter_and_torch
origin is not iterable
Normalises the origin argument to an array before use
0/2
3/0
place_block_clear_of_self
no support block below target
Checks the block below the target and returns early when absent
3/1
14/0
dig_and_seal_pit
timeout after 5100 ms
Replaces tick waits with a timed sleep
2/1
1/0
Appendix
Table 14: Examples of skill revisions triggered by failed calls (DS-V4.1F). Calls are counted as successes/failures before the revision (since the skill was written or last revised) and after it. One example per distinct error message is taken in run order, among revisions after which the skill did not fail again.
Run
Deaths ↓
XP/life ↑
Tool era
Obsidian
Nether
Raw-1
17.3
35.2
dmd.(16.5day,2/3)
—
—
Raw-2
8.0
29.1
dmd.(22.0day,1/3)
—
—
Raw-3
10.7
35.6
dmd.(9.0day,2/3)
—
—
Raw-4
32.0
16.5
dmd.(11.0day,3/3)
—
—
Raw-5
15.7
9.6
dmd.(15.5day,2/3)
—
—
Raw-6
16.3
19.5
dmd.(9.0day,2/3)
—
—
Appendix
Table 15: Per-run values of the ablated conditions (DeepSeek-V4.1-Flash, 30 game days, world seed 7), one row per run of three agents (including the Raw condition), in launch order and in the conventions of Table 13 ; milestone counts name the agents as in Table 13 (personas in Table 8 ).
Figure 12: Scaling in group size and time horizon (GLM-5.3-Flash). Mean milestones per agent, bands ±1 sd across agents; each curve is a single run. (a) N=3 , 5 , 8 agents, 30 game days. (b) OpenKAGE and the baseline over 60 game days at N=3 .
Backbone
Method
LLM calls
Steps / agent
DS-V4.1F
Voyager
439
66.8
Echo
336
56.7
Planner-only
136
39.3
OpenKAGE
164
32.5
Raw
179
36.6
−S
150
30.1
Appendix
Table 16: Inference cost per game night (30 game days, ten runs of each cell). Calls are LLM calls of the whole group of three agents; steps are executed environment steps per agent.
Planner-only
OpenKAGE
Component
Calls
Text (k)
Calls
Text (k)
Planning and execution
136.0
98
110.1
155
Reflection
—
—
46.4
221
Aggregation
—
—
7.5
38
Total
136
98
164
414
Wall-clock hours per run
7.6
12.1
Appendix
Table 17: Inference cost on DS-V4.1F (ten runs per condition, three agents): calls and saved text volume per game night by component, and mean wall-clock hours of a complete 30-night run. Text volume is saved message characters divided by 1.7 , an estimate of tokens; each message is counted once, so it is not billed tokens, which also include re-sent history. The runs carry no token log.
Multi-modal agents have made significant progress in interacting with open worlds. However, their ability to self-evolve remains limited. Supervised fine-tuning enables imitation but not active experience organization, while outcome-based reinforcement learning promotes exploration yet underuses intermediate trajectory information. Therefore, we propose EvolvingBrain, which uses an adaptive skill system as the brain for agent self-evolution, consisting of two following components. Stratified Skill Self-Evolution: extracts Procedural Patterns and Problematic Patterns from model-specific trajectories, summarizes experience by generating skills, and evolves through multiple interactions with the open world, ultimately forming a Stratified Skill Architecture consisting of Domain, Plan, and Action/Trap Skills, which provides the agent with a multi-granularity knowledge repository and a novel evolution approach. State-Aware Adaptive Execution: summarizes long-horizon interactions into Structured Abstractive Memory and uses it to guide the State-Aware Skill Fusion Strategy, enabling the agent to utilize knowledge according to its current state. Experiments on VAB-Minecraft and DiscoveryWorld demonstrate that EvolvingBrain improves success rates on open-world tasks and exhibits generalization capabilities. Our code is provided in the supplementary materials and will be publicly released in the camera-ready version.
Runxuan Liu, Jingqi Shi, Baoxin Wang +10
Harbin Institute of Technology · iFLYTEK · Meituan +1
Autonomous agents are increasingly deployed across sequences of related tasks, yet they often fail to retain and improve their knowledge over time. Existing approaches convert interaction trajectories into memories or reusable skills, but experience is not automatically reliable knowledge. Automatically collected skills and memories can propagate errors, mislead, and corrupt useful knowledge. To address this gap, we introduce Knowledge Evolution from Expertise and Lessons (KEEL), an agent-agnostic framework that uses verifier evidence to transform successful behavior into reusable skills and failures into corrective lessons. As new evidence accumulates, individual skill claims can be supported, contradicted, replaced, or retired. On SkillFlow benchmark, KEEL improves task success by 4.2 to 21.7 percentage points over the no-knowledge baseline, compared with gains of 2.4 to 9.6 points from SkillFlow's default method. Ablations show that skills and lessons are complementary. These results show that reliable continual agent learning depends not simply on accumulating experience, but on continually validating and evolving the knowledge derived from it. Our code is available at: https://github.com/microsoft/KEEL-TTCL26.
Wenzhe Fan, Benjamin Steenhoek, Gaurav Mittal +1
University of Illinois Chicago, IL, USA · Microsoft, Redmond, WA, USA
Autonomous agents operating in open-world tasks -- where the completion boundary is not given in advance -- face denominator blindness: they systematically underestimate the scope of the target space. Forage V1 addressed this through co-evolving evaluation (an independent Evaluator discovers what "complete" means) and method isolation (Evaluator and Planner cannot see each other's code). V2 extends the architecture from a single expedition to a learning organization: experience accumulates across runs, transfers across model capabilities, and institutional safeguards prevent knowledge degradation. We demonstrate two claims across three task types (web scraping, API queries, mathematical reasoning). Knowledge accumulation: over six runs, knowledge entries grow from 0 to 54, and denominator estimates stabilize as domain understanding deepens. Knowledge transfer: a weaker agent (Sonnet) seeded with a stronger agent's (Opus) knowledge narrows a 6.6pp coverage gap to 1.1pp, halves cost (9.40 to 5.13 USD), converges in half the rounds (mean 4.5 vs. 7.0), and three independent seeded runs arrive at exactly the same denominator estimate (266), suggesting organizational knowledge calibrates evaluation itself. V2's contribution is architectural: it designs institutions -- audit separation, contract protocols, organizational memory -- that make any agent more reliable upon entry. The accumulated experience is organizational, model-agnostic, and transferable, stored as readable documents that any future agent inherits regardless of provider or capability level.