cs.CLSep 26, 2026
SaveExpVoyager: Direct Experience Navigation for Dynamic Agent Skill Synthesis
Organizations: Yonsei University
Abstract
Learning from experience in LLM agents has become a key paradigm for developing self-evolving agents that continuously learn and expand their capabilities. Within this paradigm, synthesizing the agent skill has emerged as a promising solution for transforming accumulated experience into reusable procedural knowledge, serving as an important layer for the harness system that supplies agents at runtime. Despite its potential, existing approaches largely abstract past experience into fixed procedural knowledge before downstream demands are known, which risks discarding knowledge that later becomes critical while retaining instance-specific details irrelevant to future tasks. In this paper, we reframe agent skill synthesis as a dynamic navigation problem over past experience, where agents actively explore accumulated trajectories on demand for the current task with targeted and fine-grained access to experience knowledge. To this end, we propose ExpVoyager, a novel framework in which a skill curator navigates raw experience across different views and resolutions, continually identifying reusable procedural knowledge from what it observes while tracking remaining knowledge needs that guide where to navigate next. Extensive experiments demonstrate both the effectiveness and versatility of ExpVoyager, showing consistent improvements in downstream task performance, continual gains as the experience space scales, and practical compatibility with existing skills under efficient experience access.
Figures & tables
Figure 1: Experiments on how much knowledge useful for future tasks is preserved in skills. Here, we use raw experience as the 100% reference for preservation.
Figure 2: Experiments on how effectively existing experience retrieval methods access target-relevant knowledge across different retrieval depths . For Ours, is the average number of unique sources accessed during navigation.
Figure 3: Conceptual overview of ExpVoyager .
| Methods | ALFWorld | WebShop | ScienceWorld | |||||
|---|---|---|---|---|---|---|---|---|
| SR ( ) | Steps ( ) | SR ( ) | Score ( ) | Steps ( ) | SR ( ) | Score ( ) | Steps ( ) | |
| Executor : Qwen3.5-9B | ||||||||
| Base (ReAct) ( Yao et al., 2023 ) | 41.4 1.3 | 34.1 | 19.6 0.5 | 50.3 0.4 | 7.1 | 27.0 1.3 | 37.2 1.2 | 11.5 |
| + AWM ( Wang et al., 2025d ) | 45.6 1.8 | 35.1 | 24.2 0.4 | 52.0 1.3 | 7.9 | 32.6 1.7 | 42.7 1.0 | 8.4 |
| + RBank ( Ouyang et al., 2026b ) | 43.5 2.2 | 33.9 | 20.3 1.4 | 41.6 1.7 | 9.9 | 29.6 0.6 | 34.1 0.6 | 7.9 |
| + Trace2Skill ( Ni et al., 2026 ) | 45.4 1.7 | 34.3 | 22.6 0.7 | 45.7 1.0 | 9.1 | 31.9 1.3 | 37.6 0.7 | 9.8 |
Table 1: Main results on ALFWorld, WebShop, and ScienceWorld, using Qwen3.5-9B as curator .
| Methods | Curator | ALFWorld | WebShop | SciWorld | |||
|---|---|---|---|---|---|---|---|
| SR | Steps | SR | Steps | SR | Steps | ||
| Base | – | 53.9 | 29.2 | 24.7 | 7.7 | 48.2 | 14.9 |
| + ExpVygr | Qwen3.5-9B | 69.6 | 22.0 | 37.9 | 6.1 | 55.6 | 11.3 |
| + ExpVygr | Gemma4-31B | 71.3 | 21.8 | 38.4 | 5.7 | 60.8 | 11.2 |
| + ExpVygr | GPT-5.4-mini | 75.9 | 19.7 | 38.1 | 5.5 | 62.1 | 13.1 |
Table 2: Performance comparison of diverse curator backbones with ExpVygr using a Gemma4-31B executor .
Figure 4: Online evaluation on ALFWorld and ScienceWorld. The x-axis shows task stream progress, and the y-axis reports cumulative success gain over the base agent.
Figure 7: Comparison with agentic search extensions of baselines under matched experience-access budgets. denotes navigation rounds, and E2E token spend cover the full task-time pipeline.
Figure 9: Experience-access behavior of ExpVoyager . For (b), knowledge types are classified from the open question at each navigation round.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Example of a Navigation State update in ALFWorld. The curator uses navigation observation to revise knowledge and open questions , while newspaper availability in the target remains unverified. Orange and green highlight prior and updated content, respectively; matching numbers link corresponding content, and marks a new question.
Figure 11: Generated skill.md for the same ALFWorld task. Synthesized after the final navigation round, the skill provides a procedure for locating and placing a newspaper, with checks for source availability, identifier accuracy, and placement recovery. The full generated Markdown is shown.
| Methods | Pick | Look | Clean | Heat | Cool | Pick2 | Avg. SR | Steps |
|---|---|---|---|---|---|---|---|---|
| Executor : Qwen3.5-9B | ||||||||
| Base (ReAct) ( Yao et al., 2023 ) | 70.5 1.6 | 48.7 4.4 | 32.1 2.1 | 22.9 3.6 | 21.3 2.3 | 52.8 2.4 | 41.4 1.3 | 34.1 |
| + AWM ( Wang et al., 2025d ) | 67.6 3.3 | 48.7 4.4 | 49.4 2.2 | 35.4 3.6 | 32.0 4.0 | 40.3 2.4 | 45.6 1.8 | 35.1 |
| + RBank ( Ouyang et al., 2026b ) | 83.8 1.6 | 41.0 4.4 | 34.6 2.2 | 18.8 6.3 | 41.3 2.3 | 41.7 4.2 | 43.5 2.2 | 33.9 |
| + Trace2Skill ( Ni et al., 2026 ) | 65.7 2.9 | 38.5 0.0 | 55.6 3.7 | 25.0 6.2 | 33.3 2.3 | 54.2 0.0 | 45.4 1.7 | 34.3 |
| + SkillTTA ( Wang et al., 2026 ) | 81.0 1.7 | 30.8 7.7 | 67.9 4.3 | 29.2 3.6 | 54.7 2.3 | 43.1 2.4 | 51.1 2.8 | 30.4 |
Table 4: Results on ALFWorld across different subtask types, using Qwen3.5-9B as the curator .
| Budget ( ) | ExpVoyager | AWM | ReasoningBank | SkillTTA | ||||
|---|---|---|---|---|---|---|---|---|
| Tokens | SR | Tokens | SR | Tokens | SR | Tokens | SR | |
| ALFWorld No-skill: 51,880.50 tokens/task; SR 41.4%. | ||||||||
| One-shot | – | – | 110,392.94 | 45.6 | 94,361.06 | 43.5 | 123,427.34 | 51.1 |
| B01 ( ) | 159,389.51 | 60.4 | 156,245.11 | 47.9 | 161,216.50 | 47.0 | 162,389.29 | 53.1 |
| B02 ( ) | 239,787.24 | 62.9 | 244,231.09 | 48.8 | 233,414.12 | 45.5 | 239,236.17 | 51.9 |
| B03 ( ) | 315,366.21 | 63.4 | 282,331.08 | 47.2 | 300,416.35 | 46.0 | 313,346.31 | 48.9 |
Table 5: Detailed results on the ALFWorld and ScienceWorld in Figure 7 . Tokens denote End-to-end tokens per task across the full task-time pipeline. B01–B06 index budget groups, and is the number of ExpVoyager navigation rounds. Baselines use iterative retrieval in B01–B06. Following each works original setup, one-shot settings are top-3 for AWM and SkillTTA and top-1 for ReasoningBank. Bold marks the highest SR within each budget group for each benchmark.
| Benchmark | Method | SR / Score | E2E tokens/task |
|---|---|---|---|
| ALFWorld | ExpVoyager (alone) | 63.7 | 385,314 |
| AWM + ExpVoyager | 65.3 | 320,103 | |
| ReasoningBank + ExpVoyager | 64.9 | 298,582 | |
| Trace2Skill + ExpVoyager | 59.6 | 293,893 | |
| WebShop | ExpVoyager (alone) | 61.2 | 390,997 |
| AWM + ExpVoyager | 64.24 | 329,760 |
Table 6: Comparison of standalone ExpVoyager and its combinations with pre-constructed skills, corresponding to the lower panels of Figure 6 . Performance denotes success rate (%) on ALFWorld and score on WebShop, with higher values indicating better performance. E2E tokens are reported per task and cover the full task-time pipeline, including navigation and execution. Gray rows denote combinations with pre-constructed skills. Bold indicates improvements over standalone ExpVoyager within each benchmark: higher performance or lower token cost.
Figure 12: Successful example (ALFWorld): distinguishing source-specific observations from target-relevant knowledge. The absence of a coffee table in the target’s initial observation redirects navigation from the placement action to alternative newspaper locations. The curator interprets a step-level search_exp match together with the cross-step context returned by inspect_traj , distinguishing a reusable search-and-placement procedure from a source-specific object location. The final skill retains a location check and fallback search as execution guidance.
Figure 13: Successful example (ALFWorld): connecting actions to their observed results. Through inspect_traj , the curator connects observations of the same soap bar before and after cleaning, while recorded reasoning in a step-level match clarifies when the cleaning action can proceed. The synthesized skill preserves cleaning before placement and retains verification of its result as an execution-time check. The frozen executor follows this ordering and completes the task.
Figure 14: Successful example (ALFWorld): preserving action ordering through cross-step context. The cross-step context returned by inspect_traj distinguishes an early placement from the final placement after cooling. A step-level search_exp match supplies a concrete cooling command. The curator interprets these navigation observations together and retains the required cooling-before-placement ordering in the synthesized skill.
Figure 15: Successful example (WebShop): retaining target conditions as execution-time checks. Recorded reasoning accessed through search_exp warns against purchasing a default variant, while inspect_traj exposes a purchase without option selection after opening a new product. The synthesized skill distinguishes selectable options from descriptive attributes and retains the corresponding execution-time checks. The frozen executor selects the requested color and size before purchasing.
Figure 16: Successful example (WebShop): checking target conditions against available options. Navigation observations support execution-time checks of required attributes and recovery guidance for replacing an unsuitable product. During target execution, grey appears in the first product’s description but is unavailable among its selectable colors. The frozen executor switches to another candidate and selects its grey option before purchasing.
Figure 17: Successful example (ScienceWorld): identifying required conditions across source trajectories. The curator interprets a failed sandwich trajectory together with recorded reasoning from another recipe to identify that the ingredients must share a container before mixing. The synthesized skill turns this procedural knowledge into explicit ingredient-transfer and mixing actions. The frozen executor moves the ingredients into a cup and successfully produces the sandwich.
Figure 18: Successful example (ScienceWorld): interpreting recorded reasoning for the current target. A search_exp match over the reasoning field in the step-level view exposes the comparison purpose behind examining additional animals, while inspect_traj establishes the required focus-action order. The curator separates source-specific candidates from the reusable comparison procedure. The synthesized skill guides the frozen executor to apply this procedure to the animals available in the current environment.
Figure 19: Failure example (ALFWorld): losing action ordering during skill synthesis. Navigation observations show that acquisition and placement should alternate, but the final skill requires both remote controls to be acquired before placement. The frozen executor continues searching while holding one remote and never issues a placement action.
Figure 20: Failure example (WebShop): failing to recheck target conditions during execution. The frozen executor selects the requested color and size but later purchases after reopening the product without reselecting those options. The empty options map in the purchase record shows that earlier selections do not establish the required state at purchase time.
Figure 21: Failure example (ScienceWorld): translating required conditions into execution guidance. The Navigation State retains the requirement that ingredients share a container, but the final skill incorrectly proposes mix kitchen as an ingredient-transfer action. The frozen executor repeatedly attempts to mix without first moving the ingredients into a common container. The required condition is identified during navigation but is not translated into actions that establish it.
More in this topic
LLM Agent Skill Learning
Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.
See all 402 papers