Long-horizon agent interactions generate useful but noisy experience, and retraining models to absorb it is expensive. Context-evolving agents therefore need memory extraction methods that improve with more test-time compute without relying on gold labels. We propose RefCon, which combines sequential self-refinement with parallel self-contrast to extract higher-quality memories without gold labels. Evaluated on AppWorld and BFCL-V3 across multiple context-evolving agent frameworks, RefCon delivers strong and consistent gains, including relative improvements of 21.6% on ACE and 16.6% on ReMe over no-scaling baselines, while a diversity-focused variant (DivCon) achieves a 35.5% gain on ReasoningBank. RefCon consistently outperforms existing baselines without ground-truth labels, and generalizes across model scales and to software engineering tasks, where it surpasses even ground-truth baselines. We further analyze the accuracy-token trade-off and scaling behavior, showing RefCon maintains favorable efficiency and continues to improve as more trajectories are used, unlike diversity-only scaling which saturates earlier.
Figures & tables
Figure 1: Memory extraction with (a) parallel scaling using self-contrast, and (b) sequential scaling with self-refine, compared to our proposed method (c) RefCon, combining both strength of parallel and sequential scaling.
Figure 2: Overview of context-evolving agent workflow utilizing RefCon scaling for memory extraction.
Methods
AppWorld
BFCL-V3
Avg@3
Pass@3
Avg@3
Pass@3
TGC
SGC
TGC
SGC
Baseline
ReAct
63.77±4.60
35.10±10.80
87.72
68.42
52.67±4.11
62.00
With Gold Labels/Ground-Truth
ACE (with Ground-Truth)
78.00±5.34
50.9±8.96
96.49
89.47
66.67±4.71
82.00
Table 1: Performance of context-evolving agent methods on AppWorld and BFCL-V3 Using GLM-4.6.
Methods
AppWorld
Avg@3
Pass@3
TGC
SGC
TGC
SGC
ACE
ACE (Without Scaling)
59.63±2.9
42.10±4.33
77.19
73.68
ACE (Parallel Scaling)
63.73±9.55
43.87±13.83
84.21
78.95
ACE (Sequential Scaling)
64.30±1.55
42.10±7.45
84.21
78.95
Table 2: Comparison of RefCon and DivCon against MaTTS methods for context-evolving agents on AppWorld.
Figure 3: Relative improvement of compared to ReAct baseline across different frameworks and LLMs.
Method
Iter 1
Iter 2
Iter 3
Methods Without Refinement
ReAct
50.67±1.15
—
—
RB (Par)
54.00±2.00
—
—
ACE (w/ GT)
57.33±1.15
—
—
Methods With Refinement
Self-Refine
50.67±1.15
52.67±1.15
54.00±0.00
Table 3: Performance of context-evolving agent methods on SWE-Bench-Verified (Mini) using GLM-4.6.
Figure 4: Accuracy (Avg@3) and prompt token tradeoff for each MaTTS methods.
Figure 5: Scaling of RefCon and DivCon.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Avg@3 improvement across similar task within the same scenario on AppWorld sorted by chronological run order.
Methods
AppWorld
BFCL-V3
Avg@3
Pass@3
Avg@3
Pass@3
TGC
SGC
TGC
SGC
Baseline
ReAct
73.68±2.05
45.61±2.48
92.98
78.95
61.33±0.94
72.00
With Gold Labels/Ground-Truth
ACE (with Ground-Truth)
80.12±2.19
54.39±4.96
96.49
89.47
69.33±3.40
84.00
Appendix
Table 4: Performance of context-evolving agent methods on AppWorld and BFCL-V3 Using Gemma 4 31B.
Methods
AppWorld
BFCL-V3
Avg@3
Pass@3
Avg@3
Pass@3
TGC
SGC
TGC
SGC
Baseline
ReAct
18.71±2.19
10.53±4.30
29.82
15.79
53.33±3.40
72.00
With Gold Labels/Ground-Truth
ACE (with Ground-Truth)
18.13±4.60
8.77±2.48
29.82
15.79
56.00±4.32
80.00
Appendix
Table 5: Performance of context-evolving agent methods on AppWorld and BFCL-V3 Using Qwen3.5 9B.
Methods
AppWorld
Avg@3
Pass@3
TGC
SGC
TGC
SGC
ACE
First Run
68.40±5.72
43.87±6.57
85.96
68.42
Second Run (First Refinement)
70.17±6.57
42.1±8.57
92.98
84.21
Third Run (Second Refinement)
72.53±5.82
43.87±6.57
92.98
78.95
Appendix
Table 6: RefCon accuracy across runs for each method.
Methods
AppWorld
Avg@3
Pass@3
TGC
SGC
TGC
SGC
ACE
First Run
69.00±2.16
42.10±4.33
91.23
78.95
Second Run (First Alternative)
65.50±2.98
40.37±6.57
89.47
73.68
ReasoningBank
Appendix
Table 7: DivCon accuracy across runs for each method.
Section: strategies_and_hard_rules
Content: For Spotify authentication, always call apis.supervisor.show_account_passwords() to retrieve stored credentials instead of creating dummy passwords.
Section: strategies_and_hard_rules
Content: When collecting songs across multiple libraries (songs, albums, playlists), use set() to deduplicate song_ids from all sources before processing.
Section: strategies_and_hard_rules
Appendix
Table 8: Example of memories extracted using ACE with RefCon scaling.
Title: Authenticate Before Accessing Private Data
Description: Always authenticate with the service before attempting to access private user data like libraries.
Content: When working with music streaming services, private library data (songs, albums, playlists) requires authentication. First explore available APIs, then use stored credentials to obtain an access token before making any data retrieval requests.
Title: Handle Pagination for Complete Data Retrieval
Description: Implement pagination loops to retrieve all items from paginated API responses.
Content: Many APIs return data in pages with limits. Use while loops that increment page_index until an empty response is returned, collecting all items across pages to ensure complete data retrieval.
Appendix
Table 9: Example of memories extracted using ReasoningBank with RefCon scaling.
Original Memory
When To Use: When working with paginated API responses
Content: All trajectories successfully implemented pagination with while loops and page_index incrementing. This consistent pattern across all successful trajectories shows that proper pagination handling is critical for complete data retrieval. The pattern of checking for empty responses to break the loop is reliable.
When To Use: When handling authentication tokens in multi-step workflows
Content: All trajectories successfully extracted and reused the access_token from login responses across multiple API calls. The consistent pattern of storing spotify_access_token = login_result[’access_token’] and passing it to subsequent calls demonstrates proper token management. None of the trajectories encountered token expiration issues, suggesting the token lifespan was sufficient for the workflow duration.
Appendix
Table 10: Example of memories extracted using ReMe with RefCon scaling.
Retrieved Memory (Sequential Scaling)
When To Use: When you encounter verification issue in Venmo
Content: If you encounter issues with optional verification steps (like checking Venmo status), proceed with the main task (sending messages) to ensure completion. The primary goal of delivering the message is more important than perfect filtering, especially when verification may be unreliable.
When To Use: When filtering contacts by a condition and finding no matches, but the user has requested to send a specific message
Content: If no recipients match the specified filter criteria, consider alternative interpretations of the user’s request before concluding the task. The message content itself may provide clues about the intended audience.
Retrieved Memory (RefCon)
Appendix
Table 11: Retrieved memory comparison of sequential (failed) and RefCon (success) scaling.
Sequential Scaling (Failed)
RefCon (Successful)
… Assistant: contacts_without_venmo = [] for contact in all_contacts: email = contact[’email’] search_result = apis.venmo.search_users( access_token=venmo_access_token, query=email, page_limit=1) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [] Assistant: contacts_without_venmo = [] for contact in all_contacts: email = contact[’email’] search_result = apis.venmo.search_users( access_token=venmo_access_token, query=email) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [] Assistant: contacts_without_venmo = [] for contact in all_contacts: full_name = f"{contact[’first_name’]} {contact[’last_name’]}" search_result = apis.venmo.search_users( access_token=venmo_access_token, query=full_name) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [] …
… Assistant: all_contacts = siblings + roommates contacts_without_venmo = [] for contact in all_contacts: search_result = apis.venmo.search_users( access_token=venmo_access_token, query=contact[’email’]) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [] Assistant: print(apis.venmo.search_users( access_token=venmo_access_token, query=’kevin.blac@gmail.com’)) Environment: [{"first_name": "Kevin", "email": "kevin.blac@gmail.com", …}] Assistant: all_contacts = siblings + roommates contacts_without_venmo = [] for contact in all_contacts: search_result = apis.venmo.search_users( access_token=venmo_access_token, query=contact[’email’]) found = any(user[’email’] == contact[’email’] for user in search_result) if not found: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [{"contact_id": 801, "first_name": "Martin", …}] …
Appendix
Table 12: Trajectory comparison with memory retrieved from sequential (failed) and RefCon (success) scaling.
Step
Successful Trajectory
Failed Trajectory
1–2
(Identical) Retrieve account passwords; log in to Spotify to obtain access_token .
3–7
(Identical) Inspect API schemas for show_song , show_song_privates , show_song_library , show_album_library , show_playlist_library .
8–11
(Identical) Collect 79 unique song IDs from all three libraries; filter to 18 R&B songs via show_song() genre check.
12
Read play_count from show_song ⬇ rb_songs_with_play_count = [] for song_id in all_song_ids : song_info = apis . spotify . show_song ( song_id = song_id ) if song_info and ’ r & b ’ in song_info [’ genre ’]. lower (): rb_songs_with_play_count . append ({ ’ title ’: song_info [’ title ’], ’ play_count ’: song_info [’ play_count ’] }) show_song returns a rich public schema including play_count : {song_id, title, album_id, duration, artists, genre, play_count , rating, ...} Genre filtering and play count retrieval are done in a single loop . All 18 R&B songs receive their correct play counts.
Read play_count from show_song_privates ⬇ rb_songs_with_play_count = [] for song in rb_song_details : song_id = song [’ song_id ’] private_info = apis . spotify . show_song_privates ( song_id = song_id , access_token = spotify_access_token ) if private_info : play_count = private_info . get (’ play_count ’, 0) rb_songs_with_play_count . append ({ ’ title ’: song [’ title ’], ’ play_count ’: play_count }) show_song_privates only exposes per-user interaction flags: {liked, reviewed, in_song_library, downloaded} There is no play_count field. The .get(’play_count’, 0) fallback silently returns 0 for all 18 songs , producing no error and no warning.
13
Sorted by real play counts: ⬇ [" Mysteries of the Silent Sea ", " Crimson Veil ", " Haunted Memories ", " Fire and Ice "]
All counts equal 0; order is arbitrary: ⬇ [" Shadows of the Past ", " When Fate Becomes a Foe ", " The Curse of Loving You ", " Lost in a Moment ’ s Grace "]
Appendix
Table 13: Step-by-step comparison of the successful and failed trajectories. Steps 1–11 are identical in both; the slight divergence occurs at Step 12, propagating to a wrong final answer at Step 13.
Self-Contrast Prompt
You are an expert AI analyst comparing multiple step sequences which might be successful or failed to extract differential insights.
Your task is to compare and contrast these trajectories to identify the most useful and generalizable strategies as memory items using self-contrast reasoning.
Focus on critical decision points, technique variations, and approach differences.
COMPARATIVE ANALYSIS FRAMEWORK:
- DECISION CONTRAST: Compare critical decisions made in success vs failure cases
- TECHNIQUE VARIATIONS: Identify different approaches and their outcomes
Appendix
Table 14: Memory extraction prompt used for our context-evolving agent utilizing RefCon. We adopt ReMe memory format, but utilize self-contrast reasoning to extract memory.
Self-Refine Prompt for AppWorld
Let’s carefully re-examine the previous trajectory, including your reasoning steps and action taken. Pay special attention to whether you used the best API sequence and whether you used the API correctly. If you find inconsistencies, correct them. If everything seems correct, make it more efficient. Now, solve the same problem again from scratch.
Self-Diversity Prompt for AppWorld
Below is the previous trajectory, the solution might be correct or wrong. Now solve the same problem using a DIFFERENT reasoning approach. Focus on exploring alternative strategies.
Appendix
Table 15: RefCon and DivCon trajectory generation prompt for AppWorld.
Critique Trajectory Prompt for Self-Refine
You are an expert reviewer analyzing an AI assistant’s multi-turn tool-calling trajectory. Your job is to identify mistakes, missed actions, and suboptimal decisions. For each turn in the trajectory, evaluate:
1. Did the assistant call the appropriate tools? If a user requested an action (e.g., book, cancel, update), did the assistant actually make a tool call, or did it just respond with text?
2. Were the tool arguments correct? Check for wrong parameter values, missing required arguments, or arguments that contradict the user’s request.
3. Did the assistant use information from previous tool responses correctly? For example, if a lookup returned an ID, did the assistant use that ID in subsequent calls?
4. Were there any unnecessary or redundant tool calls?
5. Did the assistant follow the logical sequence of operations? (e.g., lookup before booking, authenticate before accessing protected resources)
Appendix
Table 16: RefCon and DivCon trajectory generation prompt for BFCL-V3.
Critique Trajectory Prompt for Self-Refine
You are an expert reviewer analyzing an AI assistant’s multi-turn software engineering trajectory. Your job is to identify mistakes, missed files, and suboptimal debugging decisions. For each turn in the trajectory, evaluate:
1. Did the assistant explore the repository effectively? Did it locate the relevant source files and classes, or did it waste turns on unrelated directories?
2. Was the bug localization accurate? Check if the assistant correctly identified the root cause of the issue before attempting a fix.
3. Did the assistant use the environment and test tools correctly? For example, if a reproduction script was created, did the assistant analyze the output to guide the patch?
4. Was the generated patch functional and minimal? Identify if the assistant introduced unnecessary changes or failed to follow the repository’s coding style.
5. Did the assistant follow a logical debugging sequence? (e.g., search → reproduce → fix → verify via tests)
Appendix
Table 17: RefCon trajectory generation prompt for SWE-bench Verified (Mini).
Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce EpiCon, a shared multimodal memory framework for agent collective learning without updating host model parameters. EpiCon links question-level memory evolution to a persistent experience bank through two independently trained 2B models: a memory controller and a tree self-organizer. The controller jointly refines textual guidance and visual evidence across attempts and selectively includes visual memory. The self-organizer consolidates lessons hierarchically and retrieves experience and rules for new problems. We evaluate EpiCon on eleven benchmarks spanning four multimodal task domains, using two harnesses and multiple backbones. A frozen bank improves other systems even with a single solving attempt. A second harness raises the original system's macro-average score by 2.6 points across eleven benchmarks. Across four host configurations, EpiCon improves macro-average scores by 1.7 to 4.9 points over No Memory and reduces memory-operation time by 67% to 74% relative to backbone-sized memory models.
Ziyun Zeng, Hang Hua, Shaden Alshammari +3
MIT-IBM Computing Research Lab · University of Rochester · Massachusetts Institute of Technology
Long-horizon large language model (LLM) agents are fundamentally limited by context. As interactions become longer, tool descriptions, retrieved memories, and raw environmental feedback accumulate and push out the information needed for decision-making. At the same time, useful experience gained from tasks is often lost across episodes. We argue that long-horizon performance is determined not by context length, but by how much decision-relevant information is maintained within a finite context budget. We present GenericAgent (GA), a general-purpose, self-evolving LLM agent system built around a single principle: context information density maximization. GA implements this through four closely connected components: a minimal atomic tool set that keeps the interface simple, a hierarchical on-demand memory that only shows a small high-level view by default, a self-evolution mechanism that turns verified past trajectories into reusable SOPs and executable code, and a context truncation and compression layer that maintains information density during long executions. Across task completion, tool use efficiency, memory effectiveness, self-evolution, and web browsing, GA consistently outperforms leading agent systems while using significantly fewer tokens and interactions, and it continues to evolve over time. Project: https://github.com/lsdefine/GenericAgent
Recent benchmarks for Large Language Model (LLM) agents mainly evaluate reasoning, planning, and execution. However, memory is also essential for agents, as it enables them to store, update, and retrieve information over time. This ability remains under-evaluated, largely because existing benchmarks do not provide a systematic way to assess memory mechanisms. In this paper, we study agent memory from a self-evolving perspective and introduce EvoMemBench, a unified benchmark organized along two axes: memory scope (in-episode vs. cross-episode) and memory content (knowledge-oriented vs. execution-oriented). We compare 15 representative memory methods with strong long-context baselines under a standardized protocol. Results show that current memory systems are still far from a general solution: long-context baselines remain highly competitive, memory helps most when the current context is insufficient or tasks are difficult, and no single memory form works consistently across all settings. Retrieval-based methods remain strong for knowledge-intensive settings, whereas procedural and long-term memory methods are more effective for execution-oriented tasks when their stored experience matches the task structure. We hope EvoMemBench facilitates future research on more effective memory systems for LLM-based agents. Our code is available at https://github.com/DSAIL-Memory/EvoMemBench.
Yuyao Wang, Zhongjian Zhang, Mo Chi +7
Hong Kong University of Science and Technology (Guangzhou) · Beijing University of Posts and Telecommunications · Beijing Institute of Technology +1