RefCon: Iterative Refinement and Contrastive Memory Extraction for Context-Evolving Agent

Ubaidillah Ariq Prathama; Bo Liu; Yeo Boon Hong; Yu-Xuan Huang; Yangkai Ding; Tao Yu

cs.AISep 30, 2026

RefCon: Iterative Refinement and Contrastive Memory Extraction for Context-Evolving Agent

Authors: Ubaidillah Ariq Prathama, Bo Liu, Yeo Boon Hong, Yu-Xuan Huang, Yangkai Ding, Tao Yu

Organizations: Huawei Technologies, Co., Ltd.

Abstract

Long-horizon agent interactions generate useful but noisy experience, and retraining models to absorb it is expensive. Context-evolving agents therefore need memory extraction methods that improve with more test-time compute without relying on gold labels. We propose RefCon, which combines sequential self-refinement with parallel self-contrast to extract higher-quality memories without gold labels. Evaluated on AppWorld and BFCL-V3 across multiple context-evolving agent frameworks, RefCon delivers strong and consistent gains, including relative improvements of 21.6% on ACE and 16.6% on ReMe over no-scaling baselines, while a diversity-focused variant (DivCon) achieves a 35.5% gain on ReasoningBank. RefCon consistently outperforms existing baselines without ground-truth labels, and generalizes across model scales and to software engineering tasks, where it surpasses even ground-truth baselines. We further analyze the accuracy-token trade-off and scaling behavior, showing RefCon maintains favorable efficiency and continues to improve as more trajectories are used, unlike diversity-only scaling which saturates earlier.

Figures & tables

Figure 1: Memory extraction with (a) parallel scaling using self-contrast, and (b) sequential scaling with self-refine, compared to our proposed method (c) RefCon, combining both strength of parallel and sequential scaling.

Figure 2: Overview of context-evolving agent workflow utilizing RefCon scaling for memory extraction.

Methods	AppWorld				BFCL-V3
	Avg@3		Pass@3		Avg@3	Pass@3
	TGC	SGC	TGC	SGC
Baseline
ReAct	$63.77^{\pm 4.60}$	$35.10^{\pm 10.80}$	87.72	68.42	$52.67^{\pm 4.11}$	62.00
With Gold Labels/Ground-Truth
ACE (with Ground-Truth)	$\textbf{78.00}^{\pm\textbf{5.34}}$	$50.9^{\pm 8.96}$	96.49	89.47	$66.67^{\pm 4.71}$	82.00

Table 1: Performance of context-evolving agent methods on AppWorld and BFCL-V3 Using GLM-4.6.

Methods	AppWorld
	Avg@3		Pass@3
	TGC	SGC	TGC	SGC
ACE
ACE (Without Scaling)	$59.63^{\pm 2.9}$	$42.10^{\pm 4.33}$	77.19	73.68
ACE (Parallel Scaling)	$63.73^{\pm 9.55}$	$43.87^{\pm 13.83}$	84.21	78.95
ACE (Sequential Scaling)	$64.30^{\pm 1.55}$	$42.10^{\pm 7.45}$	84.21	78.95

Table 2: Comparison of RefCon and DivCon against MaTTS methods for context-evolving agents on AppWorld.

Figure 3: Relative improvement of compared to ReAct baseline across different frameworks and LLMs.

Methods Without Refinement
Method	Iter 1	Iter 2	Iter 3
ReAct	$50.67^{\scriptstyle\pm 1.15}$	—	—
RB (Par)	$54.00^{\scriptstyle\pm 2.00}$	—	—
ACE (w/ GT)	$57.33^{\scriptstyle\pm 1.15}$	—	—
Methods With Refinement
Self-Refine	$50.67^{\scriptstyle\pm 1.15}$	$52.67^{\scriptstyle\pm 1.15}$	$54.00^{\scriptstyle\pm 0.00}$

Table 3: Performance of context-evolving agent methods on SWE-Bench-Verified (Mini) using GLM-4.6.

Figure 4: Accuracy (Avg@3) and prompt token tradeoff for each MaTTS methods.

Figure 5: Scaling of RefCon and DivCon.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Avg@3 improvement across similar task within the same scenario on AppWorld sorted by chronological run order.

Methods	AppWorld				BFCL-V3
	Avg@3		Pass@3		Avg@3	Pass@3
	TGC	SGC	TGC	SGC
Baseline
ReAct	$73.68^{\pm 2.05}$	$45.61^{\pm 2.48}$	92.98	78.95	$61.33^{\pm 0.94}$	72.00
With Gold Labels/Ground-Truth
ACE (with Ground-Truth)	$\textbf{80.12}^{\pm\textbf{2.19}}$	$\textbf{54.39}^{\pm\textbf{4.96}}$	96.49	89.47	$\textbf{69.33}^{\pm\textbf{3.40}}$	84.00

Appendix

Table 4: Performance of context-evolving agent methods on AppWorld and BFCL-V3 Using Gemma 4 31B.

Methods	AppWorld				BFCL-V3
	Avg@3		Pass@3		Avg@3	Pass@3
	TGC	SGC	TGC	SGC
Baseline
ReAct	$18.71^{\pm 2.19}$	$10.53^{\pm 4.30}$	29.82	15.79	$53.33^{\pm 3.40}$	72.00
With Gold Labels/Ground-Truth
ACE (with Ground-Truth)	$\textbf{18.13}^{\pm\textbf{4.60}}$	$\textbf{8.77}^{\pm\textbf{2.48}}$	29.82	15.79	$\textbf{56.00}^{\pm\textbf{4.32}}$	80.00

Appendix

Table 5: Performance of context-evolving agent methods on AppWorld and BFCL-V3 Using Qwen3.5 9B.

Methods	AppWorld
	Avg@3		Pass@3
	TGC	SGC	TGC	SGC
ACE
First Run	$68.40^{\pm 5.72}$	$\textbf{43.87}^{\pm\textbf{6.57}}$	85.96	68.42
Second Run (First Refinement)	$70.17^{\pm 6.57}$	$42.1^{\pm 8.57}$	92.98	84.21
Third Run (Second Refinement)	$\textbf{72.53}^{\pm\textbf{5.82}}$	$\textbf{43.87}^{\pm\textbf{6.57}}$	92.98	78.95

Appendix

Table 6: RefCon accuracy across runs for each method.

Methods	AppWorld
	Avg@3		Pass@3
	TGC	SGC	TGC	SGC
ACE
First Run	$\textbf{69.00}^{\pm\textbf{2.16}}$	$\textbf{42.10}^{\pm\textbf{4.33}}$	91.23	78.95
Second Run (First Alternative)	$65.50^{\pm 2.98}$	$40.37^{\pm 6.57}$	89.47	73.68
ReasoningBank

Appendix

Table 7: DivCon accuracy across runs for each method.

Section: strategies_and_hard_rules

Content: For Spotify authentication, always call apis.supervisor.show_account_passwords() to retrieve stored credentials instead of creating dummy passwords.

Section: strategies_and_hard_rules

Content: When collecting songs across multiple libraries (songs, albums, playlists), use set() to deduplicate song_ids from all sources before processing.

Section: strategies_and_hard_rules

Appendix

Table 8: Example of memories extracted using ACE with RefCon scaling.

Title: Authenticate Before Accessing Private Data

Description: Always authenticate with the service before attempting to access private user data like libraries.

Content: When working with music streaming services, private library data (songs, albums, playlists) requires authentication. First explore available APIs, then use stored credentials to obtain an access token before making any data retrieval requests.

Title: Handle Pagination for Complete Data Retrieval

Description: Implement pagination loops to retrieve all items from paginated API responses.

Content: Many APIs return data in pages with limits. Use while loops that increment page_index until an empty response is returned, collecting all items across pages to ensure complete data retrieval.

Appendix

Table 9: Example of memories extracted using ReasoningBank with RefCon scaling.

Original Memory
When To Use: When working with paginated API responses
Content: All trajectories successfully implemented pagination with while loops and page_index incrementing. This consistent pattern across all successful trajectories shows that proper pagination handling is critical for complete data retrieval. The pattern of checking for empty responses to break the loop is reliable.

When To Use: When handling authentication tokens in multi-step workflows
Content: All trajectories successfully extracted and reused the access_token from login responses across multiple API calls. The consistent pattern of storing spotify_access_token = login_result[’access_token’] and passing it to subsequent calls demonstrates proper token management. None of the trajectories encountered token expiration issues, suggesting the token lifespan was sufficient for the workflow duration.

Appendix

Table 10: Example of memories extracted using ReMe with RefCon scaling.

Retrieved Memory (Sequential Scaling)
When To Use: When you encounter verification issue in Venmo
Content: If you encounter issues with optional verification steps (like checking Venmo status), proceed with the main task (sending messages) to ensure completion. The primary goal of delivering the message is more important than perfect filtering, especially when verification may be unreliable.

When To Use: When filtering contacts by a condition and finding no matches, but the user has requested to send a specific message
Content: If no recipients match the specified filter criteria, consider alternative interpretations of the user’s request before concluding the task. The message content itself may provide clues about the intended audience.
Retrieved Memory (RefCon)

Appendix

Table 11: Retrieved memory comparison of sequential (failed) and RefCon (success) scaling.

Sequential Scaling (Failed)

RefCon (Successful)

\ldots

Assistant: contacts_without_venmo = [] for contact in all_contacts: email = contact[’email’] search_result = apis.venmo.search_users( access_token=venmo_access_token, query=email, page_limit=1) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [] Assistant: contacts_without_venmo = [] for contact in all_contacts: email = contact[’email’] search_result = apis.venmo.search_users( access_token=venmo_access_token, query=email) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [] Assistant: contacts_without_venmo = [] for contact in all_contacts: full_name = f"{contact[’first_name’]} {contact[’last_name’]}" search_result = apis.venmo.search_users( access_token=venmo_access_token, query=full_name) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: []

\ldots

\ldots

Assistant: all_contacts = siblings + roommates contacts_without_venmo = [] for contact in all_contacts: search_result = apis.venmo.search_users( access_token=venmo_access_token, query=contact[’email’]) if not search_result: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [] Assistant: print(apis.venmo.search_users( access_token=venmo_access_token, query=’kevin.blac@gmail.com’)) Environment: [{"first_name": "Kevin", "email": "kevin.blac@gmail.com", …}] Assistant: all_contacts = siblings + roommates contacts_without_venmo = [] for contact in all_contacts: search_result = apis.venmo.search_users( access_token=venmo_access_token, query=contact[’email’]) found = any(user[’email’] == contact[’email’] for user in search_result) if not found: contacts_without_venmo.append(contact) print(contacts_without_venmo) Environment: [{"contact_id": 801, "first_name": "Martin", …}]

\ldots

Appendix

Table 12: Trajectory comparison with memory retrieved from sequential (failed) and RefCon (success) scaling.

Step	Successful Trajectory	Failed Trajectory
1–2	(Identical) Retrieve account passwords; log in to Spotify to obtain access_token .
3–7	(Identical) Inspect API schemas for show_song , show_song_privates , show_song_library , show_album_library , show_playlist_library .
8–11	(Identical) Collect 79 unique song IDs from all three libraries; filter to 18 R&B songs via show_song() genre check.
12	Read play_count from show_song ⬇ rb_songs_with_play_count = [] for song_id in all_song_ids : song_info = apis . spotify . show_song ( song_id = song_id ) if song_info and ’ r & b ’ in song_info [’ genre ’]. lower (): rb_songs_with_play_count . append ({ ’ title ’: song_info [’ title ’], ’ play_count ’: song_info [’ play_count ’] }) show_song returns a rich public schema including play_count : {song_id, title, album_id, duration, artists, genre, play_count , rating, ...} Genre filtering and play count retrieval are done in a single loop . All 18 R&B songs receive their correct play counts.	Read play_count from show_song_privates ⬇ rb_songs_with_play_count = [] for song in rb_song_details : song_id = song [’ song_id ’] private_info = apis . spotify . show_song_privates ( song_id = song_id , access_token = spotify_access_token ) if private_info : play_count = private_info . get (’ play_count ’, 0) rb_songs_with_play_count . append ({ ’ title ’: song [’ title ’], ’ play_count ’: play_count }) show_song_privates only exposes per-user interaction flags: {liked, reviewed, in_song_library, downloaded} There is no play_count field. The .get(’play_count’, 0) fallback silently returns 0 for all 18 songs , producing no error and no warning.
13	Sorted by real play counts: ⬇ [" Mysteries of the Silent Sea ", " Crimson Veil ", " Haunted Memories ", " Fire and Ice "]	All counts equal 0; order is arbitrary: ⬇ [" Shadows of the Past ", " When Fate Becomes a Foe ", " The Curse of Loving You ", " Lost in a Moment ’ s Grace "]

Appendix

Table 13: Step-by-step comparison of the successful and failed trajectories. Steps 1–11 are identical in both; the slight divergence occurs at Step 12, propagating to a wrong final answer at Step 13.

Self-Contrast Prompt
You are an expert AI analyst comparing multiple step sequences which might be successful or failed to extract differential insights.
Your task is to compare and contrast these trajectories to identify the most useful and generalizable strategies as memory items using self-contrast reasoning.
Focus on critical decision points, technique variations, and approach differences.
COMPARATIVE ANALYSIS FRAMEWORK:
- DECISION CONTRAST: Compare critical decisions made in success vs failure cases
- TECHNIQUE VARIATIONS: Identify different approaches and their outcomes

Appendix

Table 14: Memory extraction prompt used for our context-evolving agent utilizing RefCon. We adopt ReMe memory format, but utilize self-contrast reasoning to extract memory.

Self-Refine Prompt for AppWorld
Let’s carefully re-examine the previous trajectory, including your reasoning steps and action taken. Pay special attention to whether you used the best API sequence and whether you used the API correctly. If you find inconsistencies, correct them. If everything seems correct, make it more efficient. Now, solve the same problem again from scratch.
Self-Diversity Prompt for AppWorld
Below is the previous trajectory, the solution might be correct or wrong. Now solve the same problem using a DIFFERENT reasoning approach. Focus on exploring alternative strategies.

Appendix

Table 15: RefCon and DivCon trajectory generation prompt for AppWorld.

Critique Trajectory Prompt for Self-Refine
You are an expert reviewer analyzing an AI assistant’s multi-turn tool-calling trajectory. Your job is to identify mistakes, missed actions, and suboptimal decisions. For each turn in the trajectory, evaluate:
1. Did the assistant call the appropriate tools? If a user requested an action (e.g., book, cancel, update), did the assistant actually make a tool call, or did it just respond with text?
2. Were the tool arguments correct? Check for wrong parameter values, missing required arguments, or arguments that contradict the user’s request.
3. Did the assistant use information from previous tool responses correctly? For example, if a lookup returned an ID, did the assistant use that ID in subsequent calls?
4. Were there any unnecessary or redundant tool calls?
5. Did the assistant follow the logical sequence of operations? (e.g., lookup before booking, authenticate before accessing protected resources)

Appendix

Table 16: RefCon and DivCon trajectory generation prompt for BFCL-V3.

Critique Trajectory Prompt for Self-Refine
You are an expert reviewer analyzing an AI assistant’s multi-turn software engineering trajectory. Your job is to identify mistakes, missed files, and suboptimal debugging decisions. For each turn in the trajectory, evaluate:
1. Did the assistant explore the repository effectively? Did it locate the relevant source files and classes, or did it waste turns on unrelated directories?
2. Was the bug localization accurate? Check if the assistant correctly identified the root cause of the issue before attempting a fix.
3. Did the assistant use the environment and test tools correctly? For example, if a reproduction script was created, did the assistant analyze the output to guide the patch?
4. Was the generated patch functional and minimal? Identify if the assistant introduced unnecessary changes or failed to follow the repository’s coding style.
5. Did the assistant follow a logical debugging sequence? (e.g., search $\rightarrow$ reproduce $\rightarrow$ fix $\rightarrow$ verify via tests)

Appendix

Table 17: RefCon trajectory generation prompt for SWE-bench Verified (Mini).

Explore similar work

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory
Sep 29, 2026Ziyun Zeng, Hang Hua, Shaden Alshammari +3Multimodal Memory
MIT-IBM Computing Research Lab · University of Rochester · Massachusetts Institute of Technology
GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
Apr 18, 2026Jiaqing Liang, Jinyi Han, Weijia Li +15Large Language Model Agents Self-Evolving Agents
Advantage AI Agent Lab (A3 Lab)
EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
May 18, 2026Yuyao Wang, Zhongjian Zhang, Mo Chi +7Self-Evolution
Hong Kong University of Science and Technology (Guangzhou) · Beijing University of Posts and Telecommunications · Beijing Institute of Technology +1

Sep 29, 2026, cs.CVJ/K move · Enter open · S save

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

Ziyun Zeng, Hang Hua, Shaden Alshammari, Rogerio Feris+2

MIT-IBM Computing Research Lab · University of Rochester · Massachusetts Institute of Technology

Multimodal Memory

Open paper AskPDF arXiv

Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce EpiCon, a shared multimodal memory framework for agent collective learning without updating host model parameters. EpiCon links question-level memory evolution to a persistent experience bank through two independently trained 2B models: a memory controller and a tree self-organizer. The controller jointly refines textual guidance and visual evidence across attempts and selectively includes visual memory. The self-organizer consolidates lessons hierarchically and retrieves experience and rules for new problems. We evaluate EpiCon on eleven benchmarks spanning four multimodal task domains, using two harnesses and multiple backbones. A frozen bank improves other systems even with a single solving attempt. A second harness raises the original system's macro-average score by 2.6 points across eleven benchmarks. Across four host configurations, EpiCon improves macro-average scores by 1.7 to 4.9 points over No Memory and reduces memory-operation time by 67% to 74% relative to backbone-sized memory models.

Abstract

Explore similar work

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)

EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective