Rethinking the Role of Tool Graph Memory in Multi-Turn Tool Use

Uiji Hwang; Richeng Xuan; Taeuk Kim

cs.CLOct 15, 2026

Rethinking the Role of Tool Graph Memory in Multi-Turn Tool Use

Authors: Uiji Hwang, Richeng Xuan, Taeuk Kim

Organizations: Hanyang University · Hunyuan, Tencent

Abstract

Large language model (LLM) agents solve complex tasks by calling external tools over multiple turns. Recent work has proposed tool graph memory, which stores past experience as tool transitions to guide subsequent selection. However, it remains unclear whether this component consistently improves agents' tool use or undermines it. We evaluate tool graph memory on nine models and two benchmarks, finding that it seldom improves significantly over baselines, and that its gains, where present, cost over twice the LLM calls. The memory's guidance is often incorrect, and random or no guidance retains most of the gain, implying that the benefit stems from extra computation rather than the memory itself. Moreover, even with the correct next tool suggested, many failures remain, such as giving up without a tool call. Storing the same experience instead as textual insights outperforms tool graph memory with almost no extra LLM calls. This suggests that memory serves better as a soft prior than as a prescription of the next tool, leading to a more effective and generalizable solution.

Figures & tables

Figure 1: An example of tool graph memory and insight memory. The tool graph never suggests the prerequisite search_holiday ; following the insight, the agent retrieves the holiday date first and succeeds.

Model	ToolSandbox				BFCL-v4
Model	Base	ToolNet	H-EPM	Insight	Base	ToolNet	H-EPM	Insight
Qwen3.5-2B	0.420	0.428	0.436	0.488 +	0.133	0.044 -	0.138	0.165
Qwen3-8B	0.500	0.507	0.476	0.556 +	0.171	0.129	0.179	0.242 +
Qwen3.6-35B-A3B	0.626	0.608	0.670	0.674	0.181	0.108 -	0.156	0.206
gemma-4-E4B-it	0.588	0.547 -	0.554	0.620	0.221	0.094 -	0.252	0.285 +
gemma-4-26B-A4B-it	0.690	0.655 -	0.705	0.705	0.566	0.369 -	0.519	0.531

Table 1: Performance of LLM agents on ToolSandbox and the multi-turn subset of BFCL-v4, averaged over three runs. The best method per benchmark is in bold . The plus (

+

) and minus

(-)

signs indicate significant improvements and decrements over Base (

p<0.05

). Standard deviations and

p

-values are reported in Table 6 .

Figure 2: Decisive turns in the scenarios where H-EPM (a) improves and (b) degrades on Mistral-Small-3.2-24B, by whether the tool suggestion is correct and followed; each cell gives the percentage and number of turns. Figure 7 shows the results of gemma-4-E4B-it.

Replacing the suggested tools of H-EPM
Conditions / Models	Mistral-Small-3.2-24B	gpt-oss-20b	gemma-4-E4B-it
Base (only LLMs)	0.545	0.579	0.588
Tool graph memory (H-EPM)	0.603 + (2.0 $\times$ )	0.620 (1.8 $\times$ )	0.554 (2.0 $\times$ )
with random tools	0.571 (2.3 $\times$ )	0.663 + (2.7 $\times$ )	0.553 (2.0 $\times$ )
with the correct tool (oracle)	0.682 + (1.6 $\times$ )	0.643 + (2.5 $\times$ )	0.666 + (2.0 $\times$ )
Removing a component of H-EPM

Table 2: Ablations on ToolSandbox, with + / - as in Table 1 . Parentheses give the # of tool calls relative to Base in cases each variant flips from failure to success.

Figure 3: Scenarios that H-EPM degrades on Mistral-Small-3.2-24B and gpt-oss-20b, by failure mode (§ 4.1 ), and the percentage recovered by insight memory.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

	Qwen3-8B			gemma-4-E4B-it
Method	Score	$\Delta$	$p$	Score	$\Delta$	$p$
Base	0.462	–	–	0.575	–	–
ToolNet	0.483	+0.021	0.364	0.573	$-$ 0.002	0.920
H-EPM	0.496	+0.034	0.277	0.559	$-$ 0.016	0.573
Insight	0.527	+0.065	0.023	0.583	+0.009	0.740

Appendix

Table 3: ToolSandbox with GPT-4.1 as the user simulator for both memory construction and evaluation (

n{=}207

scenarios).

\Delta

is the difference from Base, computed before rounding; significance is tested as in Table 6 .

Model	$n$	$K$	$n/K$
ToolSandbox
Qwen3.5-2B	77	25	3.1
Qwen3-8B	360	100	3.6
Qwen3.6-35B-A3B	146	5	29.2
gemma-4-E4B-it	261	15	17.4
gemma-4-26B-A4B-it	181	5	36.2

Appendix

Table 4: Number of clusters selected per model.

n

is the number of episodic summaries clustered,

K

the selected number of clusters, and

n/K

the mean number of summaries per cluster.

Panel of Figure 4 — Figure 4: The prompt used to extract insights from each cluster.

Model	Base	ToolNet	H-EPM	Insight
GPT-4.1	0.724	0.709	0.725	0.729
GPT-5.1	0.701	0.694	0.702	0.702

Appendix

Table 5: Task performance of frontier models on ToolSandbox (

n

= 207).

Model	Base	ToolNet	$\Delta$ ( $p$ -value)		H-EPM	$\Delta$ ( $p$ -value)		Insight	$\Delta$ ( $p$ -value)
ToolSandbox
Qwen3.5-2B	0.420 ±0.003	0.428 ±0.008	$+$ 0.008	(0.399)	0.436 ±0.015	$+$ 0.016	(0.469)	0.488 ±0.009	$+$ 0.068	(0.015)
Qwen3-8B	0.500 ±0.008	0.507 ±0.009	$+$ 0.006	(0.660)	0.476 ±0.010	$-$ 0.024	(0.405)	0.556 ±0.004	$+$ 0.056	(0.037)
Qwen3.6-35B-A3B	0.626 ±0.001	0.608 ±0.002	$-$ 0.018	(0.256)	0.670 ±0.005	$+$ 0.044	(0.055)	0.674 ±0.006	$+$ 0.048	(0.056)
gemma-4-E4B-it	0.588 ±0.012	0.547 ±0.008	$-$ 0.041	(0.004)	0.554 ±0.012	$-$ 0.034	(0.209)	0.620 ±0.003	$+$ 0.032	(0.267)
gemma-4-26B-A4B-it	0.690 ±0.000	0.655 ±0.003	$-$ 0.035	(0.026)	0.705 ±0.000	$+$ 0.016	(0.394)	0.705 ±0.009	$+$ 0.015	(0.423)

Appendix

Table 6: Full version of Table 1 (

n

= 207 scenarios and 160 tasks), with the standard deviation over the three runs in subscript.

\Delta

is the difference from Base, computed before rounding, with the

p

-value of a two-sided paired

t

-test over scenarios (ToolSandbox) or tasks (BFCL-v4) in parentheses. The Average row gives the mean

\Delta

over the nine models. The best method per benchmark is in bold .

Model	Base		ToolNet		H-EPM		Insight
Model	pass@3	pass 3	pass@3	pass 3	pass@3	pass 3	pass@3	pass 3
ToolSandbox
Qwen3.5-2B	0.401	0.362	0.406	0.372	0.411	0.348	0.488	0.415
Qwen3-8B	0.546	0.425	0.527	0.435	0.527	0.449	0.594	0.493
Qwen3.6-35B-A3B	0.667	0.589	0.618	0.556	0.715	0.618	0.729	0.628
gemma-4-E4B-it	0.628	0.488	0.546	0.469	0.580	0.527	0.676	0.556

Appendix

Table 7: pass@3 (solved in at least one of three runs) and pass 3 (solved in all three runs) on ToolSandbox (

n

= 207) and BFCL-v4 (

n

= 160). Best per model and per metric in bold .

Figure 5: LLM calls per scenario relative to Base of the same model. The Average bars give the ratios of the nine-model averages.

Model	Base	ToolNet	H-EPM	Insight
ToolSandbox LLM calls
Qwen3.5-2B	5.42	5.04 (0.93)	13.80 (2.55)	6.66 (1.23)
Qwen3-8B	4.78	4.79 (1.00)	16.36 (3.42)	5.56 (1.16)
Qwen3.6-35B-A3B	5.75	6.03 (1.05)	16.92 (2.94)	5.74 (1.00)
gemma-4-E4B-it	5.27	4.97 (0.94)	13.00 (2.47)	5.26 (1.00)
gemma-4-26B-A4B-it	5.90	5.52 (0.93)	15.47 (2.62)	6.04 (1.02)

Appendix

Table 8: Cost per ToolSandbox scenario (

n

= 207) and per BFCL-v4 task (

n

= 160), with the ratio to Base of the same model in parentheses, computed before rounding. The Average row gives the nine-model averages, with the ratios of these averages in parentheses. All calls made by the agent model are counted, including the summary calls of H-EPM and, on BFCL-v4, the scoring call of ToolNet; the user simulator is excluded. Token counts of these additional calls are not recorded on BFCL-v4, so only LLM calls are reported there.

	Mistral-Small-3.2-24B					gpt-oss-20b					gemma-4-E4B-it
Condition	$n$	helped	both-fail	$D$ [95% CI]	LLM	$n$	helped	both-fail	$D$ [95% CI]	LLM	$n$	helped	both-fail	$D$ [95% CI]	LLM
H-EPM	17/52	2.0 $\times$	1.4 $\times$	1.34 [0.57, 2.20]	2.8 $\times$	20/54	1.8 $\times$	1.0 $\times$	1.41 [0.89, 1.96]	2.4 $\times$	24/56	2.0 $\times$	1.3 $\times$	1.95 [0.82, 3.12]	2.7 $\times$
with random tools	19/50	2.3 $\times$	1.4 $\times$	1.84 [0.79, 2.97]	1.3 $\times$	27/47	2.7 $\times$	0.9 $\times$	2.66 [2.00, 3.34]	1.3 $\times$	26/54	2.0 $\times$	1.6 $\times$	1.09 [0.09, 2.08]	1.3 $\times$
with the correct tool (oracle)	43/30	1.6 $\times$	1.2 $\times$	1.30 [0.45, 2.20]	1.2 $\times$	29/41	2.5 $\times$	0.7 $\times$	2.69 [1.95, 3.45]	1.5 $\times$	40/39	2.0 $\times$	1.8 $\times$	0.77 [0.01, 1.51]	1.2 $\times$
w/o suggestions	27/45	1.7 $\times$	1.1 $\times$	1.63 [0.84, 2.48]	1.2 $\times$	23/52	2.7 $\times$	0.9 $\times$	2.55 [1.68, 3.51]	1.2 $\times$	25/55	1.9 $\times$	1.3 $\times$	1.25 [0.26, 2.27]	1.2 $\times$
w/o summarization	22/50	1.8 $\times$	1.3 $\times$	1.92 [0.93, 3.08]	2.7 $\times$	22/52	2.6 $\times$	1.0 $\times$	2.58 [1.82, 3.37]	2.2 $\times$	9/80	1.8 $\times$	1.1 $\times$	1.97 [1.34, 2.67]	2.1 $\times$

Appendix

Table 9: Tool calls in the scenarios that fail under Base, excluding runs aborted by an execution error, which leave no trajectory.

n

: number of helped / both-fail scenarios. helped / both-fail: tool calls per scenario relative to Base in the scenarios that the condition flips to success / that fail under both.

D

: difference in the per-scenario increase in tool calls between the two groups, with its 95% confidence interval. LLM: LLM calls per scenario relative to Base. All differences are significant (

p_{\mathrm{adj}}<0.05

(a) Components
Summarization	H-EPM’s summarize_the_task tool and a system-prompt line asking the agent to call it
Suggestion line	Appended to the last message of the conversation (the latest user turn or tool result), listing successors of the last invoked tool, e.g., next tool suggestion: set_location_service_status, set_wifi_status
Instruction	“IMPORTANT: Do not give a final answer or give up until you have tried the available tools. If a tool call fails or returns nothing useful, try another available tool or first resolve what it depends on, then call it again. Keep calling tools until the task is fully resolved.” (system prompt)

Appendix

Table 10: Details of the control conditions in Table 2 .

Panel of Figure 6 — Figure 6: An improved scenario on Mistral-Small-3.2-24B in which H-EPM ignores an incorrect suggestion and calls the required tool on its own, with the Base run on the same scenario. […] marks omitted calls or turns; gray lines are the suggestions shown to the agent; italic notes are ours.

Figure 7: Manual analysis of H-EPM on gemma-4-E4B-it, as in Figure 2 (

n

= 91 and 54).

Failure mode	Mistral-Small-3.2-24B	gpt-oss-20b	gemma-4-E4B-it	All
Call necessity	24	38	19	81
unnecessary call	14	3	5	22
missing call	10	35	14	59
Tool selection	3	0	3	6
Dependency	3	1	3	7
Argument	19	9	16	44

Appendix

Table 11: Failures of the oracle condition by failure mode (

n

= 49, 48, and 41 failed scenarios).

Panel of Figure 8 — Figure 8: Scenarios degraded by H-EPM on Mistral-Small-3.2-24B, labeled by failure mode (§ 4.1 ), with the Base run on the same scenario.

Panel of Figure 9 — Figure 9: Scenarios degraded by H-EPM on Mistral-Small-3.2-24B that Insight recovers, with the insight displayed before the decisive call.

Panel of Figure 10 — Figure 10: Scenarios on Mistral-Small-3.2-24B that Insight solves and Insight + H-EPM fails. The insight shown is the one displayed before the decisive call; the suggestion line is the one shown in the combined run.

Model	Insight	+ H-EPM
Model	Insight	block in system	block in last message
Mistral-Small-3.2-24B	0.620	0.600	0.587
gemma-4-E4B-it	0.620	0.617	0.596

Appendix

Table 12: Insight memory + H-EPM on ToolSandbox with the insight block in the system prompt (default) or in the last message.

Model	Insight	Insight + H-EPM
Qwen3.5-2B	0.488 (6.66)	0.528 (14.60)
Qwen3-8B	0.556 (5.56)	0.451 (16.29)
Qwen3.6-35B-A3B	0.674 (5.74)	0.670 (18.11)
gemma-4-E4B-it	0.620 (5.26)	0.617 (14.70)
gemma-4-26B-A4B-it	0.705 (6.04)	0.630 (12.95)
Ministral-3-8B	0.539 (5.64)	0.621 (18.37)

Appendix

Table 13: Insight memory and Insight memory + H-EPM on ToolSandbox. LLM calls per scenario in parentheses; the higher score is in bold .

Model	H-EPM	Insight (succ.)	Insight
Qwen3.5-2B	0.436	0.466	0.488
Qwen3-8B	0.476	0.499	0.556
Qwen3.6-35B-A3B	0.670	0.618	0.674
gemma-4-E4B-it	0.554	0.557	0.620
gemma-4-26B-A4B-it	0.705	0.691	0.705
Ministral-3-8B	0.672	0.622	0.539

Appendix

Table 14: Insights from successful trajectories only (succ.) against H-EPM and Insight, which uses all trajectories, on ToolSandbox; H-EPM and Insight are from Table 1 . The highest score is in bold .

Model	ToolNet	H-EPM	ExpeL	Insight
Qwen3-8B	0.507	0.476	0.552	0.556
gemma-4-E4B-it	0.547	0.554	0.599	0.620

Appendix

Table 15: ExpeL against tool graph memory and Insight on ToolSandbox. The highest score is in bold .

LLM Tool Use

Large language model tool use studies how models select, call, and coordinate external tools to extend their reasoning and take actions.

See all 293 papers

MemToolAgent: Leveraging Memory for Tool Using Agents Based on Environment and User Feedback
Jun 6, 2026Suleyman Armagan Er, Danilo Ribeiro, Yogesh Virkar +5Tool-Using Agents LLM Tool Use
AWS AI · University of Washington
ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory
Jul 13, 2026Yue Fang, Zhibang Yang, Fangkai Yang +5LLM Tool Use
School of Computer Science, Peking University, Beijing, China · Microsoft
Learning from Failures: Heterogeneous Graph Memory for Small Language Model Tool-Using Agents
Sep 23, 2026Jiaxing Li, Lei Song, Rui Dong +1Graph-Based Agent Memory Long-Horizon LLM Agents
School of Computer Science and Engineering, Southeast University, Nanjing, China

Jun 6, 2026J/K move · Enter open · S save

MemToolAgent: Leveraging Memory for Tool Using Agents Based on Environment and User Feedback

Suleyman Armagan Er, Danilo Ribeiro, Yogesh Virkar, Surafel Lakew+4

AWS AI · University of Washington

Tool-Using Agents LLM Tool Use Agent Memory Management Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Modern large language model (LLM) agents can use external tools to help users solve complex tasks. However, for problems that require learning from long-term historical events or from previous agent-environment interactions, LLM agents are required to use memory mechanisms to store and retrieve experiences. While sophisticated memory systems exist for dialogue agents, few studies have empirically examined how to improve agents' tool-using capabilities through past user-agent conversations. We propose MemToolAgent, a framework that improves tool use through memory management. Our approach contains a memory extraction module that processes past experiences into structured memory entries, and a retrieval module that dynamically selects a subset of the stored memory entries. This enables more personalized and accurate responses aligned with user preferences and feedback without requiring LLM fine-tuning. In summary, this work has three main contributions: (1) a unified memory entry format that improves both general-purpose and personalized tool use without LLM fine-tuning, (2) a reflection-based memory extraction that uses environment and user feedback to distill wrong executions into critiques to store, and (3) a retrieval module that chooses how many past experiences to use based on the memory similarity distribution. MemToolAgent achieves 29%, 80%, and 17% relative improvements compared to strong baselines on the WorkBench, NESTFUL, and PEToolBench benchmarks, respectively.

Abstract

LLM Tool Use

Explore similar work

MemToolAgent: Leveraging Memory for Tool Using Agents Based on Environment and User Feedback

ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory

Learning from Failures: Heterogeneous Graph Memory for Small Language Model Tool-Using Agents