TRACER: Trajectory-Aware Tool Retrieval for LLM Agents

Tanish Kher; Sarvesh DK; Abedelkadir Asi; Sparsha Chennarayapatna Sundaresh; Dezhi Hong

cs.CLOct 15, 2026

TRACER: Trajectory-Aware Tool Retrieval for LLM Agents

Authors: Tanish Kher, Sarvesh DK, Abedelkadir Asi, Sparsha Chennarayapatna Sundaresh, Dezhi Hong

Organizations: Microsoft

Abstract

Modern agents select tools from registries, but exposing entire registries incurs high token costs and can impair selection as they grow. We present TRACER, a retrieval front-end that indexes observed tool-call sequences in a trie and ranks continuations using a learned blend of empirical and contextual priors. On a mixed retail-and-telecom $τ^2$ environment, TRACER improves GPT-5.4's tool-selection accuracy by 4.7 points while reducing input tokens per step by 80%, exposing just 5 of 128 tools. In autonomous full-turn execution, it matches GPT-5.4's full-registry task success with 61% fewer input tokens per step. Across six environments and four models, TRACER improves tool-selection accuracy over full-registry selection in 21 of 24 model-environment pairs, with average gains of 6 to 10 points; the three exceptions trail by at most 2.2. We analyze failures in both regimes to identify directions for future work.

Figures & tables

Panel of Figure 1 — Figure 1: Selection accuracy at a fixed budget of retrieving top- $k(=5)$ tools (mixed $\tau^{2}$ Large, $128$ tools, GPT-5.4).

Panel of Table 1 — Table 1: Positioning Tracer against prior tool-retrieval methods. Marks run absent, partial, fullest form.

Panel of Figure 2 — Figure 2: Tracer ’s execution trie (schematic example).

		Mixed Tau2 Small		Mixed Tau2 Medium		Mixed Tau2 Large $\ddagger$		Tau3 Banking $\ddagger$		API- Bank †	BFCL v3
Model	Alg.	$5$	$10$	$5$	$10$	$5$	$10$	$5$	$10$	$5$	$5$	$10$
GPT-5.4	LLM-only	63.8 (3.1)	64.1 (3.3)	60.2 (3.5)	60.1 (3.4)	57.2 (1.2)	57.3 (1.1)	55.0 (2.4)	54.8 (1.6)	82.4 (6.8)	61.6 (2.8)	61.1 (2.8)
	Tracer	69.4 (1.5)	69.3 (1.7)	66.1 (2.9)	63.7 (2.6)	61.9 (1.6)	59.0 (1.7)	45.7 (4.2)	46.8 (4.2)	88.3 (7.5)	68.4 (3.7)	68.1 (3.6)
	Tracer +G	62.4 (1.5)	62.9 (2.3)	61.9 (3.3)	60.4 (3.4)	59.9 (1.7)	58.0 (1.8)	51.6 (3.2)	52.6 (3.4)	90.7 (6.2)	65.4 (3.5)	65.6 (3.8)
	Tracer -fixed	62.2 (1.3)	66.3 (1.8)	60.3 (2.4)	61.9 (2.8)	50.5 (1.1)	51.0 (2.2)	43.6 (4.4)	45.9 (4.4)	91.4 (4.5)	63.1 (2.3)	64.2 (3.4)
	Vector	26.5 (2.3)	38.6 (2.2)	26.1 (1.1)	34.0 (1.7)	27.7 (1.5)	34.4 (1.5)	9.5 (3.0)	12.5 (2.7)	72.7 (5.1)	56.4 (2.2)	59.0 (2.9)

Table 2: Teacher-Forcing tool-selection accuracy (%) at

k{=}5

and

k{=}10

Panel of Figure 3 — Figure 3: The accuracy–cost trade-off (GPT-5.4; Tracer vs. LLM-only , Tracer +G on Tau3 Banking). One path per benchmark, one point per budget $k$ up to $25\%$ of the registry; arrows point toward larger $k$ .

Registry	Model	Cand. set	Margin (pts)	Fold SD	CIC red.
Small	GPT-5.4	$43\%$	$+2.3$	$4.5$	$38\%$
Small	GPT-5.4-mini	$22\%$	$+2.2$	$5.1$	$42\%$
Medium	GPT-5.4	$33\%$	$+1.2$	$2.3$	$50\%$
Medium	GPT-5.4-mini	$33\%$	$+4.4$	$3.3$	$50\%$
Large	GPT-5.4	$23\%$	$+0.1$	$2.3$	$61\%$
Large	GPT-5.4-mini	$16\%$	$+15.2$	$2.7$	$68\%$

Table 3: Free-running at the best budget per setting.

Specific API	$N$	LLM-only	Tracer
split_order_shipment	54	14%	51%
add_gift_message	79	32%	77%
request_return_pickup	46	16%	38%
schedule_installation	57	29%	58%
file_shipping_insurance_claim	79	49%	69%
Trap rate (lower is better)	315	36.4%	16.4%

Table 4: The specificity trap (Mixed Tau2 R&T Large; GPT-5.4,

k{=}5

; strict exact-match).

Panel of Figure 4 — Figure 4: The trie keeps the unlock before the write (Tau3 Banking, GPT-5.4, $k{=}10$ ).

Benchmark	APIs for 50% of calls	also in top-5 entropy	$\Delta$ accuracy (pts) Tracer +G $-$ Tracer
Mixed Tau2 R&T	5	0	$-1.0$
Tau3 Banking	2	2	$+5.8$
BFCL v3	17	3	$-2.5$

Table 5: The global tool set (GPT-5.4,

k{=}10

Model	$\Delta$ accuracy (pts) Tracer $-$ Tracer -fixed
	Mixed Tau2 R&T	Tau3 Banking	BFCL v3
GPT-5.4	+8.0	+0.9	+3.9
GPT-5.4-mini	+5.9	+0.9	+3.5
Grok-4-20-r	+6.0	+0.6	+7.1
Mistral-Large-3	+7.6	+1.1	+5.8

Table 6: Ablation (B): learned vs. constant blend (

k{=}10

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

	Mixed Tau2 R&T			Tau3 Banking	BFCL v3	API-Bank
	Small	Medium	Large
Domains	2	2	2	1	4	1
Tasks (trajectories)	311	725	1,554	69	200	50
Turns (decision points)	1,011	1,520	2,791	497	731	50
Steps (tool calls)	3,037	4,266	7,128	2,758	1,142	131
Turns per task (mean $\pm$ sd)	3.25 $\pm$ 2.30	2.10 $\pm$ 1.96	1.80 $\pm$ 1.53	7.20 $\pm$ 3.66	3.66 $\pm$ 1.26	1.00 $\pm$ 0.00

Appendix

Table 7: Benchmark statistics.

Source model ( $\tau^{2}$ leaderboard)	Turns	Steps	Reason. (%)
Qwen3.5-397B-A17B	143	420	12.9
Claude Sonnet 4.5	142	307	50.2
GLM-5	142	389	67.6
Claude Opus 4.5	142	376	63.8
GPT-5.2	39	65	0.0
Gemini 3 Pro	18	34	0.0

Appendix

Table 8: Real

\tau^{2}

retail turn provenance.

Attributed model	Turns	Steps	Reason. (%)
Gemini 3 Flash	166	485	0.0
GPT-5.2	76	290	0.0
Gemini 3 Pro	31	98	0.0
GLM-5	25	150	50.0
Qwen3.5-397B-A17B	15	81	33.3
Claude Opus 4.5	11	42	90.5

Appendix

Table 9: Telecom turn provenance. “Authored” are turns written in-house.

Gate	What it checks	Retail	Telecom
C1	ground-truth validity	1197/1299	647/718
C2	non-substitutability	1254/1299	327/381
C3	blind human-likeness	740/1299	495/718
C4	API disjointness	50/50	100/100
C5	non-cloning	1297/1299	717/718
C7	reverse non-substitutability	632/636	245/325

Appendix

Table 10: Curation battery.

Panel of Figure 5 — Figure 5: A synthetic trajectory humans read as real.

Aspect	Where	Original $\to$ ours
Registry	Mixed Tau2 R&T	domains separate $\to$ pooled, three sizes
Tools	Mixed Tau2 R&T	real only $\to$ plus validated synthetic
Backends	BFCL v3	four, separate $\to$ merged, alternates checked
Policy	all	absent or $\tau^{2}$ $\to$ $\tau^{2}$ -style, ours
Grading	all	per task $\to$ per turn

Appendix

Table 11: Our setting against the published benchmarks.

Panel of Figure 6 — Figure 6: Full Tracer architecture.

Weight head	Recall@5 (%)
Constant $w{=}0.5$ (baseline)	80.3
Hand-picked linear (5 features)	98.4
PCA-5 linear (of 14)	98.6
$\tanh$ MLP, hidden $16$	98.5

Appendix

Table 12: Weight-head capacity. Mixed Tau2 R&T, five-fold GroupKFold ,

1{,}576

steps.

Environment	Teacher-forcing	Free-running
Mixed Tau2 R&T (Small)	$0.014\pm 0.004$	$0.012\pm 0.002$
Mixed Tau2 R&T (Medium)	$0.014\pm 0.003$	$0.015\pm 0.000$
Mixed Tau2 R&T (Large)	$0.019\pm 0.002$	$0.022\pm 0.001$
Tau3 Banking	$0.167\pm 0.037$	n/a
API-Bank	$0.255\pm 0.086$	n/a
BFCL v3	$0.024\pm 0.015$	$0.024\pm 0.015$

Appendix

Table 13: Learned blend weight

w

by environment.

Reg.	Alg.	prior_margin	top_edge_support	R_top1	query_len	n_prior_turns	$b_{w}$
Small	Tracer	$0.544\pm 0.131$	$-0.225\pm 0.110$	$-0.298\pm 0.127$	$0.181\pm 0.044$	$-0.562\pm 0.166$	$-4.337\pm 0.336$
Medium	Tracer	$0.120\pm 0.165$	$-0.417\pm 0.010$	$-0.216\pm 0.099$	$0.124\pm 0.035$	$-0.675\pm 0.162$	$-4.284\pm 0.225$
Large	Tracer	$0.051\pm 0.070$	$-0.364\pm 0.113$	$-0.139\pm 0.036$	$0.118\pm 0.029$	$-0.511\pm 0.083$	$-3.940\pm 0.113$
Small	Tracer +G	$0.543\pm 0.132$	$-0.225\pm 0.110$	$-0.298\pm 0.128$	$0.181\pm 0.044$	$-0.562\pm 0.166$	$-4.337\pm 0.338$
Medium	Tracer +G	$0.120\pm 0.166$	$-0.418\pm 0.011$	$-0.216\pm 0.099$	$0.124\pm 0.035$	$-0.675\pm 0.162$	$-4.284\pm 0.225$
Large	Tracer +G	$0.054\pm 0.073$	$-0.358\pm 0.122$	$-0.138\pm 0.037$	$0.118\pm 0.028$	$-0.512\pm 0.085$	$-3.938\pm 0.116$

Appendix

Table 14: Learned

w

coefficients, Mixed Tau2 R&T.

	Selection accuracy			Out of trie
$k$	Last-2	Full	$\Delta$	Last-2	Full	$\Delta$ acc
$5$	61.6 (1.9)	59.5 (3.8)	$+2.1$	$9.6\%$	$12.4\%$	$+1.7$
$10$	61.2 (1.8)	58.3 (3.8)	$+2.9$	$9.6\%$	$12.4\%$	$-0.2$
$20$	60.4 (2.2)	58.5 (3.7)	$+1.9$	$9.6\%$	$12.4\%$	$+0.9$
$30$	60.2 (2.0)	58.6 (4.3)	$+1.5$	$9.6\%$	$12.4\%$	$-0.2$

Appendix

Table 15: Full prefix against a two-step trie key.

Panel of Figure 7 — Figure 7: The trie blocks an over-reach (Mixed Tau2 R&T Large, GPT-5.4, $k{=}5$ ).

		Teacher-forcing						Free-running
		Mixed Tau2 R&T			Tau3 Banking	API- Bank	BFCL v3	Mixed Tau2 R&T
Model	Alg.	Small	Medium	Large				Small	Medium	Large
GPT-5.4	Tracer	10.8% ( $-59.0$ )	11.8% ( $-52.5$ )	12.7% ( $-42.6$ )	43.4% ( $-35.4$ )	2.3% ( $-88.3$ )	13.0% ( $-32.5$ )	33.0% (N/A)	40.4% (N/A)	36.7% (N/A)
	Tracer +G	10.8% ( $-47.1$ )	11.8% ( $-44.2$ )	12.7% ( $-40.5$ )	43.4% ( $-9.4$ )	2.3% ( $-57.4$ )	13.0% ( $-30.7$ )	40.8% (N/A)	40.9% (N/A)	37.3% (N/A)
	Tracer -fixed	10.8% ( $-55.9$ )	11.8% ( $-50.0$ )	12.7% ( $-34.4$ )	43.4% ( $-34.5$ )	2.3% ( $-91.4$ )	13.0% ( $-22.8$ )	38.5% (N/A)	40.2% (N/A)	34.6% (N/A)
GPT-5.4-mini	Tracer	10.8% ( $-54.6$ )	11.8% ( $-46.7$ )	12.7% ( $-36.0$ )	43.4% ( $-34.5$ )	2.3% ( $-65.1$ )	13.0% ( $-36.2$ )	35.6% (N/A)	43.6% (N/A)	39.2% (N/A)

Appendix

Table 16: Out-of-trie steps: how often, and how much worse.

Gold tool type	Steps	Tracer +G	LLM-only	$\Delta$
gated unlock $\ast$	228	0.246	0.148	$+0.098$
shell / grep	1,143	0.553	0.484	$+0.069$
KB retrieval	594	0.575	0.521	$+0.054$
write	230	0.730	0.867	$-0.136$
verification	73	0.721	0.941	$-0.219$
utility routing	197	0.423	0.689	$-0.266$

Appendix

Table 17: Banking accuracy by gold tool type (GPT-5.4,

k{=}10

;

\Delta{=}\mbox{{Tracer}+G}{}-\mbox{{LLM-only}}{}

		Small Mixed Tau2 R & T				Medium Mixed Tau2 R & T				Large Mixed Tau2 R & T $\ddagger$
Model	Alg.	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$
GPT-5.4	LLM-only	63.8 (3.1)	64.1(3.3)	64.0(3.5)	64.1(2.9)	60.2(3.5)	60.1(3.4)	60.1(3.5)	60.0(3.5)	57.2(1.2)	57.3(1.1)	57.3(1.2)	57.5(1.4)
	Tracer	69.4 (1.5)	69.3 (1.7)	69.7 (2.5)	69.4 (2.3)	66.1 (2.9)	63.7 (2.6)	63.5 (3.3)	63.1 (3.1)	61.9 (1.6)	59.0 (1.7)	58.7 (1.8)	59.0 (1.9)
	Tracer +G	62.4(1.5)	62.9(2.3)	63.0(2.7)	62.7(2.5)	61.9 (3.3)	60.4(3.4)	60.1 (3.4)	59.7(3.7)	59.9 (1.7)	58.0 (1.8)	57.9 (1.5)	58.0 (2.1)
	Tracer -fixed	62.2(1.3)	66.3 (1.8)	69.5 (1.5)	69.1 (2.5)	60.3(2.4)	61.9 (2.8)	59.5(3.7)	60.6 (3.2)	50.5(1.1)	51.0(2.2)	53.6(2.2)	55.5(2.1)
	Vector	26.5(2.3)	38.6(2.2)	53.6(3.4)	59.6(3.4)	26.1(1.1)	34.0(1.7)	45.9(3.3)	52.0(2.5)	27.7(1.5)	34.4(1.5)	43.1(1.5)	50.0(2.1)

Appendix

Table 18: Teacher-Forcing tool-selection accuracy (%): full budget sweep, Mixed Tau2 R & T. Parentheses give the across-fold SD. Best per column within each model in bold , second best underlined .

\ddagger

: averaged over 3 iterations.

		Tau3 Banking $\ddagger$			API- Bank		BFCL v3
Model	Alg.	$3$	$5$	$10$	$3$	$5$	$5$	$10$	$20$	$30$
GPT-5.4	LLM-only	54.7 (1.3)	55.0 (2.4)	54.8 (1.6)	81.6(9.1)	82.4(6.8)	61.6(2.8)	61.1(2.8)	61.4(2.9)	61.9(3.2)
	Tracer	44.3(3.5)	45.7(4.2)	46.8(4.2)	90.6(5.7)	88.3(7.5)	68.4 (3.7)	68.1 (3.6)	68.5 (2.9)	69.6 (3.0)
	Tracer +G	50.4 (3.0)	51.6 (3.2)	52.6 (3.4)	90.7 (6.2)	90.7 (6.2)	65.4 (3.5)	65.6 (3.8)	66.4 (3.3)	67.1 (3.2)
	Tracer -fixed	41.9(4.2)	43.6(4.4)	45.9(4.4)	93.0 (3.7)	91.4 (4.5)	63.1(2.3)	64.2(3.4)	65.8(2.6)	66.2(2.5)
	Vector	7.4(2.1)	9.5(3.0)	12.5(2.7)	68.5(8.4)	72.7(5.1)	56.4(2.2)	59.0(2.9)	60.2(2.9)	62.3(3.5)

Appendix

Table 19: Teacher-Forcing tool-selection accuracy (%): full budget sweep, Tau3 Banking, API-Bank and BFCL v3. Parentheses give the across-fold SD. Best per column within each model in bold , second best underlined .

\ddagger

: averaged over 3 iterations. Tau3 Banking runs to

k{=}10

, API-Bank to

k{=}5

	Small Mixed Tau2 R & T				Medium Mixed Tau2 R & T				Large Mixed Tau2 R & T $\ddagger$
Alg.	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$
LLM-only	0% (7.54)	0% (7.54)	0% (7.54)	0% (7.54)	0% (10.97)	0% (10.97)	0% (10.97)	0% (10.97)	0% (14.17)	0% (14.17)	0% (14.17)	0% (14.17)
Tracer	55% (3.36)	52% (3.64)	46% (4.06)	41% (4.46)	72% (3.09)	69% (3.43)	64% (3.98)	60% (4.41)	80% (2.81)	77% (3.22)	73% (3.89)	68% (4.53)
Tracer +G	52% (3.58)	49% (3.85)	44% (4.26)	38% (4.64)	70% (3.34)	67% (3.66)	62% (4.20)	58% (4.61)	78% (3.07)	76% (3.46)	71% (4.12)	66% (4.75)
Tracer -fixed	55% (3.39)	51% (3.67)	46% (4.07)	41% (4.46)	72% (3.12)	68% (3.50)	63% (4.05)	60% (4.43)	80% (2.79)	77% (3.32)	72% (3.99)	67% (4.68)
Vector	53% (3.54)	46% (4.09)	32% (5.10)	19% (6.11)	71% (3.21)	66% (3.74)	57% (4.73)	48% (5.68)	79% (2.91)	76% (3.44)	68% (4.46)	61% (5.46)

Appendix

Table 20: Context-injection cost (CIC), Teacher-Forcing: full budget sweep, Mixed Tau2 R & T. Reduction against LLM-only in the same column; in parentheses, mean prompt tokens per decision step in thousands. CIC is independent of the serving model, so one row per algorithm covers all four.

	Tau3 Banking $\ddagger$			API- Bank		BFCL v3
Alg.	$3$	$5$	$10$	$3$	$5$	$5$	$10$	$20$	$30$
LLM-only	0% (40.57)	0% (40.57)	0% (40.57)	0% (9.37)	0% (9.37)	0% (12.46)	0% (12.46)	0% (12.46)	0% (12.46)
Tracer	31% (27.92)	31% (28.10)	30% (28.50)	88% (1.14)	87% (1.23)	90% (1.25)	87% (1.63)	81% (2.32)	76% (3.02)
Tracer +G	30% (28.33)	30% (28.48)	29% (28.86)	86% (1.34)	85% (1.42)	87% (1.59)	84% (1.95)	79% (2.62)	73% (3.31)
Tracer -fixed	31% (27.92)	31% (28.10)	30% (28.47)	88% (1.15)	87% (1.25)	90% (1.28)	87% (1.66)	81% (2.37)	75% (3.06)
Vector	31% (28.19)	30% (28.50)	28% (29.25)	85% (1.37)	82% (1.65)	89% (1.37)	85% (1.89)	77% (2.86)	70% (3.79)

Appendix

Table 21: Context-injection cost (CIC), Teacher-Forcing: full budget sweep, Tau3 Banking, API-Bank and BFCL v3. Reduction against LLM-only in the same column; in parentheses, mean prompt tokens per decision step in thousands.

		Mixed Tau2 Small				Mixed Tau2 Medium				Mixed Tau2 Large
Model	Alg.	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$
GPT-5.4	LLM-only	47.6 (5.1)	47.2 (4.5)	46.0(5.1)	48.3 (4.0)	47.4 (3.2)	47.5 (4.5)	48.6 (4.5)	48.3(5.2)	58.2 (2.0)	57.9 (2.3)	57.9 (1.5)	58.2(2.2)
	Tracer	43.0(3.5)	45.0(5.5)	48.3 (4.5)	48.5 (5.5)	43.0(3.8)	46.2(3.3)	48.2 (3.8)	49.5 (2.3)	55.3(2.2)	56.2(1.8)	57.6(2.8)	58.3(2.3)
	Tracer +G	44.7 (4.4)	46.1 (4.8)	46.7 (4.5)	46.7(5.7)	44.2(3.4)	46.7 (3.2)	47.7(3.7)	49.0(2.8)	55.6(1.4)	56.6(2.0)	57.3(2.7)	58.5 (2.3)
	Tracer -fixed	39.8(5.1)	43.0(5.6)	45.9(3.4)	46.5(3.8)	45.2 (4.0)	44.0(3.4)	47.7(4.1)	49.7 (3.9)	56.2 (1.9)	58.1 (2.7)	58.5 (2.8)	58.6 (2.8)
	Vector	28.1(3.1)	35.1(5.4)	37.1(4.1)	41.9(5.6)	25.3(2.5)	32.1(3.7)	37.6(6.3)	40.7(6.0)	41.5(0.6)	45.6(0.8)	50.6(1.6)	52.6(1.1)

Appendix

Table 22: Free-running task success (%).

		Small Mixed Tau2 R & T				Medium Mixed Tau2 R & T				Large Mixed Tau2 R & T
Model	Alg.	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$	$5$	$10$	$20$	$30$
GPT-5.4	LLM-only	0% (6.91)	0% (6.48)	0% (6.88)	0% (6.62)	0% (11.45)	0% (11.43)	0% (11.42)	0% (11.42)	0% (11.99)	0% (11.97)	0% (11.98)	0% (11.98)
	Tracer	52% (3.31)	40% (3.91)	38% (4.28)	19% (5.39)	68% (3.61)	64% (4.07)	57% (4.88)	50% (5.69)	77% (2.75)	74% (3.12)	68% (3.87)	61% (4.63)
	Tracer +G	52% (3.30)	39% (3.94)	36% (4.37)	19% (5.36)	65% (3.96)	62% (4.32)	55% (5.10)	49% (5.87)	75% (2.98)	72% (3.35)	66% (4.09)	60% (4.81)
	Tracer -fixed	50% (3.48)	36% (4.15)	28% (4.93)	20% (5.29)	68% (3.61)	65% (4.04)	57% (4.86)	50% (5.67)	77% (2.70)	74% (3.07)	68% (3.81)	62% (4.56)
	Vector	57% (3.00)	48% (3.40)	40% (4.15)	24% (5.00)	69% (3.57)	64% (4.08)	55% (5.17)	46% (6.12)	77% (2.71)	74% (3.15)	66% (4.03)	59% (4.88)

Appendix

Table 23: Context-injection cost (CIC), Free-Running: full grid. Reduction against LLM-only in the same model-and-registry row block; in parentheses, mean prompt tokens per decision step in thousands. Unlike teacher forcing, free-running CIC differs by serving model.

Tool Selection for Language Model Agents

See all 27 papers

Aug 29, 2026

TRACER: Per-Tool Context Retention for LLM Agents via Consequence-Attributed Reinforcement Learning

Enterprise data agents answer business queries by chaining many tool calls over multiple reasoning steps, routinely accumulating hundreds of thousands of context tokens per session. Existing compression strategies typically allocate retention budgets without accounting for the downstream consequences of removing individual tool outputs. Aggressive compression may therefore trigger costly tool re-invocations that offset the initial savings. We call this the compression--consequence gap. To close it, we propose TRACER, which formulates compression as a sequential per-tool decision problem. A lightweight REINFORCE policy assigns query-conditioned retention ratios using only information available at each compression event. Its consequence-aware objective jointly accounts for task success, total token consumption, and post-compression tool re-invocations. To improve credit assignment, TRACER uses a learned outcome model to compare the predicted consequences of the selected retention ratio with those of fully retaining each tool output. On held-out production queries across three compressor backends, TRACER reduces total token consumption by 29--46% relative to keeping all context while maintaining comparable or higher task success. Compared with a tool-type-conditional static policy, TRACER provides an additional 15--18% of token savings. Interventional rollouts show that the learned per-tool credit scores correlate with measured single-tool consequences. The learned policy also yields positive savings when transferred across agent backbones and compressor architectures, and reduces token consumption by 18--25% on five held-out LOCA-bench environments. These results demonstrate the value of consequence-aware, per-tool context retention for improving the efficiency of long-horizon language agents.

Ziqi Lin, Ye Wu, Mengying Yang +4

LLM Context Management Context Compression Long-Horizon LLM Agents RL for LLM Agents

Ask PDF

Oct 16, 2026

Tool Recommendation for Multi-Step Agents: Finding Every Tool a Task Needs

A multi-step agent task fails when a single tool it needs is missing, yet the tools an agent may need are too many to list: the roughly 8,200 tools on the platform we study have definitions that together exceed the context window of any model we evaluate. We present Tool Recommendation, a deployed system that gives an agent all of them, hundreds of times more than a typical MCP server exposes, through a fixed interface of three tools. It runs at the gateway between agent and tools, executing every call under the end user's own credentials and checking it against the user's consent for each service and the tools the agent may use. We evaluate it on both a public retrieval benchmark and billed end-to-end session cost. On ToolRet, Tool Recommendation places every tool a query needs in the top ten 57% more often than BM25, a lexical ranking that commercial tool search offers, and compares favorably with published systems up to nine times its size. In paired multi-step workflows, its first recommendation covered every required tool in 93% of Codex sessions, and against a fixed list of 100 tools it cut input tokens by 17% with Codex and 40% with Claude Code and the total bill in all 14 runs, by up to 40%, with no change in task completion.

Sam Partee, River Stevens, Thierry Damiba +1

Tool Selection for Language Model Agents Tool Discovery for LLM Agents LLM Agent Efficiency

Ask PDF

Jun 11, 2026

HyperTool: Beyond Step-Wise Tool Calls for Tool-Augmented Agents

Tool-augmented LLM agents commonly rely on step-wise atomic tool calls, where each invocation, observation, and value transfer is exposed in the main reasoning trace. This creates an execution-granularity mismatch: locally deterministic tool workflows are unfolded into repeated model-visible decisions, consuming context and forcing the model to manage low-level dataflow in the trace. We introduce HyperTool, a unified executable MCP-style tool interface that changes the model-visible unit of tool execution. A model invokes HyperTool with a code block that can call existing tools through their original schemas, manipulate returned values, and pass intermediate results locally, folding deterministic tool subroutines into a single outer call. To train models to use this interface, we synthesize HyperTool-format trajectories from cross-tool compositional tasks and verify them in real MCP environments. On MCP-Universe, HyperTool improves average accuracy from 15.69% to 35.29% on Qwen3-32B and from 9.93% to 33.33% on Qwen3-8B, and surpass GPT-OSS and Kimi-k2.5 on average accuracy, showing that our HyperTool can substantially improve multi-step tool use.

Yaxin Du, Yifan Zhou, Yujie Ge +7

Shanghai Jiao Tong University · IQuest Research · Beijing University of Aeronautics and Astronautics

Tool-Augmented Language Model Agents LLM Agent Workflow Optimization LLM Tool Use LLM Agent Training Tool Use

Ask PDF

Abstract

Tool Selection for Language Model Agents

Explore similar work

TRACER: Per-Tool Context Retention for LLM Agents via Consequence-Attributed Reinforcement Learning

Tool Recommendation for Multi-Step Agents: Finding Every Tool a Task Needs

HyperTool: Beyond Step-Wise Tool Calls for Tool-Augmented Agents