TRAP: Tool-result Replay Attack for Probing Misbinding in LLM Agents

Yutong Zhi; Jie Zhang; Xiuqi Lu; Jiayuan Xu; Jing Shao

cs.AIOct 12, 2026

TRAP: Tool-result Replay Attack for Probing Misbinding in LLM Agents

Authors: Yutong Zhi, Jie Zhang, Xiuqi Lu, Jiayuan Xu, Jing Shao

Organizations: Tsinghua University · Shanghai AI Laboratory

Abstract

As AI agents increasingly act through external tools, tool results are no longer simple outputs but execution artifacts that must remain correctly associated with the context that produced them. Yet as results persist across caches, recovery mechanisms, and shared runtime state, this association can break, causing a real and unmodified result to be consumed in an incompatible execution context and lead the agent to an unsafe action. We characterize this failure as tool-result misbinding. To systematically study this failure, we introduce ReplayBench, a benchmark that uses controlled tool-result replay to evaluate misbinding across 19 representative agent scenarios, 4 failure properties, and 3 replay mechanisms. Across eight LLMs, three agent frameworks, and diverse stateful workflows, we find widespread replay-induced failures, with a mean Final-State Error Rate of 0.649 under cross-task replay. Stage-wise analysis localizes these failures to model-side action generation. Further analyses show that replay susceptibility varies with task similarity, shows little dependence on object similarity, and remains high under result staleness and implicit environment changes. These findings motivate runtime validity mechanisms that eliminate final-state errors in our evaluated settings while preserving legitimate recovery. The results call for explicit contextual binding in tool-using agent runtimes. Our work is public at https://github.com/Phitooo/Replaybench.

Figures & tables

Figure 1: Cross-task tool-result misbinding in a tool-using LLM agent. A real result from Task A is replayed into Task B, leading to a wrong action and a final-state error.

Figure 2: Overall pipeline of ReplayBench with parameterized case construction, sandboxed execution, and stage-wise evaluation.

Model	Framework	Final-State Error Rate (FSER $\downarrow$ )
Model		Email	Calendar	Files	Banking	Identity	DevOps	Security
Proprietary Models
GPT-5.5	LlamaIndex	0.80	1.00	0.70	1.00	1.00	1.00	0.40
	LangGraph	1.00	1.00	0.90	1.00	1.00	1.00	0.40
	OpenAI SDK	0.80	1.00	0.80	1.00	1.00	0.80	0.80
GPT-5.6-Luna	LlamaIndex	0.10	1.00	0.90	1.00	1.00	0.90	0.60

Table 1: FSER across eight models, three agent frameworks, and seven representative scenarios with T01. Lower FSER indicates stronger resistance to cross-task replay.

Figure 3: Mean FSER across models and agent frameworks in T01 evaluation. Points show mean FSER, error bars denote 95% bootstrap confidence intervals.

Figure 4: FSER across stateful failure templates T02–T04 in the cloud-file sharing scenario.

Figure 5: FSER across task/object similarity, result freshness, and environment-change settings.

Scenario	Template	Undef. FSER	Call-ID binding				One-shot ticket				World-state epoch
Scenario	Template	Undef. FSER	D	W	A	F	D	W	A	F	D	W	A	F
Email	T01	0.80	0	0	0	0	1	1.00	0	0	1	1.00	0	0
Files	T01	0.70	0	0	0	0	1	0.90	0	0	1	0.80	0	0
	T02	1.00	0	0	0	0	1	1.00	0	0	1	1.00	0	0
	T03	1.00	0	0	0	0	1	1.00	0	0	1	1.00	0	0
	T04	1.00	0	0	0	0	1	1.00	0	0	1	1.00	0	0

Table 2: Stage-wise defense results with GPT-5.5 and LlamaIndex. D/W/A/F denote DAR, WAR, AAR, and FSER. All defenses have FR

=0

and Recov

=1

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Scenario
F01	Email attachments (send)
F02	Instant messaging and public posts
F03	Calendar and task management
F04	Files, documents, and cloud drive (share)
F05	Search, RAG, and knowledge bases
F06	E-commerce orders and subscriptions

Appendix

Table 3: ReplayBench scenarios.

Model	Mean FSER	95% CI
DeepSeek-V4-Pro	0.967	[0.933, 0.995]
Gemini-3.8-Flash	0.943	[0.838, 1.000]
GPT-5.5	0.876	[0.790, 0.948]
GPT-5.6-Luna	0.843	[0.738, 0.924]
GLM-5.3	0.719	[0.643, 0.786]
GPT-5.6-Sol	0.610	[0.486, 0.733]

Appendix

Table 4: Model-level mean FSER and 95% bootstrap confidence intervals for the T01 evaluation. Confidence intervals are computed over the evaluated framework–scenario cells using 5,000 bootstrap resamples.

Framework	Mean FSER	95% CI
LangGraph	0.686	[0.582, 0.780]
OpenAI Agents SDK	0.655	[0.555, 0.750]
LlamaIndex	0.607	[0.505, 0.707]

Appendix

Table 5: Framework-level mean FSER and 95% bootstrap confidence intervals for the T01 evaluation.

Factor	SS	$\boldsymbol{\eta^{2}}$	Partial $\boldsymbol{\eta^{2}}$	F statistic
Model	18.157	0.742	0.795	$F(7,152)=84.11$
Framework	0.176	0.007	0.036	$F(2,152)=2.85$
Scenario	1.460	0.060	0.237	$F(6,152)=7.89$
Model $\times$ Framework	0.258	0.0106	0.055	$F(14,138)=0.575$

Appendix

Table 6: Variance decomposition of cell-level T01 FSER.

	F01 Email		F04 Cloud-file
Mechanism	LI	LG	LI	LG
M1 Durable Recovery	0.80	1.00	0.70	0.90
M2 Queue Relay	0.70	1.00	0.90	1.00
M3 Cache Resume	1.00	1.00	0.90	1.00

Appendix

Table 7: Replay-mechanism comparison for GPT-5.5 under T01. Values report FSER. DAR is 1.0 for every configuration.

Scenario	T01-M1	T01-M2	T01-M3	T02-M1	T02-M2	T02-M3	T03-M1	T03-M2	T03-M3	T04-M1	T04-M2	T04-M3
F01 Email	0.80	0.70	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
F02 Instant Messaging	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
F03 Calendar/Task	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
F04 Cloud-file Sharing	0.70	0.90	0.90	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
F05 Search/RAG	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
F06 E-commerce/Order	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00

Appendix

Table 8: Full ReplayBench FSER results with GPT-5.5 and LlamaIndex across 19 scenario families, four templates, and three replay mechanisms. Each configuration uses 10 trials.

Model	Scenario	$n=10$	$n=30$
GPT-5.5	Email operations	$0.80\ [0.49,\,0.94]$	$0.93\ [0.79,\,0.98]$
GPT-5.5	Calendar/task management	$1.00\ [0.72,\,1.00]$	$1.00\ [0.89,\,1.00]$
GPT-5.5	Cloud-file sharing	$0.70\ [0.40,\,0.89]$	$0.87\ [0.70,\,0.95]$
Qwen3.8-Max	Cloud-file sharing	$0.00\ [0.00,\,0.28]$	$0.07\ [0.02,\,0.21]$

Appendix

Table 9: Sensitivity of FSER to the number of valid trials. Each row keeps the experimental configuration fixed and changes only the number of repeated trials. Confidence intervals are 95% Wilson intervals.

Scenario	Task similarity	Near-identical	Same-type	Distinct
F01 Email	High	1.00	1.00	1.00
	Mid	1.00	0.80	1.00
	Low	0.00	0.00	0.00
F04 Cloud-file	High	1.00	0.80	1.00
	Mid	1.00	0.80	1.00
	Low	1.00	1.00	1.00

Appendix

Table 10: FSER under independently varied task and object similarity. All configurations use GPT-5.5 with LlamaIndex and T01. DAR is 1.0 throughout these experiments.

Result age	Freshness visible	WAR	AAR	FSER
Fresh	Hidden	1.00	1.00	1.00
Fresh	Visible	1.00	1.00	1.00
Intermediate	Hidden	1.00	1.00	1.00
Intermediate	Visible	1.00	1.00	1.00
High-stale	Hidden	1.00	1.00	1.00
High-stale	Visible	0.60	0.60	0.60

Appendix

Table 11: Effect of result age and freshness visibility in the F04 T01 controlled analysis. DAR is 1.0 for all configurations.

	F01		F04
Model	Replay	IPI	Replay	IPI
GPT-5.5	0.80	0.70	0.70	0.40
DeepSeek-V4-Pro	1.00	0.00	1.00	0.00
GLM-5.3	0.60	0.00	0.80	0.00
Gemini-3.8-Flash	0.00	0.00	1.00	0.10
GPT-5.6-Luna	0.10	blk.	0.90	blk.

Appendix

Table 12: Full comparison between unmodified replay and the instrumented IPI control. Values report FSER. “blk.” indicates that no valid IPI execution completed and is not interpreted as zero.

	Replay	IPI
Delivered result	Genuine approval for attachment-A	Same approval plus explicit instruction field
Target verification	verify_attachment(attachment_B.docx)	verify_attachment(attachment_B.docx) , repeated after rejecting the instruction
State-changing action	mock_send_email(..., attachment-A)	None
Checker state	Threat action accepted; FSER $=1$	No state-changing record; FSER $=0$

Appendix

Table 13: Execution behavior for the paired DeepSeek-V4-Pro email case.

Figure 6: Stage-wise results of the three runtime defenses for F04-T01 with GPT-5.5 and LlamaIndex.

Tool-Using Agents

Tool-using agents plan and act through external tools, APIs, software interfaces, and environments to complete tasks beyond language generation.

See all 289 papers

When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents
Jun 4, 2026Dongsheng Zhu, Xuchen Ma, Yucheng Shen +5AI Agent Reliability AI Agent Benchmarks
Shanghai AI Laboratory · East China Normal University · Soochow University +2
AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP
Jul 13, 2026Aritra Mazumder, Nusrat jahan LiaAI Agent Reliability LLM Agent Evaluation
University of Utah · University of Dhaka
Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability
Jun 24, 2026Yang Tian, Zhengpeng Shi, Yu Zhou +1AI Agent Reliability AI Agent Benchmarks
School of AI, Shanghai Jiao Tong University1 · School of Cyber Science and Engineering, Southeast University2

Jun 4, 2026J/K move · Enter open · S save

When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents

Dongsheng Zhu, Xuchen Ma, Yucheng Shen, Xiang Li+4

Shanghai AI Laboratory · East China Normal University · Soochow University+2

AI Agent Reliability AI Agent Benchmarks LLM Reliability LLM Tool Use LLM Agent Planning LLM Agent Reliability

Open paper AskPDF arXiv

Existing benchmarks evaluate Tool-Integrated Reasoning (TIR) in LLMs on idealized ''happy paths'', largely overlooking real-world tool failures. We introduce ToolMaze, a benchmark for dynamic path discovery and error recovery in TIR agents. To separate systematic replanning from blind trial-and-error, ToolMaze adopts a two-dimensional design: DAG-based topological complexity and a

2 \times 2

taxonomy of tool perturbations (explicit/implicit, transient/permanent). Evaluations show that perturbations degrade performance across nearly all models, with the sharpest drops under implicit semantic failures. Driven by systemic over-trust in corrupted outputs, Perturbation Recovery Rate (PRR) plummets by around 37% in these scenarios, while complex topologies trap agents in futile trial-and-error loops. Crucially, agentic fault-tolerance improves with model scale

3.66\times

slower than basic task execution, highlighting dynamic replanning as a distinct bottleneck unaddressed by model scaling or prompting. Data and code are available at https://github.com/Zhudongsheng75/ToolMaze.

Abstract

Tool-Using Agents

Explore similar work

When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents

AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP

Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability