AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations

Yuanjian Xu; Jinyi Han; Jianing Hao; Yuxuan Sun; Fanyang Lu; Richeng Xuan; Guang Zhang; Zhichao Hu

cs.AIOct 15, 2026

AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations

Authors: Yuanjian Xu, Jinyi Han, Jianing Hao, Yuxuan Sun, Fanyang Lu, Richeng Xuan, Guang Zhang, Zhichao Hu

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · East China Normal University · University College London · Tencent Hunyuan

Abstract

Real-world tasks are often complex and noisy, posing substantial challenges for LLM agents. Yet existing evaluations lack a systematic framework for jointly characterizing potential perturbation targets and types. We use this two-dimensional view to conduct a unified study that quantifies the robustness of current frontier models, traces the mechanisms behind perturbation-induced failures, and draws implications for robustness-oriented training. Through controlled evaluations on APEX-Agents and Toolathlon, we find that perturbations consistently degrade performance, while the dominant vulnerabilities vary across task environments. Trajectory-level analysis reveals that agents rarely recognize altered inputs and often propagate corrupted information without verification. A focused cost analysis further shows that successful perturbed trajectories can require substantially more computation. These findings motivate training objectives that cover diverse forms of noise and reward verification, uncertainty awareness, and recovery. We provide a rigorous benchmark-construction pipeline. Code and data will be released at https://github.com/xuyj233/AgentPerturb.

Figures & tables

Figure 1: Perturbation design. The left panel shows the four perturbation targets in the agentic interaction loop, and the right panel illustrates the three perturbation types with representative examples. Each type can be applied to any target, and each instance perturbs a single target.

Figure 2: Perturbation-construction pipeline. An LLM selects a compatible target and type and instantiates a concrete edit from demonstrations. The evaluation protocol is adapted when recovery requires clarification, and an independent verifier retains only single-target, goal-preserving, solvable, and executable instances.

Model	APEX-Agents		Toolathlon
Model	Clean	Perturbed	Clean	Perturbed
GPT-5.6-Sol	85.5	80.6 $\bm{\downarrow}$ 5.7%	34.0	29.7 $\bm{\downarrow}$ 12.6%
Claude Opus 5	81.0	68.2 $\bm{\downarrow}$ 15.8%	32.0	28.3 $\bm{\downarrow}$ 11.6%
DeepSeek-V4-Pro	84.0	80.3 $\bm{\downarrow}$ 4.4%	32.0	24.1 $\bm{\downarrow}$ 24.7%
DeepSeek-V4-Flash	88.0	47.0 $\bm{\downarrow}$ 46.6%	28.0	21.4 $\bm{\downarrow}$ 23.6%
Kimi-K3	40.0	39.0 $\bm{\downarrow}$ 2.5%	32.0	23.0 $\bm{\downarrow}$ 28.1%

Table 1: Overall performance on APEX-Agents and Toolathlon. Perturbed cells report the score followed by the relative degradation

100(C-P)/C

, marked with a downward arrow. Averages are taken over all ten models, and evaluation parameters are given in Appendix A .

Figure 3: Marginal effects of perturbation targets and types. Bars report the clean-minus-perturbed score difference in percentage points, averaged equally over the five models with per-family scores on both benchmarks. Solid bars denote APEX-Agents and hatched bars denote Toolathlon. Both panels share one scale, and larger values indicate greater degradation.

Figure 4: Effects of individual target–type combinations. Each cell reports the clean-minus-perturbed score difference in percentage points, averaged equally over the same five models as Figure 3 . Both panels share one scale, and the outlined cell marks the largest decrease in each benchmark.

Figure 5: Capability versus robustness. Each point plots a model’s family-level Artificial Analysis Intelligence Index v4.1.1 score against its relative degradation

100(C-P)/C

, where lower values indicate greater robustness. Colors denote providers, and triangles mark the second model from the same provider. Dashed lines are least-squares fits with 95% confidence bands, so a downward slope means that more capable models degrade less.

Figure 6: Recovery cost and token efficiency of Hunyuan-3-High. Panel a reports tokens per successful trajectory and panel b successes per million tokens, for each perturbation family. Dashed lines mark clean baselines, shaded regions mark outcomes worse than clean, colors encode targets, and hatching encodes perturbation types.

Figure 7: Failure awareness and first-error behavior. The 30 episodes are DeepSeek-V4-Flash failures whose clean counterparts succeed. Panel a shows the highest awareness grade reached in each trace, and panel b the behavior at the first observable error. Colors encode awareness.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	APEX-Agents	Toolathlon
Shared across benchmarks
Agent scaffold	Codex CLI, native function calling
System prompt	Appendix F
Retry policy	3 attempts with exponential backoff from 30 s, on rate-limit and timeout errors
Success oracle	Native grader
Clean episodes	25

Appendix

Table 2: Evaluation protocol. The upper block lists the controls shared by every run, and the lower block lists the two limits that differ between benchmarks.

Model	AA Index	Proxy configuration
GPT-5.6-Sol	61	Family max.; max effort
Claude Opus 5	63	Family max.; max effort
DeepSeek-V4-Pro	53	Reasoning; max effort
DeepSeek-V4-Flash	52	Reasoning; max effort
Kimi-K3	60	Family max.; max effort
Kimi-K2.6	45	Reasoning

Appendix

Table 3: External capability scores. Scores are Artificial Analysis Intelligence Index v4.1.1 family-level proxies used in Figure 5 . The last column gives the configuration from which each proxy is taken.

Axis	Code	Label	Operational definition
Awareness	A1	No explicit signal	The trace contains no explicit indication that a value, conclusion, or answer may be problematic.
	A2	Tentative signal	The trace contains a tentative or one-off indication that a value, conclusion, or answer may be problematic.
	A3	Sustained problem statement	The agent explicitly and persistently identifies a specific value, conclusion, or answer as problematic.
	A4	Input-channel attribution	The agent attributes the problem to the integrity of an instruction, context source, or tool channel.
Behavior	E1	Invented content	The agent supplies an identifier, value, or other content unsupported by the available evidence.
	E2	Requirement omitted	Required content or an output constraint is absent from the deliverable.

Appendix

Table 4: Failure-attribution labels. Awareness grades (A1–A4) capture only signals verbalized in the trace, and behavior labels (E1–E6) are assigned at the first observable error.

Level	Patch operation	$\bm{n}$	Drop	$\bm{\Delta}$ cost
L1	replace_description	19	21.1	+97,616
L2	replace_enum	6	16.7	$-2,452$
L3	add_fake_tool	23	4.3	$-19,691$
L4	remove_tool	19	5.3	+11,652

Appendix

Table 5: Effects of tool-contract edits by patch operation. Results are for Hunyuan-3-High. Drop is the paired success drop in percentage points against each base task’s clean outcome, and

\Delta

cost compares perturbed and clean mean token cost among successful trajectories. Color intensity encodes drop magnitude.

Level	Representative edit
L1	Rewrite the description of read_text_file to claim that only listed file extensions are readable. apex-0097f20d/T_mistake
L2	Remove list from the calendar action enum. apex-075c6f8f/T_missing
L3	Add a nonexistent zip_folder tool with a plausible compression interface. apex-0097f20d/T_redundant
L4	Remove read_text_file from the advertised tool set. apex-0097f20d/T_missing

Appendix

Table 6: Representative tool-contract edits. Each row shows one patch per edit level, compressed to the field that changes, and gray text gives the source perturbation.

Figure 8: Correlations between perturbation families. (a) Partial Spearman correlations of relative degradation across $n=75$ model–task pairs, controlling for clean performance and model identity. (b) The 66 pairwise correlations grouped by the component the two families share, with vertical bars marking group means.

Group	Tool calls	Turns	Input tokens	Output tokens	$\bm{n}$
By perturbation type
Missing	0.89 [0.66, 1.37]	0.87 [0.62, 1.30]	1.00 [0.75, 1.24]	1.02 [0.72, 1.44]	96
Incorrect	0.90 [0.67, 1.22]	1.00 [0.63, 1.18]	0.92 [0.64, 1.17]	0.98 [0.68, 1.47]	93
Redundant	0.93 [0.56, 1.43]	0.91 [0.60, 1.42]	0.91 [0.57, 1.27]	0.99 [0.72, 1.39]	90
By perturbation target
$U$ User utterance	0.93 [0.67, 1.10]	0.82 [0.63, 1.18] *	0.92 [0.62, 1.11]	0.91 [0.71, 1.30]	69

Appendix

Table 7: Rollout length under perturbation. Entries are median perturbed-to-clean ratios with first and third quartiles in brackets, so values below one indicate shorter perturbed rollouts. Blue and orange medians mark decreases and increases, shaded entries denote nominal significance (

p<.05

), and

n

is the number of matched trajectory pairs.

Quantity	$\bm{\rho}$	95% CI
Context-Missing & User-Incorrect	0.66	$[0.50,\,0.81]$
Tool-Redundant & Tool-Missing	0.61	$[0.41,\,0.79]$
Context-Missing & User-Redundant	0.60	$[0.44,\,0.77]$
Procedure-Incorrect & User-Incorrect	0.13	$[-0.18,\,0.46]$
Same target ( n = 12)	0.45	$[0.35,\,0.56]$
Same type ( n = 18)	0.37	$[0.26,\,0.49]$

Appendix

Table 8: Bootstrap intervals for family correlations. Entries are cluster-bootstrap percentile intervals for the partial Spearman correlations of Figure 8 . The upper block gives the three strongest and the weakest family pair, the middle block groups all 66 pairs by what the two families share, and the lower block reports the contrasts between these groups.

Evaluation	Perturbation target				Type
	$U$	$P$	$C$	$T$	Miss.	Red.	Inc.
Foundation-model robustness
PromptRobust	$\triangle$	–	–	–	–		$\triangle$
SCORE	$\triangle$	–	–	–	–	–	–
Math-RoB	$\triangle$	–	–	–			–
Agentic robustness

Appendix

Table 9: Coverage of prior robustness evaluations. A check marks direct coverage and a triangle a partial or analogous match, expressed in our terminology.

T

denotes the static tool contract rather than runtime tool outputs, so interventions on observations or environment state may have type coverage without a target check.

Target	Schematic edits
$U$	Missing: remove the requested cache TTL. Redundant: append an earlier deployment request after a topic switch. Incorrect: change the target branch from main to release .
$P$	Missing: omit the required approval condition. Redundant: retain an obsolete review workflow beside the current policy. Incorrect: replace “test before merge” with “merge before test.”
$C$	Missing: drop the earlier traceback needed for diagnosis. Redundant: carry over discussion from an unrelated repository. Incorrect: attribute a configuration value to the wrong module.
$T$	Missing: hide a required tool argument. Redundant: add duplicate aliases for the same repository operation. Incorrect: expose search-code while the server registers search_code .

Appendix

Table 10: Examples of the 12 perturbation families. All examples are set in one repository-coding task, and each group changes only the listed target.

Panel of Figure 9 — Figure 9: Case Study One, an unrecovered precondition. A Missing perturbation removes the acquirer’s multiple from the carried-over plan sketch. The highlighted row marks the step at which the clean and perturbed runs diverge.

Panel of Figure 10 — Figure 10: Case Study Two, unverified evidence selection. An Incorrect perturbation redirects the carried-over plan sketch from precedent toward an internal policy note. The highlighted row marks the step at which the clean and perturbed runs diverge.

Target	attack_face	Source surface	Returned replacement	Domain
$U$	instruction	instruction.md	Full rewritten instruction	Coding, MCP
$P$	plan	Rough plan / skill note	Markdown appended to instruction	MCP
$C$	context	Synthetic prior conversation	Serialized role–content turns	MCP
$T$	tool_schema	MCP tools/list	Serialized schema patch operations	MCP

Appendix

Table 11: Target-to-generator mapping. Each row lists the generator input, source surface, returned replacement, and applicable domain for one perturbation target.

Missing	Quietly remove information that is genuinely needed, rather than decorative, so that an agent that follows the surface blindly cannot complete the task.
Incorrect	Introduce a realistic human-like orthographic, naming-convention, or configuration error that remains recoverable without changing the task’s true answer.
Redundant	Keep the truth intact while adding plausible but irrelevant material, ghost entries, duplicates, or verbose detail that dilutes attention.

Appendix

Table 12: Corruption semantics. Each row gives the instruction injected into the shared generation prompt for one perturbation type. The implementation label mistake corresponds to Incorrect .

LLM Agent Evaluation

Large language model agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently.

See all 1,194 papers

Uncovering Robustness Weaknesses in LLM Agents via Persona Evolution with Task-Integrity Constraints
Oct 12, 2026Yutao Xie, Jingbo Yang, Guanyu Yao +3LLM Agent Evaluation User Simulation
University of California, Santa Barbara · Cresta
ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents
Aug 23, 2026YiShan Zheng, Yuan Wu, Yi ChangLLM Agent Evaluation LLM Tool Use
Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use
Jul 1, 2026Song-Lin Lv, Weiming Wu, Rui Zhu +2Fine-Tuning Distribution Shift
School of Intelligence Science and Technology, Nanjing University, Nanjing, China · National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

Oct 12, 2026J/K move · Enter open · S save

Uncovering Robustness Weaknesses in LLM Agents via Persona Evolution with Task-Integrity Constraints

Yutao Xie, Jingbo Yang, Guanyu Yao, Xiaoyu Yang+2

University of California, Santa Barbara · Cresta

LLM Agent Evaluation User Simulation

Open paper AskPDF arXiv

Interactive-agent evaluations commonly rely on standardized LLM-based user simulators that exhibit broadly cooperative behavior, potentially overlooking failures that arise under diverse but realistic user interactions. This can lead to overly optimistic estimates of agent robustness and provide limited failure signals for training agents that generalize across users. Existing approaches largely rely on predefined user behaviors, which may miss agent- or domain-specific weaknesses, while failure-oriented search can reduce task success by invalidating the underlying task rather than exposing genuine robustness failures. To fill this gap, we introduce TIPER, a task-integrity-preserving persona evolution method that automatically discovers agent-specific user policies exposing robustness weaknesses while keeping the underlying task valid and solvable. TIPER evolves persistent user personas through reflection-guided behavioral mutations and task-wise Pareto selection under a task-integrity constraint. Across four interactive-agent benchmarks, TIPER yields lower held-out avg@3 scores in all 12 GPT-5.4 settings, with observed reductions of 5.8 to 33.7 percentage points, and in all 28 agent-domain pairs spanning seven agents, with a median observed reduction of 22.9 points. Further analysis yields a behavior-to-failure taxonomy that connects discovered user behaviors to recurring agent failure modes. Together, these results show that TIPER enables more diagnostic robustness evaluation of interactive agents and provides informative failure analysis for future agent robustness improvement. Code repository: https://github.com/UCSB-NLP-Chang/TIPER.

Abstract

LLM Agent Evaluation

Explore similar work

Uncovering Robustness Weaknesses in LLM Agents via Persona Evolution with Task-Integrity Constraints

ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents

Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use