Post-training binds the competence of an LLM agent to the interface it was trained on. Semantics-preserving perturbations of that interface, such as renaming an action or changing the notation, leave the underlying task unchanged yet make the agent worse at it. Scoring each sample independently means interface randomization only broadens this dependence from a single interface to the training interface distribution, so out-of-distribution perturbations still degrade the agent. We propose Cross-Interface Policy Regularization (CIPR), a self-supervised objective that replaces action names with uninformative identifiers and presents each training sample under two interfaces in which equivalent actions are identified by construction. CIPR penalizes the KL divergence between the action distributions induced under the two interfaces, so the policy behaves consistently across them. We evaluate CIPR in both supervised and reinforcement-learning post-training, where it improves robustness. On agentic function calling, held-out interface perturbations reduce the accuracy of a post-trained agent by nearly 25%, and CIPR recovers 10.5% while leaving accuracy on the unchanged interface intact.
Figures & tables
Figure 1: Interface robustness of a tool-using agent (schematic). In each column, we post-train an agent to fetch weather forecasts (top) and then test it at inference (bottom) on a renamed interface ( Interface 1 ) and on one with an added, cheaper forecast tool ( Interface 2 ). The label on each arrow into an agent says whether the test interface is in that agent’s training distribution. Standard post-training on one interface fails both. Multi-interface training handles the renaming, which it saw during training, but misses the new tool. CIPR, which also compares the policy across interfaces, chooses by description in both cases.
Figure 2: Overview of CIPR. We first canonicalize the interface, renaming the two tools of a weather query f01 and f02 , short for function_01 and function_02 . We then build a perturbed interface that adds a degraded twin of every tool, such as the deprecated forecast f73 , and assigns all tools new identifiers in a new order. Each bar chart is an action distribution, the probability of calling each tool or none ( ∅ ). The rescoring consistency KL holds one response fixed and compares these distributions at each of its decisions, before and after its tool names are relabeled in the reasoning and the call. The rollout consistency KL lets each interface produce its own responses, so the reasoning can differ, and compares their average distributions, in which the twin f73 has probability zero under the canonical interface. Probabilities are illustrative.
In distribution ↑
Held-out perturbations ↑
Model
Tool names
Original ↑
Twins & reordering
Typos & distractors
Cost & lat. (explicit)
Cost & lat. (implicit)
Mean
Gap ↓
Qwen2.5-3B, tools in prompt
Base
original
63.7
n/a
59.9
24.2
17.7
33.9
29.8
canonical
61.5
43.5
59.3
41.9
28.5
43.2
18.3
ToolRL (GRPO)
original
73.3
n/a
71.6
29.3
20.9
40.6
32.7
canonical
70.7
49.1
68.9
46.0
33.5
49.5
21.2
Table 1: Full-call accuracy (%) on BFCL under interface perturbations. Tool names gives the names each model sees at evaluation. Each model is evaluated with the names it was trained on, as it would be deployed, so Base and ToolRL (GRPO) see the original names, and the multi-interface and CIPR models, which are trained with canonical names, see canonical names through the canonicalization adapter. Gray rows evaluate Base and ToolRL (GRPO) through the canonicalization adapter as well. In distribution applies the perturbation used in training (Section 3.2 ), which is defined on canonical names, so rows with the original names show n/a. Mean averages the three held-out groups, and Gap is original accuracy minus the mean. Bold marks the best value per backbone, excluding gray rows. All 95% confidence intervals have half-widths of at most 2.0 points, and Appendix D reports each perturbation type.
Figure 4
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Qwen2.5-3B
Qwen3.5-4B
Training examples
3,920
3,920
Learning rate
10−6
10−6
Prompts per update
512
256
Optimization minibatch
128
128
Training budget
15 epochs (about 105 updates)
15 epochs (about 105 updates)
Rollouts per interface ( G )
4
4
Appendix
Table 3: Hyperparameters of the reinforcement-learning runs.
Operation
Change
Example
Twin expansion
Copy every tool and degrade the copy’s description or required arguments. The original stays available.
A forecast tool gains a copy marked deprecated; a date-range query loses its end date in favor of an unrelated width argument.
Relabeling
Assign new neutral identifiers to the expanded tool list and update every reference to them.
f01 becomes f42 , and its twin becomes f73 .
Tool reordering
Shuffle the tools after expansion, regardless of whether a tool is an original or a twin.
Twins and originals do not occupy fixed positions.
Argument reordering
Shuffle argument declarations while keeping their names, types, descriptions, and requiredness.
(a,b,c) becomes (c,a,b) with the same values bound.
Appendix
Table 4: Construction of the perturbed training interface. The query is unchanged, and f01 abbreviates function_01 . These operations build the training pairs and the in-distribution evaluation of Table 1 ; Table 5 lists the held-out perturbations.
Figure 4: One BFCL item before and after canonicalization (top), and under the cost perturbation with the query endings of the explicit and implicit versions (bottom). Descriptions are unchanged by canonicalization; tool and argument names are replaced.
Group
Types
What changes
Typos & distractor tools
Typo
Character-level typos, with letters swapped, substituted, dropped, or doubled, in about one in five query words of four or more letters.
RedunTool
For each tool the task calls, an unrelated tool whose name matches that tool’s name except in the last word.
Dup-NoDesc
For each tool the task calls, a duplicate with the same name, no description, and no parameters.
Dup-Desc
The same duplicate with the original description and no parameters.
Dup-WrongP
The same duplicate with no description and the parameters of another tool.
Dup-DescWP
The same duplicate with the original description and the parameters of another tool.
Appendix
Table 5: Held-out perturbation types used on BFCL, adapted from RobustBench-TC and regenerated for all 2,501 items. In all six cost and latency types the correct tool receives a new name and the familiar name goes to the costlier or slower equivalent, which RobustBench-TC does only in its abbreviated variants, and the implicit version is ours. None is used during training.
Base
ToolRL (GRPO)
original
canonical
original
canonical
+ multi- interface
+ CIPR
Original interface
63.7
61.5
73.3
70.7
74.0
74.9
Typos & distractor tools
Typo
59.1
57.4
68.7
67.3
69.6
70.5
RedunTool
63.2
62.6
72.5
69.5
73.5
73.6
Dup-NoDesc
58.3
62.2
72.4
71.9
73.6
75.1
Appendix
Table 6: Accuracy (%) on BFCL by perturbation type for Qwen2.5-3B, with tools in the prompt. Types are those of Table 5 . Base and ToolRL (GRPO) are evaluated with the original names and, in gray, through the canonicalization adapter; the multi-interface and CIPR models use canonical names. With canonical names, each cost and latency entry averages the two tool orders.
Base
ToolRL (GRPO)
+ CIPR
original
canonical
original
canonical
+ multi- interface
4B
2B
9B
Original interface
84.2
82.0
84.8
84.6
84.8
85.0
74.9
85.8
Typos & distractor tools
Typo
79.2
78.0
80.2
79.9
79.6
80.7
68.1
81.8
RedunTool
84.5
81.8
84.7
85.0
84.6
85.1
75.5
85.9
Dup-NoDesc
84.4
82.4
84.9
85.0
85.2
85.0
74.7
86.0
Appendix
Table 7: Accuracy (%) on BFCL by perturbation type for Qwen3.5 with native function calling. All columns are Qwen3.5-4B except the last two, which train CIPR on Qwen3.5-2B and Qwen3.5-9B. Rows and conventions as in Table 6 .
Airline
Retail
Training
Original
Typos & distractors
Cost & lat. explicit
Cost & lat. implicit
Original
Typos & distractors
Cost & lat. explicit
Cost & lat. implicit
Qwen2.5-3B, tools in prompt
Base
42.0
38.0
37.7
37.0
6.7
6.2
6.4
5.6
ToolRL (GRPO)
38.0
39.4
42.0
37.8
5.5
6.3
6.6
5.5
+ multi-interface
40.0
37.4
40.3
37.3
6.6
6.4
6.3
6.1
+ CIPR
44.0
40.3
39.2
39.5
6.5
6.2
5.9
6.1
Appendix
Table 8: Mean task reward ( ×100 ) on τ2 -bench. Rows and conventions follow Table 1 , with Base and ToolRL (GRPO) using the original names and the multi-interface and CIPR models canonical names. Original is 50 simulations on airline and 74 to 93 scored simulations of the 114 retail tasks, with 95% confidence half-widths of 12 to 14 points on airline and 4 to 11 on retail.
Cost & latency (explicit)
Perturbed interface
Consistency KL
Original
Typos & distractor tools
Original order
Costlier first
Mean
Twins, reordered
actions
74.7
72.1
60.5
46.9
53.7
–
73.5
71.3
56.7
46.7
51.7
–, twin penalty
73.6
70.7
56.4
46.4
51.4
Reordered
actions
73.5
71.8
36.3
59.9
48.1
actions, G=16
75.5
72.9
26.3
66.4
46.3
Appendix
Table 9: Every ablation run on Qwen2.5-3B, with canonical names (accuracy, %). The last three columns split the explicit cost and latency perturbations of Table 2 by tool order, the original order and the one in which the costlier or slower equivalent comes first. Configurations shared with Table 1 differ from it by at most 0.5 points.
Figure 5: Failures of Qwen3.5-4B under the implicit cost and latency perturbations, as a percentage of the 30,012 perturbed items. The classes partition the failures, so the bars below Any failure sum to it up to rounding.
Figure 6: Instances of the failure classes of Qwen3.5-4B under the implicit cost and latency perturbations. Descriptions are abbreviated, bracketed amounts are the stated costs, and […] marks omitted text.
Corruption level
Model
Clean
Slight
Medium
Heavy
Union
Corrupted
Original names
Qwen2.5-3B
Base
39.0 ±9.5
37.1 ±6.7
38.1 ±6.7
21.4 ±5.5
28.6 ±9.0
31.7 ±3.3
ToolRL (GRPO)
40.0 ±9.5
41.0 ±6.9
41.0 ±6.7
22.9 ±5.5
23.8 ±8.1
33.3 ±3.4
+ multi-interface
40.0 ±9.5
40.0 ±6.7
41.0 ±6.7
23.8 ±5.7
25.7 ±8.6
33.6 ±3.4
Appendix
Table 10: Content-Filling accuracy (%) on the RoTBench First-Turn split under original names and canonical names. The last column pools the four corrupted levels. Subscripts are half-widths of 95% confidence intervals, computed per item and paired between original and canonical names; n=105 for Clean and Union and n=210 for the other levels. Bold marks the best entry per column within a backbone.
Large language model post-training adapts pretrained models into more useful, reliable, and specialized assistants through supervised and preference-based learning.
Real-world tasks are often complex and noisy, posing substantial challenges for LLM agents. Yet existing evaluations lack a systematic framework for jointly characterizing potential perturbation targets and types. We use this two-dimensional view to conduct a unified study that quantifies the robustness of current frontier models, traces the mechanisms behind perturbation-induced failures, and draws implications for robustness-oriented training. Through controlled evaluations on APEX-Agents and Toolathlon, we find that perturbations consistently degrade performance, while the dominant vulnerabilities vary across task environments. Trajectory-level analysis reveals that agents rarely recognize altered inputs and often propagate corrupted information without verification. A focused cost analysis further shows that successful perturbed trajectories can require substantially more computation. These findings motivate training objectives that cover diverse forms of noise and reward verification, uncertainty awareness, and recovery. We provide a rigorous benchmark-construction pipeline. Code and data will be released at https://github.com/xuyj233/AgentPerturb.
Yuanjian Xu, Jinyi Han, Jianing Hao +5
The Hong Kong University of Science and Technology (Guangzhou) · East China Normal University · University College London +1
Tool-augmented language-model agents rely on interface text: tool descriptions, argument schemas, and the candidate tool list shown at inference time. These interfaces often change after deployment even when the underlying tool semantics remain fixed, creating a practical reliability problem for function-calling systems. We study this failure mode as interface drift and evaluate whether a lightweight, training-free repair layer can recover robustness without retraining the base model. Our method, SchemaShield-Lite, combines schema validation, candidate-set retry for missing tool names, and one-shot constrained repair. On Qwen3.5-9B, it raises DICE-300 accuracy from 0.717 to 0.813 and BFCL-200 accuracy from 0.580 to 0.805, recovering 19/22 and 33/45 drift-induced failures with no observed repair harms. Naive retry does not explain the gains. Ablations show that candidate-set drift, especially distractor tools, accounts for most observed damage, while description and schema edits alone are less harmful. The results position validation-and-repair as a practical robustness layer for deployed LLM-agent tool use.
Aniruddha Chattopadhyay, Kanishka Halder, Partha Sarathi Roy
Interactive-agent evaluations commonly rely on standardized LLM-based user simulators that exhibit broadly cooperative behavior, potentially overlooking failures that arise under diverse but realistic user interactions. This can lead to overly optimistic estimates of agent robustness and provide limited failure signals for training agents that generalize across users. Existing approaches largely rely on predefined user behaviors, which may miss agent- or domain-specific weaknesses, while failure-oriented search can reduce task success by invalidating the underlying task rather than exposing genuine robustness failures. To fill this gap, we introduce TIPER, a task-integrity-preserving persona evolution method that automatically discovers agent-specific user policies exposing robustness weaknesses while keeping the underlying task valid and solvable. TIPER evolves persistent user personas through reflection-guided behavioral mutations and task-wise Pareto selection under a task-integrity constraint. Across four interactive-agent benchmarks, TIPER yields lower held-out avg@3 scores in all 12 GPT-5.4 settings, with observed reductions of 5.8 to 33.7 percentage points, and in all 28 agent-domain pairs spanning seven agents, with a median observed reduction of 22.9 points. Further analysis yields a behavior-to-failure taxonomy that connects discovered user behaviors to recurring agent failure modes. Together, these results show that TIPER enables more diagnostic robustness evaluation of interactive agents and provides informative failure analysis for future agent robustness improvement. Code repository: https://github.com/UCSB-NLP-Chang/TIPER.