Organizations: Lero, the Research Ireland Centre for Software, University of Limerick, Ireland. · The Chinese University of Hong Kong, Shenzhen, China. · Xi’an Jiaotong University, China. · Peking University, China. · South China University of Technology, China. · Genesys Cloud Services Inc.
Tool-using language agents increasingly perform regulated customer-service workflows, where a correct final state is insufficient: agents must also provide disclosures, obtain consent, and follow required procedures. Existing tool-agent benchmarks largely score final business states, accepting trajectories that reach correct outcomes through non-compliant processes. We introduce cxbench, a benchmark that evaluates outcome and process compliance separately across six regulated domains. Each task combines a customer request, a tool-based environment, and explicit process requirements. Rule cards link cited regulatory obligations to observable trace evidence. A hybrid grader combines deterministic checks for ordering and authority with an LLM judge for semantic requirements. We define a false-pass as an outcome-pass run that fails process checks. Across seven models and 120 tasks, 55.5% of the 840 retained executions are false-passes, representing 80.1% of outcome-passing runs; only 13.8% pass both checks. Process-inclusive evaluation changes the observed model ordering and exposes differences hidden by outcome-only scoring. cxbench makes these failures inspectable through trace-linked criteria.
Figures & tables
Figure 1: Two traces with the same final state. An outcome-only oracle accepts both, whereas cxbench identifies Trace B as a false-pass because it omits the written-confirmation notice required in this case under Regulation E (see footnote 1 ). Draft confirmation is a separate cxbench runtime control rather than a statutory requirement.
Benchmark
Process
Statute
False-pass
Runtime submit
AgentBench / OSWorld ( Liu et al., 2024 ; Xie et al., 2024 )
×
×
×
×
tau-bench / tau2-bench ( Yao et al., 2024 ; Barres et al., 2025 )
×
×
×
×
TheAgentCompany ( Xu et al., 2024 )
×
×
×
×
LegalBench ( Guha et al., 2023 )
×
×
×
×
SOP-Bench ( Nandi et al., 2025 )
×
×
×
×
SOPBench / SOP-Maze ( Li et al., 2025 ; Wang et al., 2025b )
✓
×
×
×
Table 1: Comparison with representative benchmarks on process evaluation, regulatory grounding, false-pass reporting, and runtime-controlled submission. A checkmark indicates support for the listed feature; a cross indicates its absence, rather than the absence of all policy-related evaluation.
Figure 2: The cxbench evaluation pipeline. The agent interacts with a simulated customer and business tools, producing a final state and an interaction trace. Outcome evaluation checks the final state; process evaluation checks the trace using deterministic rules and an LLM judge. In the default mode, the runtime submits drafts after customer confirmation.
Produce
Role
Responsibility
Execution
Agent
Interacts with the customer and uses available tools to complete the task while following the applicable requirements.
Execution
Customer simulator
Provides information based on private customer facts and confirms drafts whose required fields match those facts.
Execution
Runtime
Executes tool calls, records the interaction, and controls submission in mediated-submit mode.
Evaluation
Rule layer
Checks deterministic conditions, including event ordering, tool permissions, and agreement between confirmed and submitted fields.
Evaluation
LLM judge
Assesses semantic requirements using a fixed rubric and task-specific checklist.
Validation
Expert reviewers
Reassesses selected traces to examine the validity of the automated evaluation.
Table 2: Roles in cxbench . The agent, customer simulator, and runtime produce the interaction; the rule layer and LLM judge evaluate it. Expert reviewers independently reassess selected traces for validation and do not determine the reported process verdict.
Figure 3: From regulatory requirements to process evaluation. A rule card connects a cited requirement to observable trace evidence and an evaluation criterion (top). Deterministic checks and an LLM judge assess the requirements applicable to each task and jointly determine the process verdict (bottom). A correct outcome accompanied by a failed process verdict is a false-pass.
Model
Evaluated
Outcome-pass
Both-pass
False-pass
records
Rate (%)
Count
Rate (%)
95% CI
Rate (%)
Count
glm-5.1
120
77.50
26
21.67
[15.24, 29.85]
55.83
67
gpt-5.4
120
64.17
25
20.83
[14.53, 28.95]
43.33
52
gpt-5.4-mini
120
61.67
18
15.00
[9.70, 22.47]
46.67
56
qwen3.6-plus
120
70.83
18
15.00
[9.70, 22.47]
55.83
67
deepseek-v4-flash
120
77.50
16
13.33
[8.38, 20.56]
64.17
77
Table 3: Per-model results over all 120 tasks. Evaluated records include task failures. Outcome-pass measures outcome correctness; both-pass requires passing both outcome and process checks; false-pass denotes outcome-pass with process-fail. Rates and Wilson 95% confidence intervals (CI) are percentages; CIs apply to the both-pass rate.
Mode
Outcome pass
Process pass
Both pass
False pass
Mediated-submit
64.17
21.67
20.83
43.33
Direct-submit
67.50
0.83
0.83
66.67
Direct-submit (checks excluded)
67.50
20.00
18.33
49.17
Table 4: Submission-control comparison for gpt-5.4 . All rows use the same 120 recorded traces. The third row excludes submission-mechanism checks from both the hard-rule and LLM-judge components and is a rescoring condition rather than an additional execution mode. All entries are percentages.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Element
Meaning
Schema or runtime representation
g
Customer goal
customer_goal
m0
Opening message
seed_message
s0
Initial execution state
initial_state , hidden_facts , and runtime-initialized history and drafts
U
Domain tool interface
tools , resolved against the domain’s registered tool implementations
Λ
Policy context and process requirements
policy , including source references, process requirements, and associated grading specifications; referenced rules
K
Fields requiring confirmation
confirmation_contract
Appendix
Table 5: Correspondence between the task representation and the task schema. Registered domain implementations provide the tool behavior and state initialization used with these fields.
Figure 4: Selected components of a banking task specification for an unauthorized ACH debit: (a) the customer goal, opening message, and initial business state; (b) domain-specific tool interfaces; (c) statute-linked policy requirements; and (d) the associated grading configuration. Private customer facts, confirmation fields, and other details are omitted for brevity.
Private customer facts, customer-facing messages, and presented draft fields used to generate replies and confirmation decisions.
Runtime
Task initialization data, registered tools, runtime mode, confirmation fields, drafts, and recorded interaction events.
Outcome checker
Customer goal, relevant reference facts, and final business state.
Process rule layer
Applicable deterministic rules and the recorded evidence needed to check ordering, permissions, and confirmation consistency.
LLM judge
The interpretation rubric, task-specific checklist, relevant task context, and recorded execution evidence.
Appendix
Table 6: Information used by the execution and evaluation components. Access is described by function; it does not imply that each component receives the complete task file.
Component
Description
Source provision
Regulation E, 12 CFR § 1005.11(b)(2), concerning written confirmation of an oral error notice.
Task conditions
The notice is oral, and the institution requires written confirmation.
Required information
The written-confirmation requirement and the address to which it should be submitted.
Trace evidence
Communication of the required information during the recorded interaction.
Benchmark criterion
The disclosure precedes the request for customer confirmation of the filing draft.
Appendix
Table 7: Regulation E disclosure example: the source obligation, applicable task conditions, required information, and benchmark evaluation criterion.
Category
Scope
Pipeline gates
The draft–confirm–submit ordering requirements shared across domains.
Domain ordering
Precedence between domain-specific steps, such as reading a policy summary before checking reporter authority.
Disclosure gates
Required disclosures before designated customer-confirmation events.
Semantic prohibitions
Restrictions such as unsupported outcome promises, prohibited collection of secrets, and unlawful disconnection threats.
Appendix
Table 8: Categories in the process-rule catalog. Definitions are presented here; observed rule failures are reported in Appendix G.2 .
Domain
Read
Check
Draft
Notice
Submit
Banking
6
5
3
4
2
Telecommunications
5
4
2
3
2
Insurance
6
5
3
3
2
Healthcare
5
3
1
2
1
Utilities
5
5
4
3
1
Government benefits
5
4
2
2
1
Appendix
Table 9: Registered tools by domain and tool category. Counts include aliases.
Figure 5: Outcome, hybrid process, and both-pass rates across models and domains. Each cell uses all 20 tasks for one model–domain pair. All panels share a 0–100% color scale.
Domain
Outcome pass
Both pass
False pass
Banking
94.29
5.71
88.57
Government benefits
84.29
4.29
80.00
Healthcare
13.57
2.14
11.43
Insurance
80.71
17.86
62.86
Telecommunications
67.14
26.43
40.71
Utilities
75.71
26.43
49.29
Appendix
Table 10: Pooled results by domain. Each domain contains 140 evaluations. All entries are percentages.
Figure 6: Failure rates for the 16 most frequently failing hard rules across the seven models. Each rate is normalized by the number of records whose task configures that rule for the corresponding model, so denominators vary by rule. Failures include missing required events as well as incorrect event ordering.
Mode
Outcome pass
Process pass
Both pass
False pass
Mediated-submit
64.17
21.67
20.83
43.33
Direct-submit
67.50
0.83
0.83
66.67
Direct-submit (checks excluded)
67.50
20.00
18.33
49.17
Appendix
Table 11: Submission-control ablation for gpt-5.4 . All rows use the same 120 task specifications and recorded traces. The third row excludes submission-mechanism checks from both the hard-rule and LLM-judge components; it is a rescoring condition rather than an additional execution mode. All entries are percentages.
Agent
gpt-5.6-sol
gpt-6-astra
Δ (pp)
gpt-5.4
20.83
20.00
−0.83
gpt-5.4-mini
15.00
17.50
+2.50
gpt-4o-mini
2.50
2.50
0.00
glm-5.1
21.67
28.33
+6.67
qwen3.6-plus
15.00
14.17
−0.83
qwen3.5-plus
8.33
5.83
−2.50
Appendix
Table 12: Both-pass rates under two judge configurations. Each model uses the same 120 traces for both judges. Rates are percentages; Δ is the change from gpt-5.6-sol to gpt-6-astra in percentage points.
gpt-5.6-sol : Pass
gpt-5.6-sol : Fail
Expert: Pass
23 (TP)
4 (FN)
Expert: Fail
10 (FP)
83 (TN)
Appendix
Table 13: Comparison of gpt-5.6-sol verdicts with expert labels on 120 traces generated by gpt-5.4-mini . These traces were used during judge calibration. Process pass is the positive class.
Tool-using large language model (LLM) agents are increasingly deployed in settings where their reliable behavior is governed by strict procedural manuals. Ensuring that such agents comply with the rules from these manuals is challenging, as they are typically written for humans in natural language while agent behavior manifests as an execution trace of tool calls. Existing evaluations of LLM agents rely on manually constructed benchmarks or LLM-based judges, which either do not scale or lack reliability for complex, long-horizon manuals. To overcome these limitations, we present MANTRA, a framework for automatically synthesizing machine-checkable compliance benchmarks from natural-language manuals and tool schemas. MANTRA independently generates (i) a symbolic world model capturing procedural dependencies, and (ii) a set of trace-level compliance checks for a given task, and validates their consistency using SMT solving. A structured repair loop resolves inconsistencies, requiring human intervention only as a fallback. %This yields benchmarks that are formally validated. Importantly, MANTRA supports arbitrary domains and long procedural manuals, and provides a tunable notion of task complexity which is utilized to automatically derive challenging tasks accompanying compliance checks. Using MANTRA, we build a new benchmark suite with 285 tasks across 6 domains scaling to 50+ page manuals with minimal human effort. Empirically, we show that the compliance checks are richer with stronger constraint enforcement compared to existing benchmarks. Additionally, the granularity of the checks can be used for debugging the agents' failure modes. These results demonstrate that combining automated benchmark generation with formally grounded validation methods enables scalable and reliable benchmarking of tool-using agents.
Ashwani Anand, Ivi Chatzi, Ritam Raha +1
Max Planck Institute for Software Systems, Kaiserslautern, Germany
While Large Language Models (LLMs) have evolved into tool-using agents, they remain brittle in long-horizon interactions. Unlike mathematical reasoning where errors are often rectifiable via backtracking, tool-use failures frequently induce irreversible side effects, making accurate step-level verification critical. However, existing process-level benchmarks are predominantly confined to closed-world mathematical domains, failing to capture the dynamic and open-ended nature of tool execution. To bridge this gap, we introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories. The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement. It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity. Extensive experiments reveal key insights: (1) weaker policy models exhibit inflated ratios of correct steps due to early termination; (2) distinguishing neutral and erroneous actions remains a significant challenge for current models; and (3) process-derived signals provide complementary value to outcome supervision, significantly enhancing test-time scaling. We hope AgentProcessBench can foster future research in reward models and pave the way toward general agents. The code and data are available at https://github.com/RUCBM/AgentProcessBench.
Shengda Fan, Xuyan Ye, Yupeng Huo +9
Renmin University of China Beijing, China · Beijing Jiaotong University Beijing, China · Shanghai Jiao Tong University Shanghai, China +1
Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails. A model that never calls a needed tool and a model that calls the tool but ignores the result can look similar under final task accuracy. We introduce ToolFailBench, a diagnostic benchmark for measuring tool-use failures across 1,000 tasks in finance, medicine, law, cybersecurity, and real estate. Tool-required tasks return values the model wouldn't guess, forcing it to trust the tool while control tasks attach the same tools but should be answered directly. We label each trace with Tool-Skip, Result-Ignore, Output-Fabrication, and Unnecessary-Tool-Use, using a rule classifier and two LLM judges aggregated by majority vote. Across 19 headline models, the best reaches 86.33% Clean Tool-Use Rate, showing that faithful tool use is not saturated. More importantly, models with similar aggregate scores fail in different ways: most stay disciplined on no-tool controls, while Llama-3.1 models show an Always-Call pattern, and at the same parameter scale Llama-3.1-70B and Qwen2.5-72B differ by 89 percentage points on control-task accuracy. Tool-use evaluation should measure not only whether agents call tools, but whether they use tool outputs correctly and avoid tools when none is needed.