Large language model (LLM) agents increasingly act as service consumers. They select operations and report whether a task is complete. In an asynchronous distributed service, an accepted call or immediate state read does not establish that the required state became observable by its deadline. We introduce execution-grounded completion contracts, which treat completion as a service-level outcome. An independent control plane observes a deadline-bound service-state predicate, admits completion, and directs recovery while the agent performs the task. We evaluate placement and deletion in a live multi-provider data service with heterogeneous models and an injected transfer delay. Across policies, some tasks that missed the deadline were accepted and some that met it were rejected. Under enforcement, accepted completions remained consistent with later observations; all included healthy tasks completed, and unresolved faulted tasks terminated as unverified. The results show that completion of these tasks can be governed by independent observation of service state rather than by the agent's self-assessment.
Figures & tables
Approach
Evidence checked
Decision point
Deadline
Attribution (agent or service)
Action after a failed check
AgentSpec [ 27 ]
The agent’s trajectory and next action
On a trigger: state change, before an action, or agent finish
No
No
Stop, ask the user, self-examine, or run a corrective action
ToolGate [ 15 ]
Tool preconditions and results against the agent’s trusted symbolic state
At each tool call
No
No
Block the call or discard its result
STRATUS [ 9 ]
Severity of the managed system’s state
After each mitigation transaction
No
No
Undo the transaction
Verify-Gated Completion [ 20 ]
Claim and evidence packets, common ground, and task state
When the agent proposes completion
No
No
Withhold completion; recovery is a separate path
This paper
A pre-specified predicate over live backend state, observed outside the agent’s session
At completion, from observations up to the deadline
Yes
Yes, from runtime evidence
Withhold completion; the deadline violation stays recorded; recovery routed by the attributed cause
Table 1: Comparison of agent-verification approaches and their verification semantics.
Figure 1: Completion-contract architecture (red: agent, non-authoritative for completion; green: contract control plane; blue: distributed service). An operator instantiates a task contract from a template. The verifier observes service state outside the agent’s context; its verdict gates completion, and the attribution informs recovery.
Placement task
Deletion task
Goal
“replicate this file to site B, and confirm it arrived”
“delete this file from everywhere — not just hidden; confirm it is fully removed”
Fulfillment condition
the file’s byte count at the target site equals the write-time expected size
the path is absent and no site retains bytes for the pre-captured file identifier
Verification
state polled every 3 s to the deadline; the service’s own status signals are diagnostic only
path-absence and per-site residue probes on the pre-captured identifier, polled to the deadline
Recovery
routed by attribution: re-invoke with the canonical target (agent) / wait (service) / terminate as unverified
same routing (Sect. 5.3 )
Table 2: Evaluated contract instances. The predicates and expected values were fixed before each run; the agent was the obligated component in both tasks.
Mode
Task
Healthy I/P
Fault I/P
Primary use
ungated
placement
56/56
56/56
RQ1; RQ2
ungated
deletion
56/56
—
RQ1; organic violations
gated
placement
56/56
56/56
RQ3
gated
deletion
50/56
—
RQ3
Table 3: Study cells used by the three RQs, reported as included/planned. A dash denotes a condition not run, not a zero outcome. Full cells hold seven models, eight runs each.
Read-back class
n (of 21)
What the recorded read showed
Read, then ignored
6
Nothing at the target; reported done anyway.
Job status, not data
5
Took the transfer job’s successful submission as arrival; the target itself still held zero bytes.
Late-but-real convergence
5
Placement genuinely converged, but only past the 30-second deadline.
Wrong signal (directory aggregate)
4
Read a nonzero parent-directory total, mistaking it for the target file.
No read-back
1
No read-back issued before reporting done.
Table 4: Recorded read-back preceding each of the 21 false completion reports in the healthy ungated placement cell. Counts sum to 21.
Completion policy
Healthy FP
Healthy FA
Fault FP
self-report
21/56 (38%)
9/56 (16%)
37/56 (66%)
call/health monitoring
14/56 (25%)
1/56 (2%)
47/56 (84%)
transcript judge
0/56
20/56 (36%)
1/56 (2%)
state at report time
1/56 (2%)
0/56
0/56
state until deadline (reference)
0/56
0/56
0/56
Table 5: Placement-policy error incidence, shown as n/56 (percentage) within each cell. Healthy placement contains 23 Violated and 33 Fulfilled reference outcomes; all 56 injected- fault outcomes are Violated.
Deadline T (s)
10
20
25
30 (used)
35
40
healthy late cases at T
37
22
13
4
2
1
fault passes
0
0
0
0
0
0
Table 6: Deadline sensitivity from re-scoring the same 56 healthy and 56 faulted placement trajectories at each T . “Healthy late cases” counts trajectories observed to converge within the recorded horizon, but only after T ; the 19 never-convergent healthy trajectories remain Violated but are not in that row. “Fault passes” counts faulted trajectories satisfied by T .
Completion policy
FP incidence
FP among Violated
self-report
12/56 (21%)
12/16 (75%)
call/health monitoring
11/56 (20%)
11/16 (69%)
transcript judge
0/56
0/16
state at report time
9/56 (16%)
9/16 (56%)
state until deadline (reference)
0/56
0/16
Table 7: Deletion false passes as trial incidence and conditional on the 16 Violated among 56 ungated trials. The transcript judge also false-alarmed on all 40 Fulfilled.
Attribution quantity
Observed result
ground-truthable against injected cause
55/56 (98%)
evidence observable
37/55 (67%)
shadowed by missing valid operation
18/55 (33%)
agreement when evidence observable
37/37 (100%)
Table 8: Attribution accounting for the 56 injected-fault placement trials. One truth- neither trial lies outside the injected-cause denominator; shadowed trials remain in the ground-truthable denominator.
Gate outcome
Healthy (56)
Fault (56)
verified completions
56
53
later-contradicted completions
0
0
unverified terminations (explicit)
0
3
trials recovered after a failed first attempt
29 (52%)
53 (95%)
runtime-activity index per verified completion (mean)
17.88
32.58
Table 9: Outcomes of the 112 gated placement trials. “Recovered” means the first attempt failed and a later attempt was verified; it does not revise the first verdict.
Model
Rescued (/8)
By re-invoke
By re-check
Qwen3-Coder-30B
6
6
0
Llama-3.3-70B
5
5
0
GLM-4.7-Flash
4
4
0
GLM-5.2-FP8
4
0
4
claude-sonnet-5
8
1
7
claude-haiku-4-5
2
0
2
Table 10: First-attempt failures recovered by mechanism in the 56 healthy gated placement trials. Each model contributes eight trials.
Runtime-activity index
Model
Healthy
Injected delay
gemma-4-31B
9.12
28.00
claude-haiku-4-5
11.88
24.00
GLM-5.2-FP8
15.50
31.71
Llama-3.3-70B
19.25
50.50
claude-sonnet-5
20.50
20.12
Table 11: Unweighted runtime-activity index per verified placement completion. Each entry sums verifier probes, agent invocations, and gate retries, divides by the number of verified completions, and gives the three event types equal weight.
Ungated
Gated
Model
Condition
in time
USD
reached
USD
Spend change
claude-haiku-4-5
healthy
8 / 8
0.2532
8 / 8
0.2826
+11.6%
fault
0 / 8
0.3343
8 / 8
0.3518
+5.2%
claude-sonnet-5
healthy
5 / 8
1.5733
8 / 8
1.8380
+16.8%
fault
0 / 8
1.8811
8 / 8
2.0627
+9.7%
Table 12: Recorded model charges (USD) in eight-trial cells for the two SDK-billed models, with and without the gate. Ungated counts completions within the deadline; gated counts trials that reached the predicate, including after recovery. Service and infrastructure costs are excluded: the federation is shared, and its cost cannot be reliably attributed to individual tasks; the verifier’s cost is reported as its probe count.
Figure 1: Experimental setup. The Onedata services run on Kubernetes (k3s) and are installed from the public Helm charts; each Oneprovider serves POSIX storage. The harness host runs the agent loop, the onedata-mcp tool server, the verifier, and the gate.
Path
Content
recordings/
779 files: one JSON record per trial, grouped by study population, and the runs of the verifier-overhead benchmark
Large language model (LLM) agents increasingly execute tasks asynchronously: a main agent continues working while its subagents run, and the task state may change before their results arrive. The main agent must decide which work to update or preserve and which checks to repeat, while avoiding prohibited changes. Final task success alone does not reveal whether these results were handled correctly, and natural asynchronous arrivals are hard to reproduce. We introduce Async-RBench, a benchmark for asynchronous task execution in LLM agents, with 200 executable tasks covering eight scenarios. A scenario controller regulates result delivery using predefined task-state conditions, while an Event Response Contract (ERC) specifies what each response must change, preserve, avoid, and re-verify. The Dynamic Replanning Score (DRS) weights the response process and outcome equally and is reported separately from task success. We evaluate nine agents and find DRS scores of 19.2 to 45.1, with the lowest scores on duplicate reports and subagent failures. Under our execution protocol, all nine agents obtain lower task success with asynchronous than with batched delivery (26.0% vs. 29.7% on average). For three agents, prompting the main agent to rebuild its plan after every result improves DRS by only 2.6 to 3.5 points. Together, these findings highlight the challenges of reliable asynchronous task execution and the need to evaluate how agents handle incoming results alongside end-to-end task success.
Detao Zeng, Guilin Ou, Nuo Chen +5
University of Electronic Science and Technology of China
Large language model (LLM) agents have increasingly advanced service applications, such as booking flight tickets. However, these service agents suffer from unreliability in long-horizon tasks, as they often produce policy violations, tool hallucinations, and misaligned actions, which greatly impedes their real-world deployment. To address these challenges, we propose NOD (Navigator-Operator-Director), a heterogeneous multi-agent architecture for service agents. Instead of maintaining task state implicitly in dialogue context as in prior work, we externalize a structured Global State to enable explicit task state tracking and consistent decision-making by the Navigator. Besides, we introduce selective external oversight before critical actions, allowing an independent Director agent to verify execution and intervene when necessary. As such, NOD effectively mitigates error propagation and unsafe behavior in long-horizon tasks. Experiments on τ2-Bench demonstrate that NOD achieves higher task success rates and critical action precision over baselines. More importantly, NOD improves the reliability of service agents by reducing policy violations, tool hallucinations, and user-intent misalignment.
Zixu Yang, Hang Zheng, Nan Jiang +5
1X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · 2Shanghai Innovation Institution, Shanghai, China · 3Jiangsu Key Lab of Language Computing, Suzhou, China
Concurrent actions in large language model (LLM) agent environments require arbitration even when each proposal is individually valid. We implement a typed snapshot-settlement contract and audit three distinct properties: order sensitivity, useful progress, and replay consistency. Five settlement policies are tested in 28,800 exhaustive permutation trials and 2,160 scripted multistep episodes. Joint policies are spatially order-invariant conditional on fixed priorities, yet conservative rejection completes only 31.25% of agents in a six-agent doorway task versus 90.28% for random tickets; the paired improvement is 59.03 percentage points (95% bootstrap interval: 50.00-68.06). All policies preserve the tested spatial constraints, and priority arbitration still misses the independent small-instance optimum. A separate full-state journal audit exactly replays 156 checkpoints and rejects 1,332 constructed corruptions with a retained terminal anchor. The evidence concerns execution semantics, not human realism or long-run fairness.
Haotian Chen, Bowen Ye, Yuning Zhang +1
School of Cyber Science and Technology, University of Science and Technology of China · School of Education, Shanghai Jiao Tong University · SWJTU-Leeds Joint School, Southwest Jiaotong University