Long-horizon business workflows fail in two ways: errors accumulate as a chain of activities lengthens, and the knowledge an activity needs is written in a different vocabulary from the user's request. We study whether such workflows can be made reliable without changing the language model. Our method keeps the backbone fixed and moves the workflow definition, state and domain knowledge into versioned control assets outside the model. A state machine and machine-checkable acceptance gates enforce these assets during execution; an independent auditor grades the resulting trace. The assets are updated only after a regression check and human approval .We evaluate the method on three deployed workflows exceeding 200 steps. On Domain A, a 120-case end-to-end evaluation set shows unattended completion rising from 50% to 100%, retrieval recall from 56% to 91.67%, and end-to-end latency dropping from 440 to 159 minutes, with scenario-analysis recall rising from 45% to 93%. Two other domains were taken over by teams that did not write the assets, one in four days and one in seven days. A controlled ablation separates the two failure sources. The code is available at https://github.com/ZTE-AICloud/Co-Sight/tree/master/Co-Sight%20Harness.
Figures & tables
Method
Goal
Control locus
Knowledge supply
Deterministic
Evaluation
Replication
Long-horizon
AFlow / DGM ( Zhang et al., 2025 ; Zhang et al., 2026 )
workflow graph
in-workflow
none
no
outcome
no
no
RHO / AHE
pass rate
in-harness
partial
partial
outcome
no
partial
HarnessPro ( Pan et al., 2026b )
pass + cost
in-harness
none
partial
synthesised tests
no
no
DSPy / GEPA
prompt or program
in-prompt
partial
no
outcome
no
no
ReasoningBank / DC / BATS
single component
in-skill / memory
partial
no
outcome
no
no
Ours
completion, quality, transfer
external K
dual engines
state machine + hooks
GoldEval , ability/regression split
3 deployed domains
200+ steps
Table 1: Comparison of harness optimization and control approaches. Knowledge supply and cross-team replication are the two dimensions on which our setting differs from prior work.
Figure 1: Overview of the proposed method and its implementation.
Shared across domains (verbatim)
Per-domain configuration constants
K∩θ=∅ ; Compile(u) with Gate(K) ; δ with InvViol=0 ; generation–evaluation separation; read-only auditing; ability/regression separation; regression-gated admission with human approval
subagent split granularity (11); context-window grades (4–8k / 12–16k / 20–32k); contents of the 8 review-rule classes; golden-set contents (120-case golden set); φ lexicon; Q , Qrt , rmax , pass@3 , pass3 ; HITL high-risk list
Table 2: Shared invariants versus per-domain configuration constants.
Table 3: Deployment instantiations of the four control planes.
(a) completion and content quality
Metric
Before
After
Change
Unattended completion
50%
100%
+50 pp
Generation recall
68%
91.25%
+23.25 pp
Generation precision
67%
84.38%
+17.38 pp
Retrieval recall
56%
91.67%
+35.67 pp
Requirements-analysis latency
169 min
56.4 min
−67%
Table 4: Domain A main results.
Figure 2: Illustration of the product-form failure mechanism (purely schematic; the measured 200+ step / 50% baseline is reported in Table 4 (a) and is not a point on these curves).
Figure 3: Execution gating and evidence generation in the deployed system. The gate labels and the bounded-rework label denote declared contracts; the released runtime enforces the explicit verdict at the terminal state and treats the rework budget as a design contract (Appendix A). The golden set contains 120 cases; pass@3 and pass3 are deployment thresholds, not measured pass rates.
Domain
Effort
Outcome
Domain A (first task force)
original team
100% completion; −64% latency; three of four recall targets reached
1 day manual stitching → 5 h under gate review; V0.5 release
Table 5: Cross-team transfer.
Figure 4: Tuning dynamics and evaluation-system evolution. In (a) the six rounds are measured on the scenario-analysis slice of the 120-case set; the dotted 80% line is a reference level of this retrieval-recall curve and is not the four-activity target of Table 4 (b). Panel (b) shows the judgement granularity of the evaluation system; no performance values are implied.
Invariant / line
Intermediate observable
End-to-end outcome (measured)
Grade
(2) Deterministic enforcement: DoD gates, externalized state, bounded rework
a latent failure becomes an observed, repairable event at ai
CR50%→100% and L440→159 min (120-case set, Table 4 a)
Domain B 2×4 days, stability 50%→90% ; Domain C 1×7 days, 1 day →5 h (Table 5 )
B
Table 6: Attribution chain from invariants to measured end-to-end outcomes. A = reproducible deterministic check; B = field measurement with a stated sample size; C = descriptive deployment observation.
Arm
Component added
CR (%)
95% CI
R / P / F1
L (min)
Wilson
CP
B0
none: monolithic agent, long prompt, no K
50.0
[41.2, 58.8]
[40.7, 59.3]
0.45 / 0.19 / 0.267
440
B1
DoD gates, externalized state, state machine
75.0
[66.6, 81.9]
[66.3, 82.5]
0.55 / 0.24 / 0.334
250
B2
dual-engine retrieval and semantic alignment
60.0
[51.1, 68.3]
[50.7, 68.8]
0.82 / 0.38 / 0.519
400
B3
both lines (Full)
90.0
[83.3, 94.2]
[83.2, 94.7]
0.88 / 0.42 / 0.569
170
B4
longer prompt and more retries, compute-matched to B3
60.0
[51.1, 68.3]
[50.7, 68.8]
0.56 / 0.24 / 0.336
415
Table 7: Additive controlled ablation ( n=120 , R=13 ). Completion is reported as a percentage with its 95% confidence interval computed by the Wilson method and by the Clopper–Pearson exact method. R , P and F1 are proportions in [0,1] . L is the end-to-end latency in minutes. All quantities are means over the R=13 runs. This study is not pooled with Table 4 .
Arm
Component removed
CR (%)
95% CI
R / P / F1
L (min)
Wilson
CP
A0
all: monolithic baseline
50.0
[41.2, 58.8]
[40.7, 59.3]
0.45 / 0.19 / 0.267
440
A1
none (Full reference)
88.3
[81.4, 92.9]
[81.2, 93.5]
0.88 / 0.42 / 0.569
159
A2
DoD gate and bounded rework
65.8
[57.0, 73.7]
[56.6, 74.2]
0.83 / 0.39 / 0.531
388
A3
externalized state and checkpoint recovery
70.8
[62.2, 78.2]
[61.8, 78.8]
0.82 / 0.38 / 0.519
340
A4
generation–evaluation separation
84.2
[76.6, 89.6]
[76.4, 90.2]
0.90 / 0.43 / 0.582
174
Table 8: Subtractive controlled ablation ( n=120 , R=13 ). Completion is reported as a percentage with its 95% confidence interval computed by the Wilson method and by the Clopper–Pearson exact method. R , P and F1 are proportions in [0,1] . L is the end-to-end latency in minutes. All quantities are means over the R=13 runs. A4 and A5 have the same completion count but differ in content quality. This study is not pooled with Table 4 .
Language-model agents increasingly participate in operational workflows where approvals, binds, and payments create commitments. An approval valid when issued may no longer authorize execution after relevant facts, evidence, policy, or authority change. We call this the evaluation--execution gap. We specify a state-bound runtime assurance layer that separates probabilistic measurement from deterministic adjudication, binds an execution allowance to the decision-relevant basis of an approval, invalidates that allowance after relevant logged changes, and rechecks admissibility at dispatch. This distinguishes the continued validity of a prior approval from a current-state policy check. We describe eight failure modes, an assurance model, a reference architecture, and prospective interventions with retrospective remediation. In a scripted 42-case replay, the state-bound configuration identifies 15 later-invalidated approvals that a deliberately non-re-evaluating point-in-time comparator does not revisit; it holds 14 affected actions before dispatch and compensates one landed effect. These results illustrate behavior under the stipulated policies and event orders, not superiority over dispatch-time enforcement designs or production effectiveness. Under explicit interception, ingestion, dependency, and serialization assumptions, the model establishes dispatch-time validity within the registered execution surface.
Human approval provides effective control only if rejected work stops and authorized work completes. We formalize refusal safety and bounded authorization completeness, and audit three execution boundaries: the model selects a call, the harness schedules and resumes it, and the workflow routes the decision to a protected effect. Paired decisions, exact traces, and native business-state witnesses support experiments with three models, five prompt arms, four runtime approval designs, two additional framework replays, and 100 public workflow sources. The paired model panel leaves seven of 300 approvals unfinished despite zero denied effects; parallel confirmation loses an approved branch in three tasks per model, restoration omits 336/672 CrewAI approvals, and four of 59 comparable n8n workflows execute after Reject. Comparing intent, generated calls, dispatch, and native effects identifies the first observable divergence and its repair target. This yields a practical audit contract that makes both refusal safety and authorization completeness independently verifiable, converting human approval from a procedural formality into an enforceable control boundary between intent and real-world effect.
Bangjie Deng, Xiaoxuan Lu, Wenjun Chen +8
University of Pennsylvania · Independent Researcher · Columbia University +7
Much of the automation handed to language-model agents is procedural: each step needs little intelligence, but a long unattended run must follow a skill or SOP stated once at the start. We show that agents drift from such procedures over long runs, and that restating the instructions or resuming a stopped agent only moves the drift to another shortcut. We study an executable finite-state machine (FSM) that keeps the workflow state outside the model's context and enforces it at three points: tool calls, state transitions and the end of the agent's turn. In a pre-registered experiment on 200-account SOP-Bench batches, Claude Haiku 4.5 with the FSM decided on average 139 accounts correctly and by the procedure, against 48 for the strongest baseline (95% CI of the difference [46, 130]), and kept this rate to the end; across three models, the gain tracked how much the plain agent drifted. Because the runtime, not the model, supplies this long-horizon reliability, the FSM can let small models carry procedural work: Haiku 4.5 with it averaged more such decisions than plain Claude Sonnet 4.6 at 38% of the cost. Analyses on SWE-bench Verified, τ2-bench, and staged and cross-context workflows further suggest that reliable long-horizon automation can come from stronger execution control, not only stronger models.
Yutao Sun, Haokun Li, Mingshuai Chen +4
Zhejiang University, China · Zhejiang Key Laboratory of Digital-Intelligence Service Technology · University of Oxford, UK +1