We present CANDID (Candidate-Driven Intervention for Disciplined Execution), an execution-control framework designed to enhance the reliability of local computer-use agents (CUAs) in resource-constrained settings. While local CUAs feature robust baseline proficiency in single-step action prediction, they frequently fail at critical junctions due to rigid action loops or premature DONE decisions. Rather than imposing imprecise cognitive loads through textual reflection, CANDID repurposes parallel test-time scaling into a disciplined, execution-level intervention at sparse, failure-prone points. We introduce a loop-recovery mechanism to filter candidates away from repeated behaviors and a conservative, unanimous DONE gating policy that requires absolute consensus to terminate. Experiments on OSWorld demonstrate that CANDID's structural control significantly improves success rates, outperforming feedback-based alternatives with minimal execution overhead.
Figures & tables
Figure 1: Operational workflow of a computer-use agent. The agent takes the task instruction and a multimodal history of h past [screenshot, response] pairs as input to generate Python-executable GUI actions up to a maximum step budget ( s ).
Figure 2: Failure modes of local CUAs on the OSWorld benchmark. Stacked bars show task outcomes: success, repetitive action loop, premature false success, step budget exhaustion without loops, and other failures. (a) Varying recent history depth ( h∈{0,1,4} ) with s=50 . (b) Varying maximum step budget ( s∈{15,50,100} ) with h=4 .
Feedback
Metric
Rate (%)
Loop Breaker
Repeated exact action
61.5
Repeated type/target
64.3
Failure after recovery
92.8
Verifier
Repeated DONE after rejection
73.4
Next action repeats recent target
44.3
Failure after rejection
75.9
Table 1: Behavioral outcomes after feedback interventions on Qwen3-VL-30B-A3B. Despite receiving Loop Breaker or Verifier feedback, the agent frequently repeats the same behavior or continues to fail, indicating that feedback alone offers limited recovery capability.
Figure 3: A local agent can generate valid alternative candidate actions at failure-prone states. In (a), the agent repeatedly presses Ctrl+V despite an empty clipboard, while another candidate types the URL directly. In (b), the agent prematurely declares task completion before modifying the document, while another candidate continues execution by clicking the target text for editing.
Figure 4: Overview of CANDID, our candidate-based execution-control framework for local computer-use agents. CANDID intervenes at two high-risk control points: (a) Loop Recovery: A loop detector monitors the execution history for repetitive local behavior and triggers structured candidate generation and filtering to escape the loop. (b) Unanimous DONE Gating: When the baseline agent proposes DONE , CANDID samples additional action candidates from the same context and executes DONE only when all valid candidates agree on termination.
Qwen3-VL 8B
Qwen3-VL 30B-A3B
OpenCUA-7B
Exact-action loop rate
32.12%
29.79%
7.09%
Unanimity rate
88.2%
78.2%
40.8%
Avg. unique alts.
1.23
1.45
2.34
Multi-type events
1.1%
4.3%
23.4%
Applied control
Loop
Loop + DONE
DONE
Table 2: Model-specific diagnostics with k=4 resampled candidates and the resulting control configuration. Exact-action loop rate denotes the fraction of steps that repeat the preceding action. Avg. unique alts. denotes the average number of unique non- DONE alternatives in disagreement cases, while multi-type events denotes the fraction of disagreements spanning multiple coarse action types.
Model
Method
Trials
Domain Accuracy (%)
Overall (%)
Office (117)
Daily (78)
Prof. (49)
Workflow (93)
OS (24)
Qwen3-VL-30B-A3B (s=50)
Baseline
5
37.43
48.15
41.63
18.25
36.67
35.31 ± 0.65
CANDID
5
39.42
49.76
46.94
22.52
46.67
38.80 ± 1.24 (+3.49)
Qwen3-VL-30B-A3B (s=100)
Baseline
5
37.54
48.23
41.23
18.78
36.16
35.42 ± 1.34
CANDID
5
40.63
50.59
48.57
23.82
42.93
39.68 ± 0.77 (+4.26)
Qwen3-VL-8B (s=50)
Baseline
5
32.74
43.37
44.49
14.77
44.57
32.82 ± 0.98
Table 3: Main evaluation results on the OSWorld benchmark, averaged over multiple trials (trial counts noted per row). Values in parentheses next to domain names indicate the task count. Overall accuracy is reported as mean ± standard deviation; values in gray parentheses represent the absolute improvement over each model’s baseline.
Method
Domain Accuracy (%)
Overall (%)
Office (117)
Daily (78)
Professional (49)
Workflow (93)
OS (24)
Baseline
37.43
48.15
41.63
18.25
36.67
35.31
Test-Time Scaling Method
GTA-1
12.74
38.41
48.98
9.77
54.17
25.19
Feedback-Based Methods
Self-Guidance
34.96
36.36
44.90
18.26
50.00
33.30
Table 4: Comparison with inference-time control baselines using the Qwen3-VL-30B-A3B-Instruct. We compare CANDID against a GTA-1 Yang et al. (2025) test-time scaling method, step-wise Self-Guidance, and VLAA-GUI-style Han et al. (2026) feedback methods, including Loop Breaker, Verifier, and their combined variant. VLAA-GUI Loop Breaker is averaged over four runs, Verifier and Combined over three runs, and Baseline and CANDID over five runs; GTA-1 and Self-Guidance are based on a single run.
Figure 5: Failure-mode distribution and accuracy across component configurations on Qwen3-VL-30B-A3B ( s=50 ). Loop Recovery primarily reduces repetitive loops, Unanimous DONE reduces premature false success, and their combination achieves the highest average score.
Baseline
Majority
Unanimous
Accuracy (%)
26.06
27.24
30.60
Hit Max
19.33
29.33
64.67
False Success
241.33
232.00
171.67
Redirection (%)
–
5.53
49.93
Table 5: Comparison of majority and unanimous DONE selection on OpenCUA-7B with a 100-step budget.
Baseline
Per-Step Sampling
CANDID
Accuracy (%)
35.31
23.48
38.80
Hit Max
135.0
239.0
116.6
False Success
97.4
44.0
105.2
Avg. Steps
25.61
36.89
24.40
Table 6: Comparison of dense per-step sampling and sparse intervention with CANDID on Qwen3-VL-30B-A3B with a 50-step budget. Hit Max and False Success are non-mutually-exclusive trajectory statistics; a task may satisfy both conditions.
Figure 6: Effect of candidate count on task accuracy across the two local CUAs. Results with four candidates are averaged over five runs, while the remaining settings are averaged over three runs. Shaded regions denote ±1 standard deviation across repeated runs.
Statistic
Loop Recovery
DONE Gating
Total Steps
9371
9371
Triggered
2496 (26.6%)
429 (4.6%)
Intervention Applied
1426 (57.1%)
196 (45.7%)
Original Action Kept
1070 (42.9%)
233 (54.3%)
Selector Invoked
1043 (41.8%)
96 (22.4%)
Selector Bypassed
1453 (58.2%)
333 (77.6%)
Table 7: Operational statistics of the candidate-based control framework on Qwen3-VL-30B-A3B. Percentages for “Triggered” are computed over all execution steps, while the remaining percentages are computed over the triggered steps.
Figure 7: Efficiency comparison of baseline and CANDID on Qwen3-VL-30B-A3B. Execution time includes model inference, action execution, and environment initialization, with average steps and token usage.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Trigger reason
Condition
Intuition
repeated_action_type_and_target
The proposed action has a target coordinate, and at least r−1 recent actions within the window have the same action type and a target within tolerance τ of the proposed target.
The agent is repeatedly operating on the same local screen region.
repeated_targetless_action_type
The proposed action type is scroll or wait , and at least r−1 recent actions in the window have the same action type.
The agent is repeatedly scrolling or waiting without a concrete target.
repeated_exact_text_or_key_action
The proposed action type is type or key , and the normalized full action string appears at least r−1 times in the recent window.
Text entry and keyboard commands require exact matching because the payload matters.
repeated_response_and_action_type
The previous two assistant responses are identical after normalization, and there is at least one recent action with the same action type as the proposed action.
The model is reproducing the same reasoning or action description while continuing the same type of operation.
repeated_cyclic_action_pattern
The recent action-signature sequence plus the proposed action ends with a repeated pattern of length 2–4.
The agent alternates among a small set of actions rather than repeating one literal action.
Appendix
Table 8: Rule-based loop trigger conditions. Here r is loop_recovery_min_repeats , τ is loop_recovery_target_tolerance , and the recent action window is loop_recovery_window . In our main results, these are r=3 , τ=80 , and a window of 5 actions.
Module
Diversity Metric
1
2
3
4
Loop Recovery
Exact action
209
435
662
1,190
Loop Recovery
Action type
2,095
389
12
0
DONE Gating
Exact action
238
96
73
22
Appendix
Table 9: Candidate diversity at triggered control points.
w
τ
r
Trials
Overall (%)
5
80
3
5
34.84 ± 1.27
3
80
3
3
36.05 ± 0.71
7
80
3
3
35.94 ± 0.85
5
0
3
3
36.88 ± 0.44
5
40
3
3
35.02 ± 1.28
5
120
3
3
34.96 ± 0.67
Appendix
Table 10: Sensitivity analysis of the loop detector hyperparameters using Qwen3-VL-8B with a step budget of s=50 . Results are reported as mean task accuracy (%) ± standard deviation over multiple trials.
Method
TP
FP
TN
FN
Loop Recovery
26
148
8
10
Unanimous DONE Gating
18
67
114
70
Appendix
Table 11: Task-level outcomes of triggered control decisions. Final task success is used as a proxy because OSWorld does not provide per-step ground-truth control labels.
Method
Triggered
TP
FP
TN
FN
VLAA-GUI Verifier
229
113
95
17
4
Unanimous DONE Gating
269
132
101
36
0
Appendix
Table 12: Task-level comparison of termination-gate decisions on Qwen3-VL-30B-A3B with a 50-step budget.
Method
Precision
Recall
Specificity
Accuracy
VLAA-GUI Verifier
54.3%
96.6%
15.2%
56.8%
Unanimous DONE Gating
56.7%
100.0%
26.3%
62.5%
Appendix
Table 13: Precision, recall, specificity, and accuracy of the termination gates.
Model
Method
Domain Accuracy (%)
Overall (%)
Office (43)
Web (30)
Windows System (24)
Coding (24)
Media (21)
Windows Utilities (12)
Qwen3-VL-30B-A3B
Baseline
2.33
36.67
33.33
33.33
9.81
25.00
21.47
CANDID
2.33
33.33
50.00
29.17
24.09
25.00
24.71 (+3.24)
Qwen3-VL-8B
Baseline
0.00
20.00
16.67
12.50
14.29
0.00
10.39
CANDID
0.00
23.33
20.83
12.50
14.29
0.00
11.69 (+1.30)
Appendix
Table 14: Evaluation results on 154 WindowsAgentArena tasks with a step budget of s=50 and history length h=4 . Values in parentheses next to domain names indicate the task count. Results are based on a single run; values in gray parentheses represent the absolute improvement over each model’s baseline.
Deploying autonomous computer-use agents (CUAs) locally is increasingly important for privacy, cost efficiency, and practical usability, yet improving their performance under strict hardware constraints remains challenging. While recent studies show that inference-time scaling can improve frontier computer-use agents through additional computation during execution, its effectiveness for resource-constrained local models remains poorly understood. We present a systematic empirical study of inference-time scaling in local CUAs across contextual, temporal, structural, and parallel dimensions. We evaluate Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark. Our results show that additional computation often yields diminishing returns while changing failure modes. Contextual scaling provides historical grounding that improves trajectory stability and task accuracy, but its gains saturate as token cost increases and failures shift from repetitive or stalled trajectories toward premature false successes. Temporal scaling similarly reduces max-step stalls, yet does not substantially improve task success, indicating that longer horizons often extend erroneous trajectories rather than correct them. We further find that structural decomposition can introduce planning and formatting overhead in local two-stage agents, while parallel scaling partially mitigates these failures at a substantial computational cost. Overall, our findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks designed around the capabilities and limitations of local models.
Modern GUI-agent frameworks achieve strong desktop task performance with frontier API models, yet persistent control information often remains implicit in growing interaction trajectories. At each step, the planner reconstructs the active task stage, accumulated evidence, and runtime feedback before deciding the next action. This dependence becomes more pronounced under weaker local reasoning backbones. Across four representative state-of-the-art frameworks, replacing GPT-5 with Qwen3.5-9B reduces average OSWorld SR-100 from 60.9% to 35.2%. Trajectory annotation further identifies at least one control failure in 94.7% of failed trajectories. To address this problem, we introduce LocalLSTC, a training-free architecture that organizes control by temporal scope, maintaining persistent cross-step state to guide short-term execution commitments. The framework comprises two complementary mechanisms. Long-to-Short Planning forms each commitment from persistent state, while Short-to-Long Control integrates execution outcomes back into that state for progress assessment, recovery, and termination. With Qwen3.8-27B, LocalLSTC reaches 73.7% SR-100 on OSWorld and 68.5% on WindowsAgentArena, achieving the strongest local OSWorld result and a new state of the art on WindowsAgentArena. Ablations further support contributions from both mechanisms. Together, these results show that temporal organization of control information can reduce cross-step control failures and complement backbone scaling in locally deployed GUI agents.
Weiming Li, Helen Paik, Yulei Sui
School of Computer Science and Engineering The University of New South Wales Sydney, Australia
Recent advances in foundation-model agents have enabled increasingly capable embodied systems for long-horizon planning and interaction. However, embodied agents require fast, closed-loop decisions as they continuously perceive and act on a changing environment. Deploying strong cloud agents leaves the platform idle for seconds between consecutive actions, while lightweight local models respond quickly but are more vulnerable to errors. Existing hierarchical architectures typically assign models fixed roles, whereas conventional model routers make stateless per-request decisions. We present CARE, a critic-guided adaptive execution framework that dynamically determines when to escalate control from a local executor to a stronger cloud model. A lightweight critic model monitors execution reliability and triggers escalation when local execution becomes unreliable; once escalated, control persists until the agent recovers, avoiding step-wise switching. To support such control transfer, a shared handoff memory maintains execution context and task-level progress across executors, enabling the model to intervene without restarting from scratch. Across diverse embodied benchmarks, CARE improves task success by 6.3-18.3 percentage points over the strongest evaluated collaboration baseline. CARE further approaches or exceeds Cloud-only success while reducing median time-between-action latency by 28.8-82.6%.
Wei Zhang, Lirong Xiang, Charles Fleming
University of Illinois Urbana-Champaign · Cornell University · Cisco