LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents

Qingni Wang; Changdae Oh; Yuzhe Yang; Sharon Li; Jiachen Yang; Ang Li; Xin Eric Wang

cs.AIOct 15, 2026

LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents

Authors: Qingni Wang, Changdae Oh, Yuzhe Yang, Sharon Li, Jiachen Yang, Ang Li, Xin Eric Wang

Organizations: Simular · University of Wisconsin–Madison

Abstract

Computer-use agents increasingly tackle long-horizon tasks, where running a frontier model at every step is expensive and a small open-weight model alone often fails. Weak-to-strong escalation offers a middle ground where a weak agent acts by default and hands control to a strong one only when needed. The core challenge is deciding when to escalate, which requires an uncertainty signal cheap enough to check at every step. However, even after handoff, the strong agent can still get lost, but a black-box API that the strong agent is usually served exposes no internals to monitor. To this end, we propose LEASH, a training-free framework that unleashes the strong agent only when the weak agent's latent uncertainty signals it is needed, and keeps it on a leash afterward. LEASH has three layers: (1) initial task uncertainty for early routing, (2) step-level uncertainty for runtime escalation, and, after handoff, (3) a shadow observer for continued oversight of the strong black-box agent. Here, we use latent uncertainty as a fuel for all three layers. Specifically, we compare the weak model's hidden states with those from its past successful and failed trajectories, and a state that looks more like the failures signals higher uncertainty. During weak-agent execution, our uncertainty estimator reuses activations already produced during action generation, so it needs no extra forward pass or dedicated monitor training. Experiments on OSWorld across four contemporary model backbones show that LEASH consistently improves over standalone weak agents and outperforms random escalation by 6.55--11.94 points at matched escalation rates. With Qwen3.5-35B-A3B, LEASH achieves 73.66% task success at $0.4117 per task, matching always-on Claude Opus 5 (72.90%) while reducing inference cost by 19.2%.

Figures & tables

Figure 1: Accuracy–cost comparison between LEASH and random task-level routing on Qwen3.5-35B-A3B. The area spanned by random routing is obtained from repeated random task allocations across different escalation budgets; the band shows

\pm 1

standard deviation over 10,000 allocations. At matched inference cost, LEASH achieves 9.1 percentage points higher task success.

Figure 2: Overview of uncertainty-aware weak-to-strong escalation and post-handoff oversight. A weak agent goes through the task until its uncertainty scores trigger initial or step-level escalation; otherwise, the weak agent continues. After escalation is triggered, the strong agent retains control while the weak agent changes its role from an actor to a shadow observer for continued monitoring. The feedback path from weak observer to strong actor enables uncertainty-triggered reflection and intervention. In short, LEASH unleashes the strong agent only when the weak agent’s latent uncertainty signals it is needed, and keeps it on a leash afterward.

Figure 3: Construction of the query and action reference banks for latent space uncertainty estimation.

Agent	Escalation Method	Esc. (%)	Success (%) $\uparrow$	$\Delta$ Success	Cost ( $)$ \downarrow$
EvoCUA-8B	-	0.0	42.37	–	–
	Random	45.4	59.98	+17.61	0.3031
	LEASH	45.4	67.18	+24.81	0.3627
Qwen3.5-35B-A3B	-	0.0	37.40	–	–
	Random	64.4	61.72	+24.32	0.3768
	LEASH	64.1	73.66	+36.26	0.4117

Table 1: Task success and estimated inference cost on OSWorld. Random escalation routes a matched fraction of tasks to Opus 5 uniformly at random; success is averaged over 10,000 random allocations. Esc.% denotes the fraction of tasks handed off through the initial or runtime gate and excludes post-handoff Shadow monitoring.

\Delta

Success is the absolute improvement over the corresponding standalone agent. Cost is reported in USD per task.

Backbone	Method	Esc. (%)	Success (%) $\uparrow$	Cost ( $)$ \downarrow$
EvoCUA-8B	-	0.0	42.37	–
	StepWise ( Wei et al., 2026 )	49.6	58.40	0.2824
	Progress Advantage ( Oh et al., 2026a )	46.9	62.98	0.3533
	LEASH	45.4	67.18	0.3627

Table 2: Comparison with existing escalation and uncertainty estimation methods, StepWise ( Wei et al., 2026 ) and Progress Advantage ( Oh et al., 2026a ) , with EvoCUA-8B on OSWorld.

Stage	Backbone	Probability-based	Euclidean	kNN	Pooled LDA (Ours)
Initial	Nemotron-3 (30B-A3B)	0.575	0.692	0.672	0.749
	Qwen3.8-27B	0.671	0.541	0.572	0.600
Runtime	Nemotron-3 (30B-A3B)	0.488	0.583	0.489	0.609
	Qwen3.8-27B	0.540	0.564	0.522	0.575

Table 3: AUROC of uncertainty estimators. For the probability-based column, we report the best-performing uncertainty statistic in each setting. Initial and runtime AUROC use task outcomes and Manager-assigned subgoal outcomes, respectively.

Configuration	Manager	Escalation	Shadow	Success (%) $\uparrow$	Cost ( $)$ \downarrow$
Weak agent alone	$\times$	$\times$	$\times$	37.40	–
+ Manager	$\checkmark$	$\times$	$\times$	41.15	0.1372
+ Escalation	$\checkmark$	$\checkmark$	$\times$	67.94	0.4050
LEASH	$\checkmark$	$\checkmark$	$\checkmark$	73.66	0.4117

Table 4: Component ablation of LEASH on Qwen3.5-35B-A3B. Checkmarks indicate enabled components. Shadow includes post-handoff oversight and checkpoint intervention.

Subset	Tasks	Shadow (%)	No Shadow (%)	Opus only (%)
Both escalated	131	69.5	67.9	64.9
Shadow triggered	31	64.5	48.4	51.6
Escalated, not triggered	137	74.5	69.3	74.5

Table 5: Subset analysis of post-handoff Shadow monitoring with Qwen3.5-35B-A3B. Both escalated contains tasks handed off by both Shadow and No-Shadow policies. Shadow triggered contains tasks with at least one checkpoint trigger, while Escalated, not triggered contains escalated tasks without a trigger. Opus only reports always-on Opus accuracy on the same task IDs.

Bank size	Qwen3.5-35B-A3B	EvoCUA-8B
40	$0.701\pm 0.100$	$0.633\pm 0.099$
60	$0.744\pm 0.060$	$0.654\pm 0.061$
80	$0.749\pm 0.036$	$0.681\pm 0.049$
99	$0.770\pm 0.021$	$0.696\pm 0.022$

Table 6: Sensitivity to reference bank size. Initial-stage AUROC is reported as mean

\pm

standard deviation across subsamples.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Backbone	Layer
EvoCUA-8B	35 / 36
Qwen3.5-35B-A3B	39 / 40
Nemotron-3-Nano-Omni (30B-A3B)	51 / 52
Qwen3.8-27B	63 / 64

Appendix

Table 7: Hidden layer used for each weak backbone (0-indexed layer / number of layers).

Gate configuration	Success (%) $\uparrow$	Cost ( $)$ \downarrow$
Initial only	59.92	0.3122
Runtime only	71.65	0.4169
LEASH	73.66	0.4117

Appendix

Table 8: Gate configurations for Qwen3.5-35B-A3B. Each configuration retains the Manager; the first two disable post-handoff Shadow monitoring and checkpoint intervention.

Panel of Figure 6 — Figure 6: Manager prompt excerpts. The strict verification instruction is appended to the replanning system prompt; it is not a standalone verifier prompt.

Panel of Figure 7 — Figure 7: Handoff and checkpoint prompts. Handoff retains up to six recent weak-agent steps by default; bracketed screenshot markers indicate image inputs, not literal prompt text. The neutral checkpoint requests reassessment without asserting that execution has stalled and explicitly permits continuing the current approach.

Figure 8: Runtime escalation case study I: font standardization. The weak agent becomes stuck in repeated macro-interface interactions. After handoff at step 11, the strong agent switches to a terminal-based python-pptx strategy and successfully completes the task.

Figure 9: Runtime escalation case study II: photo selection. The weak agent enters a repeated open–inspect–close loop. After handoff at step 8, the strong agent changes to keyboard navigation and multi-selection and successfully creates the requested archive.

Figure 10: Runtime escalation case study III: extension installation. The weak agent continues browsing without completing the installation. After handoff at step 24, the strong agent switches to a terminal-based installation strategy and successfully installs the requested extension.

Computer-Use Agents

Computer-use agents perceive and operate graphical interfaces and software tools to complete tasks across websites, apps, and desktops.

See all 160 papers

Act or Escalate? Evaluating Escalation Behavior in Automation with Language Models
Mar 31, 2026Matthew DosSantos DiSorbo, Harang JuLLM Decision-Making AI-Assisted Decision Making
Harvard Business School · Johns Hopkins Carey Business School
Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents
Aug 25, 2026Nadeem ShaikhAdaptive Model Routing LLM Agent Orchestration
Independent Researcher Melbourne, Australia
Step-level Optimization for Efficient Computer-use Agents
Apr 29, 2026Jinbiao Wei, Kangqi Ni, Yilun Zhao +2Computer-Use Agents AI Agent Monitoring
Y Yale NLP Lab · CUniversity of North Carolina at Chapel Hill

Mar 31, 2026J/K move · Enter open · S save

Act or Escalate? Evaluating Escalation Behavior in Automation with Language Models

Matthew DosSantos DiSorbo, Harang Ju

Harvard Business School · Johns Hopkins Carey Business School

LLM Decision-Making AI-Assisted Decision Making Selective Prediction Language Model Calibration

Open paper AskPDF arXiv

Effective automation hinges on deciding when to act and when to escalate. We model this as a decision under uncertainty: an LLM forms a prediction, estimates its probability of being correct, and compares the expected costs of acting and escalating. Using this framework across five domains of recorded human decisions-demand forecasting, content recommendation, content moderation, loan approval, and autonomous driving-and across multiple model families, we find marked differences in the implicit thresholds models use to trade off these costs. These thresholds vary substantially and are not predicted by architecture or scale, while self-estimates are miscalibrated in model-specific ways. We then test interventions that target this decision process by varying cost ratios, providing accuracy signals, and training models to follow the desired escalation rule. Prompting helps mainly for reasoning models. SFT on chain-of-thought targets yields the most robust policies, which generalize across datasets, cost ratios, prompt framings, and held-out domains. These results suggest that escalation behavior is a model-specific property that should be characterized before deployment, and that robust alignment benefits from training models to reason explicitly about uncertainty and decision costs.

Abstract

Computer-Use Agents

Explore similar work

Act or Escalate? Evaluating Escalation Behavior in Automation with Language Models

Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents

Step-level Optimization for Efficient Computer-use Agents