EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation

Zeru Shi; Jindong Jiang; Ryo Hachiuma; Yonggan Fu; Byung-Kwan Lee; Peter Belcak; Saurav Muralidharan; Ayush Agarwal

cs.CLOct 13, 2026

EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation

Authors: Zeru Shi, Jindong Jiang, Ryo Hachiuma, Yonggan Fu, Byung-Kwan Lee, Peter Belcak, Saurav Muralidharan, Ayush Agarwal,

Organizations: NVIDIA · Rutgers University

Abstract

Complex and long-horizon agent tasks often rely on frontier models deployed in the cloud to achieve strong performance. However, using these models for every step of a long-horizon trajectory incurs substantial inference cost, even though not all steps may require the same level of model capability. In this paper, we find that allocating the LLM to only a small subset of key steps can achieve performance comparable to full LLM execution while significantly reducing inference cost. Motivated by this observation, we propose EdgeWise, an edge-cloud collaboration framework that deploys an SLM on the local device as the primary solver and selectively delegates critical steps to a cloud LLM when stronger model capacity is needed. To enable autonomous delegation without additional models at inference time, we introduce a step-level RL method that jointly optimizes task solving and delegation, with fine-grained credit assignment for individual delegation decisions. We also introduce an external judge-based delegation strategy and use it to systematically investigate key design choices in edge-cloud collaboration, including when to delegate, how delegation decisions should be made, what context should be shared, and how frequently the cloud LLM should be invoked. Experiments show that EdgeWise matches or outperforms pure-LLM execution while reducing inference cost up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1. We further deploy EdgeWise on a local workstation, demonstrating comparable cost savings in a practical edge-cloud setting.

Figures & tables

Figure 1: The upper panel illustrates the overall pipeline of EdgeWise, while the lower panel provides an example of how EdgeWise operates during task execution.

Figure 2: The leftmost and rightmost points represent Qwen3.5-9B-only and GPT-5.5-only, respectively, with four intermediate LLM usage ratios. The left figure shows accuracy as the LLM ratio increases, while the right reports the corresponding average cost over three runs. Each point is averaged over three runs, with shaded regions indicating standard deviation. This figure shows that, with a cloud usage rate of only around 40%, delegating the right steps to the cloud model achieves performance comparable to the vanilla model while reducing cost.

Figure 3: Left: Correlation between cloud coverage and task success across action types. Right: Cloud coverage differences between successful and failed trajectories. Error bars denote 95% confidence intervals. Higher coverage of Write/Edit steps is consistently associated with task success.

Figure 4: Performance and API cost of our training-free key-step delegation method compared with the pure large-model baseline. All metrics are averaged over three independent runs.

Method	Qwen3.5-9B + GPT-5.5(xhigh)				Qwen3.5-9B + Opus4.8(xhigh)
	Pass@1		Pass@3		Pass@1		Pass@3
	Acc. (%) $\uparrow$	Cost ( $)$ \downarrow$	Acc. (%) $\uparrow$	Cost ( $)$ \downarrow$	Acc. (%) $\uparrow$	Cost ( $)$ \downarrow$	Acc. (%) $\uparrow$	Cost ( $)$ \downarrow$
Terminal-Bench2.1
Vanilla LLM	74.16	90.31	82.02	279.73	67.42	64.78	75.28	187.72
Random Delegation	66.29	74.34	79.78	209.13	58.42	54.00	67.42	158.18
RSD [ 14 ]	62.92	45.61	78.65	133.25	53.93	28.09	61.80	82.39

Table 1: Performance and cost across benchmarks and baselines. Binary and Threshold Delegation costs include the external judge. Bold and underlined denote the best and second-best results excluding vanilla LLM. The last rows report Self-Delegation changes relative to vanilla LLM.

Figure 5: The left panel shows the accuracy and inference cost under three different context configurations provided to the judge. The right panel shows the accuracy and inference cost under five different context configurations passed from the SLM to the LLM.

Figure 6: The left 3 figures shows the performance and cost on different thresholds on binary delegation. The right figure shows the comparison between accuracy and inference cost on DGXSpark and A100.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

	Pass@1	Cost
SFT-Model	57.30%	$ 53.22
Self-Delegation	71.91%	$56.96

Appendix

Table 2: The pass@1 and cost of SFT-model and Self-delegation model.

	Pass@1	Cost
Tmax-9B	28.09%	$ 0
Binary Delegation	74.16%	$62.64
GPT-5.5(xhigh)	74.16%	$90.31

Appendix

Table 6: The pass@1 and cost of using Tmax-9B as SLM of our binary delegation method in the framework.

Figure 7: LLM delegation rate in answer trajectories throughout the training process.

Figure 8: Learning curve during the training. From left to right, the three panels illustrate entropy loss, reward and the advantage of judge tokens.

Harness	Method	Qwen3.5 + GPT-5.5 (xhigh)
Harness	Method	Pass@3	Cost
Terminus-2	Vanilla SLM	29.10	0.00
	Vanilla LLM	82.02	279.73
	Self-Delegation	82.02	181.88
	$\Delta$ vs. LLM	+0.00%	-34.89%
OpenHands	Vanilla SLM	20.22	0.00

Appendix

Table 7: Pass@3 using different harness on Terminal-bench 2.1 .

Figure 9: The critic and SLM agreement with the training steps.

Figure 10: Distribution of step types handled by the LLM on Terminal-Bench 2.1 and SWE-ReBench.

Figure 11: The cloud rate under different thresholds.

Edge-Cloud Collaborative Inference

See all 28 papers

Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution
Oct 13, 2026Pengcheng Huang, Zhenghao Liu, Shixin Ren +5Edge-Cloud Collaborative Inference Edge-Cloud Computing
School of Computer Science and Engineering, Northeastern University, Shenyang, China · Department of Computer Science and Technology, Tsinghua University, Beijing, China · China Mobile Online Services Co., Ltd. Beijing, China
Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents
May 23, 2026Yuxin Zhang, Mengxue Hu, Zheng Lin +8LLM Agent Training LLM Routing
1Fudan University · 2Alibaba Group · 3The University of Hong Kong +4
CoMIC: Collaborative Memory and Insights Circulation for Long-Horizon LLM Agents in Web Cloud-Edge Systems
May 30, 2026Yannan Wang, Longli Yang, Zhen Liu +2LLM Agent Memory Edge Computing
Beijing Jiaotong University · Visiting PhD student at WMG, University of Warwick. · The Alan Turing Institute +2

Oct 13, 2026J/K move · Enter open · S save

Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution

Pengcheng Huang, Zhenghao Liu, Shixin Ren, Shi Yu+4

School of Computer Science and Engineering, Northeastern University, Shenyang, China · Department of Computer Science and Technology, Tsinghua University, Beijing, China · China Mobile Online Services Co., Ltd. Beijing, China

Edge-Cloud Collaborative Inference Edge-Cloud Computing AI Agent Benchmarks LLM Agent Evaluation Long-Horizon Agent Evaluation

Open paper AskPDF arXiv

Edge-cloud collaboration combines the lower inference overhead and greater data locality of edge-side execution with the stronger capabilities of cloud-hosted models. However, as LLM agents increasingly operate in long-horizon scenarios, cloud participation becomes interleaved throughout execution rather than a one-shot decision for each query. Such interleaved participation introduces trajectory-level effects that remain insufficiently understood. To address this gap, we introduce EC2-Bench, a benchmark for evaluating edge-cloud collaboration in long-horizon agent execution. It provides executable digital-workspace agent tasks and a unified evaluation framework covering task performance, resource usage, and privacy exposure. Using this benchmark, we empirically study diverse forms of edge-cloud collaboration. Our study reveals three findings. First, collaboration achieves a favorable balance among task performance, resource usage, and privacy exposure compared with edge-only and cloud-only execution. Second, as edge models become stronger, collaborative execution approaches cloud-only performance, and performance differences across strategies shrink, while privacy exposure remains strongly strategy-dependent. Third, stronger edge capability shifts preferred collaboration from externally controlled cloud participation and direct cloud execution toward more edge-controlled collaboration. Further analyses identify cloud-invocation decision-making and instruction following as important edge-side abilities for effective collaboration. These findings provide a systematic understanding of edge-cloud collaboration in long-horizon agent execution and insights for designing future edge agents that effectively collaborate with cloud models. Code and resources are available at https://github.com/OpenBMB/EC2-Bench.

Abstract

Edge-Cloud Collaborative Inference

Explore similar work

Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution

Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents

CoMIC: Collaborative Memory and Insights Circulation for Long-Horizon LLM Agents in Web Cloud-Edge Systems