Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution
Authors: Pengcheng Huang, Zhenghao Liu, Shixin Ren, Shi Yu, Sijia Yao, Yukun Yan, Chaojun Xiao, Maosong Sun
Organizations: School of Computer Science and Engineering, Northeastern University, Shenyang, China · Department of Computer Science and Technology, Tsinghua University, Beijing, China · China Mobile Online Services Co., Ltd. Beijing, China
Edge-cloud collaboration combines the lower inference overhead and greater data locality of edge-side execution with the stronger capabilities of cloud-hosted models. However, as LLM agents increasingly operate in long-horizon scenarios, cloud participation becomes interleaved throughout execution rather than a one-shot decision for each query. Such interleaved participation introduces trajectory-level effects that remain insufficiently understood. To address this gap, we introduce EC2-Bench, a benchmark for evaluating edge-cloud collaboration in long-horizon agent execution. It provides executable digital-workspace agent tasks and a unified evaluation framework covering task performance, resource usage, and privacy exposure. Using this benchmark, we empirically study diverse forms of edge-cloud collaboration. Our study reveals three findings. First, collaboration achieves a favorable balance among task performance, resource usage, and privacy exposure compared with edge-only and cloud-only execution. Second, as edge models become stronger, collaborative execution approaches cloud-only performance, and performance differences across strategies shrink, while privacy exposure remains strongly strategy-dependent. Third, stronger edge capability shifts preferred collaboration from externally controlled cloud participation and direct cloud execution toward more edge-controlled collaboration. Further analyses identify cloud-invocation decision-making and instruction following as important edge-side abilities for effective collaboration. These findings provide a systematic understanding of edge-cloud collaboration in long-horizon agent execution and insights for designing future edge agents that effectively collaborate with cloud models. Code and resources are available at https://github.com/OpenBMB/EC2-Bench.
Figures & tables
Figure 1: Overview of edge–cloud collaboration for long-horizon agent execution in EC 2 -Bench . The figure illustrates a representative agent workflow, four collaboration modes with different forms of cloud participation, and three research questions on collaboration effectiveness, the impact of edge capability, and how collaboration shifts as edge models become stronger.
Benchmark
Evaluation Scenario
Collab. Modes
Evaluation Dim.
RouterBench ( Hu et al., 2024a )
General LLM Tasks
Rq
Task Perf. / Cost
LLMRouterBench ( Li et al., 2026b )
General LLM and Tool-Use Tasks
Rq
Task Perf. / Cost
LLMSwitchBench ( Liu et al., 2025 )
Single-turn Query
Rq
Task Perf. / Cost
M4A1 ( Yuan et al., 2026b )
Multi-turn Dialogue
Rs
Task Perf. / Cost / Latency
EC 2 -Bench
Long-horizon Agent Execution
Rq , Rs , D , A
Task Perf. / Resource / Privacy
Table 1: Comparison of EC 2 -Bench with prior benchmarks for edge–cloud collaboration and LLM routing. Evaluation Scenario specifies the task and interaction setting considered by each benchmark. For Collab. Modes , Rq and Rs denote query-level and per-step routing, respectively, while D and A denote edge-controlled cloud delegation and cloud advisory, respectively. Evaluation Dim. summarizes the primary evaluation metrics considered by each benchmark.
Method
Performance ↑
Resource Usage ↓
Privacy Exp. ↓
Completion
Pass 3
Cloud Tok. (R/C/O)
Cloud Cost
Edge FLOPs
Edge-only
Qwen3.5-2B ( 2026a )
12.75
2.34
–
–
142.98
0.00
Qwen3.5-9B ( 2026a )
39.12
9.38
–
–
121.64
0.00
Qwen3.5-27B ( 2026a )
61.79
30.47
–
–
564.49
0.00
Qwen3.7-Plus ( 2026a )
71.34
53.12
–
–
–
0.00
Table 2: Performance of edge–cloud collaboration strategies on EC 2 -Bench . Performance : task completion and Pass 3 . Resource Usage : Cloud Tok. (R/C/O) reports raw input, cache-read input, and output tokens in millions; Cloud Cost is reported in USD; Edge FLOPs are reported in PetaFLOPs. Privacy Exposure : percentage of sensitive information exposed to the cloud (lower is better). and denote current-generation and higher-capability proxy edge models, respectively. For Qwen3.7-Plus, Edge FLOPs are not reported because its parameter count is unavailable.
Figure 2: Trade-offs of edge–cloud execution modes. and denote Qwen3.5-2B and Qwen3.5-9B. Each circle represents a collaboration mode; upper-left is better. Dashed lines interpolate between edge-only and cloud-only execution, with points above the lines indicating gains beyond the interpolation baseline.
Figure 3: Task completion and privacy exposure across edge-capability regimes. Blue points denote the mean values across edge–cloud collaboration configurations, and vertical bars indicate the range across configurations. The x-axis labels abbreviate the corresponding edge models (Qwen3.5-2B/9B/27B and Qwen3.7-Plus).
Figure 4: Analysis of edge-side capabilities for increasingly edge-led collaboration. (a) Task completion under agent-controlled invocation and a matched random baseline with the same cloud-call rate. (b) EC 2 -Bench task completion of the edge model under cloud advisory during instruction-following training.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Task Source
# Tasks
Percentage
Claw-Eval
68
53.12%
WildClawBench
22
17.19%
QwenClawBench
21
16.41%
LiveClawBench
4
3.12%
PinchBench
3
2.34%
ClawBench
3
2.34%
Appendix
Table 3: Source distribution of the EC 2 -Bench .
Task Category
# Tasks
Office & Daily Tasks
36
Information Search & Gathering
34
Safety & Security
21
Development & Operations
14
Data Analysis
13
Automation
10
Appendix
Table 4: Distribution of EC 2 -Bench tasks across agentic operation categories.
Component
Representative Example
Task Specification
Task Instruction
Identify active customers who have not been contacted for more than 30 days, review their previous email communications, and draft follow-up emails with different styles for VIP and standard customers while excluding churned customers.
Expected Behavior
Query the CRM to identify overdue active customers, exclude churned customers, retrieve relevant email records to recover previous communication context, and prepare personalized follow-up drafts according to customer tier and communication history.
Evaluation
Evaluation Checks
Customer identification (0.35), differentiation between VIP and standard follow-up styles (0.35), and follow-up draft quality (0.30). The weighted check scores are aggregated into the task completion score.
Sensitivity Annotation
sensitive_items.json contains fine-grained sensitivity units associated with sensitive information in the task environment and is used for privacy-exposure evaluation.
Appendix
Table 5: Representative standardized task instance in EC 2 -Bench . The example illustrates the task specification, evaluation criteria, and executable environment used during agent execution. For readability, some fields are shortened and reformatted for presentation while preserving their original semantics.
Domain
Category
# Units
Percentage
# Tasks
Personal Sensitive Information
P1 Identity identifier
276
8.15%
48
P2 Documents and accounts
457
13.50%
66
P3 Contact information
1178
34.80%
84
P4 Financial information
209
6.17%
48
P5 Sensitive attributes
248
7.33%
26
Subtotal
2368
69.96%
–
Appendix
Table 6: Statistics of sensitivity-unit annotations by privacy domain and fine-grained category. # Tasks denotes the number of tasks containing at least one unit from that category.
Type
Before
After
Identified Issue
Phone number
(415) 555-0298
(415) 672-3845
Common fictional-number pattern.
Passport ID
E12345678
E47291856
Obvious sequential placeholder pattern.
Cloud access key
AKIA...EXAMPLE
AKIA...Z1R6
Contains a recognizable example token.
Password
password
xK9#...Q4wR
Common default or placeholder value.
Appendix
Table 7: Representative revisions of synthetic sensitivity units by the verifier. Values are shortened where necessary for readability, and all examples are synthetic.
Agreement
Cohen’s κ
N
Human annotators
97.5%
0.945
200
Precision
Recall
F1
GPT-5.4-mini
97.7%
96.2%
96.9%
Appendix
Table 8: Validation results for privacy-exposure detection.
Model
Valid
Invalid
Precision
GLM-5.3
45
5
90.0%
Appendix
Table 9: Human validation of identified sensitivity units.
Metric
Content
Placement
Mean rating
4.62
4.59
Ratings ≥4
86.3%
88.3%
Gwet’s AC2
0.91
0.90
Appendix
Table 10: Human evaluation of 150 synthetic sensitivity units sampled from 50 tasks. Two annotators independently assess content realism and placement naturalness on a five-point Likert scale.
Method
Performance ↑
Resource Usage ↓
Privacy Exp. ↓
Completion
Pass 3
Cloud Tok. (R/C/O)
Cloud Cost
Edge FLOPs
Edge-only
Qwen3.5-2B ( 2026a )
12.75
2.34
–
–
142.98
0.00
Qwen3.5-9B ( 2026a )
39.12
9.38
–
–
121.64
0.00
Qwen3.5-27B ( 2026a )
61.79
30.47
–
–
564.49
0.00
Qwen3.7-Plus ( 2026a )
71.34
53.12
–
–
–
0.00
Appendix
Table 11: Performance of edge–cloud collaboration strategies on EC 2 -Bench . Performance : task completion and Pass 3 . Resource Usage : Cloud Tok. (R/C/O) reports raw input, cache-read input, and output tokens in millions; Cloud Cost is reported in USD; Edge FLOPs are reported in PetaFLOPs. Privacy Exposure : percentage of sensitive information exposed to the cloud (lower is better). and denote current-generation and higher-capability proxy edge models, respectively. For Qwen3.7-Plus, Edge FLOPs are not reported because its parameter count is unavailable.
Figure 5: Training dynamics of instruction-following ability and edge-side task performance. (a) Edge-only task performance on EC 2 -Bench . (b)–(d) Instruction-following performance on three instruction-following benchmarks.
Cloud Model
Input
Cache Input
Output
DeepSeek-v4-Pro
1.74
0.0145
3.48
DeepSeek-v4.1-Flash
0.15
0.003
0.6
GLM-5.3
1.4
0.26
4.40
Appendix
Table 12: Cloud-side pricing used for monetary cost estimation. Prices are reported in USD per million tokens.
Complex and long-horizon agent tasks often rely on frontier models deployed in the cloud to achieve strong performance. However, using these models for every step of a long-horizon trajectory incurs substantial inference cost, even though not all steps may require the same level of model capability. In this paper, we find that allocating the LLM to only a small subset of key steps can achieve performance comparable to full LLM execution while significantly reducing inference cost. Motivated by this observation, we propose EdgeWise, an edge-cloud collaboration framework that deploys an SLM on the local device as the primary solver and selectively delegates critical steps to a cloud LLM when stronger model capacity is needed. To enable autonomous delegation without additional models at inference time, we introduce a step-level RL method that jointly optimizes task solving and delegation, with fine-grained credit assignment for individual delegation decisions. We also introduce an external judge-based delegation strategy and use it to systematically investigate key design choices in edge-cloud collaboration, including when to delegate, how delegation decisions should be made, what context should be shared, and how frequently the cloud LLM should be invoked. Experiments show that EdgeWise matches or outperforms pure-LLM execution while reducing inference cost up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1. We further deploy EdgeWise on a local workstation, demonstrating comparable cost savings in a practical edge-cloud setting.
LLM-based Web agents are increasingly deployed on edge servers to carry out long-horizon tasks on behalf of users with low latency. However, the lightweight models that fit on edge servers often struggle with long-horizon tasks that require persistent memory, subgoal tracking, and reflection. Fine-tuning edge models after deployment is costly and difficult to scale across heterogeneous nodes, while purely local execution confines each agent to its own isolated experience. We propose CoMIC, a parameter-update-free cloud-edge framework for Collaborative Memory and Insights Circulation. Following the paradigm of ``Act at the Edge, Reflect in the Cloud'', edge agents execute locally with subgoal-oriented hierarchical memory and selective re-expansion of relevant histories, while a cloud-side LLM critic asynchronously evaluates the trajectories uploaded through a Web gateway, admits reusable experience, and circulates cross-agent guidance indexed by semantic subgoal identifiers back to all edge agents. On five long-horizon benchmarks, CoMIC enables weak edge agents to achieve non-zero success on tasks where a standard memory-based agent fails entirely, and improves their task progress over both the standard agent and purely local execution while reducing context consumption relative to the standard agent, all without updating model parameters. When stronger heterogeneous agents join the edge, the weak agents further improve in success, progress, and grounding with lower context consumption.
Yannan Wang, Longli Yang, Zhen Liu +2
Beijing Jiaotong University · Visiting PhD student at WMG, University of Warwick. · The Alan Turing Institute +2
Large language model (LLM) agents excel at solving complex long-horizon tasks through autonomous interaction with environments. However, their real-world deployment faces a fundamental device--cloud dilemma: on-device models are efficient but often brittle, while cloud models are stronger but costly in computation. State-of-the-art LLM device--cloud routers usually make coarse task-level decisions, which cannot adapt to the changing difficulty of multi-step agent interactions. To address this issue, we present Hera, a step-level device--cloud LLM agent coordinator for long-horizon tasks achieving a strong performance--cost Pareto frontier. Hera adopts a novel two-stage training paradigm: (1) imitation learning for cold-start, followed by (2) reinforcement learning that jointly optimizes task success and cloud usage efficiency. The first stage casts step-level routing as a supervised classification problem: the device agent is replayed on cloud trajectories, with each state labeled by the agreement between device and cloud actions. In the second stage, we perform cost-aware reinforcement learning by grouping identical states across trajectories and updating Hera with labels favoring higher expected return and fewer future cloud calls. We evaluate Hera on ALFWorld, WebShop, and AppWorld, where it consistently outperforms prior methods, achieving 92.5% of the cloud-only success rate with cloud use in only 46.3% of steps.
Yuxin Zhang, Mengxue Hu, Zheng Lin +8
1Fudan University · 2Alibaba Group · 3The University of Hong Kong +4