cs.DCNov 18, 2026

LEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge Devices

Authors: Zihao Wang, Ye Tian, Tajana Rosing, Nalini Venkatasubramanian

Organizations: University of California, Irvine Irvine, California, USA · University of California, San Diego San Diego, California, USA

Abstract

Evaluating distributed large language model (LLM) serving on heterogeneous edge devices requires exploring diverse model partitions, device placements, and network conditions, making repeated physical experiments costly. An efficient simulator for edge LLM serving must support diverse, fine-grained placement configurations, capture runtime behavior, and accurately predict execution times while keeping simulation overhead low. We present LEDGE, a lightweight simulator for fast, fine-grained evaluation of distributed LLM serving on heterogeneous edge devices without executing tensor operations. LEDGE constructs event graphs supporting non-uniform tensor and pipeline parallelism, estimates execution costs using reusable performance models calibrated from end-to-end measurements, and simulates shared-link contention. We evaluate LEDGE on a heterogeneous testbed of 10 Jetson and Raspberry Pi devices, covering 17 model--precision combinations across 4 LLM families. On Qwen deployments, LEDGE achieves request-level mean absolute percentage errors below 3.6% and a 135.3 times aggregate speedup over physical inference. Performance model calibrated on one LLM family yield 7.44% zero-shot end-to-end latency MAPE across three other families. Further simulations show how shared-link contention affects placement decisions and network topology constrains scaling. Across four edge environments, LEDGE completes 26,773 simulation runs in 83.8 minutes, enabling efficient exploration of deployment strategies under diverse resource conditions and serving objectives. The source code is publicly available at https://github.com/gdfwj/LEDGE.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. E2LLM: Towards Efficient LLM Serving in Heterogeneous Edge/Fog Environments

    Jun 2, 2026Truong-Thanh Le, Amir Taherkordi, Hoang-Loc La +3LLM ServingPrefill-Decode Disaggregation

  2. Frontier: Towards Comprehensive and Accurate LLM Inference Simulation

    May 20, 2026Yicheng Feng, Xin Tan, Yangtao Deng +3LLM ServingDisaggregated LLM Serving

  3. SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference

    Aug 13, 2026Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar HanawalEdge-Cloud Collaborative InferenceEdge Computing