Overview

Distributed inference studies how to divide or coordinate model execution across GPUs, edge devices, robots, sensors, and cloud services when one machine is insufficient or unsuitable. Common approaches include pipeline and tensor parallelism, mixture-of-experts placement, edge–cloud partitioning, federated or collaborative prediction, and communication-aware scheduling of requests and cached state. Systems are increasingly designed for heterogeneous hardware and changing network conditions, with simulation and real-cluster benchmarks used to assess latency, throughput, accuracy, and resource use. The central tension is that distributing computation can add communication, coordination, and reliability costs, so balancing speed and scale against privacy, consistency, and robustness remains an open challenge.

Latest papers

CardsList

81 papers8 papers in the last four weeks, up 14% on the four weeks before. 0.1% of all new papers.

  1. Multi-Node Transformers for Object Detection and Captioning Across Distributed Edge Networks

    Nov 8, 2026Saurabh Kumar, Dewant Katare, Prakriti JainObject DetectionImage Captioning

  2. A Compute Center Anyone Can Join: A Peer-to-Peer System for AI Inference

    Nov 1, 2026Yimo Lin, Zhen ZhangDistributed InferenceVerifiable Outsourced Computation

  3. STLT: A Novel System for Energy-Aware Distributed Inference in the Edge-Cloud-Space Continuum

    Oct 31, 2026Andrija Stanisic, Milos Gravara, Juan Luis Herrera +1Distributed InferenceSplit Inference

  4. ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference

    Oct 30, 2026Hung-Chun Lin, Ting-Wei Hsu, Chung-En Ho +1KV-Cache TransferDisaggregated LLM Serving

  5. InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area Network

    Oct 23, 2026Juan Parra-Ullauri, Anthony Chiu, Hari Madhukumar +6LLM InferenceLLM Inference Serving

  6. Consensus-Based Distributed Filtering for Angles-Only Tracking in Cislunar Space

    Oct 22, 2026Giovanni Romagnoli, Fabrizio Pilone, Giordana BucchioniState TrackingDistributed Inference

  7. Deploying RDMA for Distributed Inference and Training At Scale

    Oct 8, 2026Yibo Huang, Yi Cui, Yukai Huang +12Communication-Efficient Distributed TrainingDistributed Inference

  8. Cascadia: A Control-Plane-Free Alternative to Hyperconverged AI Infrastructure

    Sep 30, 2026Matias Parij, Pawan Paudel, Tate Berenbaum +1LLM ServingLLM Inference Scheduling

  9. Purlin: Separating Orchestration from the Datapath of Collectives

    Sep 29, 2026Osayamen Jonathan Aimuyo, Swapnil Gandhi, Christos KozyrakisLLM Inference EfficiencyHigh-Performance Computing

  10. Reliability Testing of Medical Model Performance under Distributed Deployment

    Sep 29, 2026Yifei Wang, Xiaohan Zhang, Youtao Ding +4Multimodal Model EvaluationDistributed Inference

  11. Kafila: Serving Large Language Models on a Trusted Set of Heterogeneous Commodity Machines

    Sep 28, 2026Murtaza Rangwala, Richard O. Sinnott, Rajkumar BuyyaLLM InferenceLLM Inference Scheduling

  12. GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs

    Sep 27, 2026Guang Yang, Fengchen LiuGenomic Foundation ModelsDistributed Inference

  13. End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services

    Sep 15, 2026Zhen Li, Jun Cai, Haoran Gao +2LLM Inference SchedulingLLM Request Scheduling

  14. Recovering Governing Dynamics from Distributed Observations via Exact Spline Merging

    Sep 15, 2026Naveen MysorePDE DiscoveryDistributed Inference

  15. MANE: A Multi-Path Adaptive Network for Edge Onloading of Deep Neural Networks

    Sep 13, 2026Sokratis Nikolaidis, Stylianos I. Venieris, Leonidas Malachias +1Edge InferenceDistributed Inference

  16. mKernel: Fast Multi-GPU, Multi-Node Fused Kernels

    Sep 11, 2026Ziming Mao, Yihan Zhang, Shawn Wei Chew +5GPU Kernel OptimizationCommunication-Efficient Distributed Training

  17. Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving

    Sep 9, 2026Hongjian Fan, Kevin Zhang, David Habinsky +1LLM Inference EfficiencyKV Caching

  18. A Fundamental Limit in Decentralized Decision-Making

    Sep 7, 2026Marco Carpentiero, Felice Scala, Vincenzo Matta +1ClassificationDistributed Inference

  19. Manifold-Aware General Coded Computing for Straggler-Resilient Distributed Computing

    Sep 1, 2026Parsa Moradi, Mohammad Ali Maddah-AliDistributed Inference

  20. Decoupled I/O-Dominant Pipelines for Large-Scale Whole-Slide Image Embedding Extraction

    Aug 27, 2026Mayanka Chandrashekar, Xi Zhang, Ethan Seefried +3Whole-Slide Image AnalysisDistributed Inference

  21. User-Assisted Collaborative Distributed Inference for Efficient QoS-Aware Autoscaling

    Aug 12, 2026Alfreds Lapkovskis, Ali Beikmohammadi, Sindri Magnússon +1Distributed InferenceOnline Resource Allocation

  22. HetRoute Heterogeneous and Cost-aware Collaborative Routing Framework for Distributed Edge MoE Inference

    Aug 1, 2026Xin Yuan, Ning Li, Wenchao Xu +2Edge InferenceCost-Aware Inference

  23. Denial of Deadline: Network-Driven Accuracy Collapse in Distributed Inference Pipelines

    Jul 27, 2026Jhonatan Tavori, Gur-Eyal Sela, Ion Stoica +1Adversarial AttacksEdge Computing

  24. Gleam: Adaptive Network-Efficient CUDA API Remoting for Cross-Device GPU Sharing over LANs

    Jul 25, 2026Zhihao Xu, Hao Zhong, Zeting Zhou +9Heterogeneous ComputingDistributed Inference

  25. Integrity of peer-to-peer distributed LLM inference under malicious nodes

    Jul 21, 2026Mert Cihangiroglu, Antonino NoceraLLM SecurityDistributed Inference