Heterogeneous Computing

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 42

All topics

Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs
May 5, 2026Yixuan Mei, Zikun Li, Zixuan Chen +5LLM Serving Cost-Aware Inference
Carnegie Mellon University · Peking University
CP $^2$ N $^2$ : A Control Plane for Heterogeneous Physical Neural Networks
May 5, 2026Stefan Fischer, Maliheh Hariri, Sebastian OtteHeterogeneous Computing
University of Luebeck, Institute of Telematics · University of Luebeck, Institute of Robotics and Cognitive Systems
ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Device
May 5, 2026Mergen Nachin, Digant Desai, Sicheng Stephen Jia +36Heterogeneous Computing Edge ML
Meta · Work done while at Meta.
VUDA: Breaking CUDA-Vulkan Isolation for Spatial Sharing of Compute and Graphics on the Same GPU
May 2, 2026Bin Xu, Pengfei Hu, Wenxin Zheng +2GPU Acceleration Heterogeneous Computing
Shanghai Jiao Tong University Shanghai, China
DPU or GPU for Accelerating Neural Networks Inference -- Why not both? Split CNN Inference
Apr 30, 2026Ali Emre Oztas, Mahir Demir, James Garside +1AI Accelerator Inference GPU Acceleration
The University of Manchester
Kernel Contracts: A Specification Language for ML Kernel Correctness Across Heterogeneous Silicon
Apr 23, 2026Cooper VeitFormal Verification Heterogeneous Computing
Ashiba Research
Tessera: Unlocking Heterogeneous GPUs through Kernel-Granularity Disaggregation
Apr 11, 2026Tiancheng Hu, Jin Qin, Zheng Wang +10GPU Acceleration LLM Inference Acceleration
SCS, Peking University, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · University of Leeds, West Yorkshire, England +4
An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU
Mar 17, 2026Ruijia Yang, Zeyi WenFine-Tuning Memory-Efficient Fine-Tuning
KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta
Dec 29, 2025Gang Liao, Hongsen Qin, Ying Wang +36GPU Kernel Optimization GPU Acceleration
KernelEvolve Team, Meta Platforms
DCC: Data-Centric Compilation of Machine Learning Kernels for Processing-In-Memory Architectures
Nov 19, 2025Peiming Yang, Sankeerth Durvasula, Ivan Fernandez +4Compute-in-Memory LLM Inference Acceleration
University of Toronto & Vector Institute · Barcelona Supercomputing Center · ETH Zürich +2
OctoPipe: Reducing Pipeline Bubbles for Heterogeneous Models via Co-Optimizing Partitioning, Placement, and Scheduling
Sep 28, 2025Jihu Guo, Tenghui Ma, Wei Gao +6Communication-Efficient Distributed Training Heterogeneous Computing
FDU & Shanghai AI Laboratory · HKUST · Independent Researcher +2
Leveraging Neural Graph Compilers in Machine Learning Research for Edge-Cloud Systems
Apr 28, 2025Alireza Furutanpey, Carmen Walser, Philipp Raith +2Efficient Neural Network Inference Heterogeneous Computing
Coovally Barcelona, Spain · Distributed Systems Group, TU Wien, Vienna · ICREA Barcelona, Spain

May 5, 2026, cs.DCJ/K move · Enter open · S save

Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs

Yixuan Mei, Zikun Li, Zixuan Chen, Shiqi Pan+4

Carnegie Mellon University · Peking University

LLM Serving Cost-Aware Inference Cloud Computing Heterogeneous Computing Online Resource Allocation

Open paper AskPDF arXiv

The usage of large language models (LLMs) has grown increasingly fragmented, with no single model dominating. Meanwhile, cloud providers offer a wide range of mid-tier and older-generation GPUs that enjoy better availability and deliver comparable performance per dollar to top-tier hardware. To efficiently harness these heterogeneous resources for serving multiple LLMs concurrently, we introduce Coral, an adaptive heterogeneity-aware multi-LLM serving system. The key idea behind Coral is to jointly optimize resource allocation and the serving strategy of each model replica across all models. To keep pace with shifting throughput demand and resource availability, Coral applies a lossless two-stage decomposition that preserves joint optimality while cutting online solve time from hours to tens of seconds. Our evaluation across 6 models and 20 GPU configurations shows that Coral reduces serving cost by up to 2.79

\times

over the best baseline, and delivers up to 2.39

\times

higher goodput under scarce resource availability.

Momentum

Related topics

Latest papers 42