Heterogeneous Computing

Follow this topic Ask about this topic

Heterogeneous computing assigns work across different resources—such as CPUs, GPUs, NPUs, FPGAs, and processing-in-memory devices—whose strengths and constraints vary. Common approaches include task and graph scheduling, workload partitioning, pipeline parallelism, compiler and kernel optimization, and joint hardware-runtime design; reinforcement learning and analytical cost models help make these decisions adaptive. Applications range from edge inference and robotics to large-model training and serving across distributed clusters. A central challenge is balancing computation against data movement, contention, and power while preserving accuracy and meeting latency targets, with solutions often depending on the specific hardware and workload.

Latest papers

80 papers13 papers in the last four weeks, up 30% on the four weeks before. 0.1% of all new papers.

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
Apr 10, 2026Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-MartínezEnergy-Aware LLM Inference LLM Inference Efficiency
Universidad Politécnica de Madrid, C. de los Ciruelos, Boadilla del Monte, 28660, Madrid, Spain · Zurich University of Applied Sciences, Switzerland, Gertrudstrasse 15, Winterthur, 8400, Zurich, Switzerland
The Principle of Maximum Heterogeneity Optimises Productivity in Distributed Production Systems Across Biology, Economics, and Computing
Apr 8, 2026Guillhem Artis, Danyal Akarca, Jascha AchterbergMulti-Agent System Optimization Distributed Optimization
Callosum, London, United Kingdom (www.callosum.com) · Imperial College London, London, United Kingdom · University of Cambridge, Cambridge, United Kingdom +1
AP-DRL: A Synergistic Algorithm-Hardware Framework for Automatic Task Partitioning of Deep Reinforcement Learning on Versal ACAP
Mar 31, 2026Enlai Li, Zhe Lin, Sharad Sinha +1Reinforcement Learning Heterogeneous Computing
The Hong Kong University of Science and Technology, Hong Kong SAR, China · Sun Yat-Sen Univeristy, Guangdong, China · Indian Institute of Technology Goa, Goa, India
ScoutAttention: Efficient KV Cache Offloading via Layer-Ahead CPU Pre-computation for LLM Inference
Mar 28, 2026Qiuyang Zhang, Kai Zhou, Ding Tang +5KV-Cache Offloading LLM Inference Acceleration
Huazhong University of Science and Technology, Wuhan, China · Huawei Technologies, Hefei, China · Research Center for High Efficiency Computing Infrastructure, Zhejiang Lab, Hangzhou, China
A Learning Method with Gap-Aware Generation for Heterogeneous DAG Scheduling
Mar 24, 2026Ruisong Zhou, Haijun Zou, Li Zhou +2Neural Combinatorial Optimization RL for Combinatorial Optimization
School of Mathematical Science, Peking University, Beijing 100871, China · State Key Laboratory of Mathematical Sciences, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China · Theory Lab, Central Research Institute, 2012 Labs, Huawei Technologies Co., Ltd, China +1
Modernizing Amdahl's Law: How AI Scaling Laws Shape Computer Architecture
Mar 21, 2026Chien-Ping LuHeterogeneous Computing AI Scaling Laws
An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU
Mar 17, 2026Ruijia Yang, Zeyi WenFine-Tuning Memory-Efficient Fine-Tuning
HKUST(GZ) Guangzhou, China · HKUST Guangzhou, China
ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning
Mar 13, 2026Bangjun Xiao, Yihao Zhao, Xiangwei Deng +9Agentic RL Heterogeneous Computing
School of Computer Science, Peking University · LLM-Core Xiaomi
Robust Heterogeneous Analog-Digital Computing for Mixture-of-Experts Models with Theoretical Generalization Guarantees
Mar 3, 2026Mohammed Nowaz Rabbani Chowdhury, Hsinyu Tsai, Geoffrey W. Burr +3Mixture-of-Experts Inference Heterogeneous Computing
Rensselaer Polytechnic Institute · IBM Research
TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
Mar 1, 2026Yudong Pan, Yintao He, Tianhua Han +7Mixture-of-Experts Inference LLM Inference Acceleration
SKLP, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure
Feb 26, 2026Jaehong Cho, Hyunmin Choi, Guseul Heo +1LLM Serving Disaggregated LLM Serving
School of Computing KAIST Daejeon, South Korea
Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review
Feb 18, 2026Luigi Capogrosso, Pietro Bonazzi, Michele MagnoIn-Sensor Computing Edge AI
Interdisciplinary Transformation University of Austria · ETH Zurich
Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
Feb 3, 2026Grigory Begunov, Alexander TyurinDistributed Optimization Stochastic Gradient Descent
AXXX, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia
KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta
Dec 29, 2025Gang Liao, Hongsen Qin, Ying Wang +36GPU Kernel Optimization GPU Acceleration
KernelEvolve Team, Meta Platforms
DCC: Data-Centric Compilation of Machine Learning Kernels for Processing-In-Memory Architectures
Nov 19, 2025Peiming Yang, Sankeerth Durvasula, Ivan Fernandez +4Compute-in-Memory LLM Inference Acceleration
University of Toronto & Vector Institute · Barcelona Supercomputing Center · ETH Zürich +2
OctoPipe: Reducing Pipeline Bubbles for Heterogeneous Models via Co-Optimizing Partitioning, Placement, and Scheduling
Sep 28, 2025Jihu Guo, Tenghui Ma, Wei Gao +6Communication-Efficient Distributed Training Heterogeneous Computing
FDU & Shanghai AI Laboratory · HKUST · Independent Researcher +2
Metadata-Guided Adaptable Frequency Scaling across Heterogeneous Applications and Devices
Sep 23, 2025Jinqi Yan, Fang He, Qianlong Sang +5Meta-Learning Multi-Task RL
School of Computer Science, Wuhan University · School of Computer Science and Artificial Intelligence, Wuhan University of Technology · OPPO Guangdong Mobile Communications Co., Ltd.
Leveraging Neural Graph Compilers in Machine Learning Research for Edge-Cloud Systems
Apr 28, 2025Alireza Furutanpey, Carmen Walser, Philipp Raith +2Efficient Neural Network Inference Heterogeneous Computing
Coovally Barcelona, Spain · Distributed Systems Group, TU Wien, Vienna · ICREA Barcelona, Spain
Ray Data: A dynamic and scalable system for heterogeneous data processing
Jan 16, 2025Frank Sifei Luan, Ron Yifeng Wang, Yile Gu +12Heterogeneous Computing
UC Berkeley · University of Washington · Anyscale
Speed-Oblivious Online Scheduling: Knowing (Precise) Speeds is not Necessary
Feb 2, 2023Alexander Lindermayr, Nicole Megow, Martin RappCombinatorial Optimization Learning-Augmented Algorithms
University of Bremen, Faculty of Mathematics and Computer Science, Germany · Faculty for Informatics, Karlsruhe Institute of Technology, Germany

Apr 10, 2026J/K move · Enter open · S save

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures

Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-Martínez

Universidad Politécnica de Madrid, C. de los Ciruelos, Boadilla del Monte, 28660, Madrid, Spain · Zurich University of Applied Sciences, Switzerland, Gertrudstrasse 15, Winterthur, 8400, Zurich, Switzerland

Energy-Aware LLM Inference LLM Inference Efficiency Energy-Efficient ML Heterogeneous Computing LLM Evaluation

Open paper AskPDF arXiv

While the large energy consumption of Large Language Models (LLMs) is recognized by the community, system operators lack guidance for energy-efficient LLM inference deployments that leverage energy trade-offs of heterogeneous hardware due to a lack of energy-aware benchmarks and data. In this work we address this gap with Watt Counts: the largest open-access dataset of energy consumption of LLMs, with over 5,000 experiments for 50 LLMs across 10 NVIDIA Graphics Processing Units (GPUs) in batch and server scenarios along with a reproducible, open-source benchmark that enables community submissions to expand this dataset. Leveraging this dataset, we conduct a system-level study of LLM inference across heterogeneous GPU architectures and show that GPU selection is crucial for energy efficiency outcomes and that optimal hardware choices vary significantly across models and deployment scenarios, demonstrating the critical importance of hardware-aware deployment in heterogeneous LLM systems. Guided by our data and insights, we show that practitioners can reduce energy consumption by up to 70% in server scenarios with negligible impact on user experience, and by up to 20% in batch scenarios.