cs.DCNov 11, 2026

Starscream: Reimagining LLM Sharding on AMD Instinct GPUs with GPU Partitioning

Authors: Sumanth Gudaparthi, Smeet Chheda, Ganesh Dasika

Organizations: AMD Research and Advanced Development

Abstract

As GPU accelerators grow larger to meet rising compute demands, they exhibit increasingly pronounced non-uniform memory access (NUMA) characteristics that challenge existing LLM frameworks. Chiplet-based architectures provide a practical path to larger GPUs while mitigating manufacturing yield constraints. Modern GPUs such as AMD Instinct MI300X employs such a multi-die design, and further offer GPU partitioning capabilities, partitioning a single physical GPU into multiple logical devices with NUMA-aware memory domains, promising faster kernel launches, and reduced memory access latencies with improved locality. However, current parallelization strategies, designed for homogeneous GPU topologies, when applied under partitioning, extend its unidimensional sharding causing significant performance degradation. Three bottlenecks emerge: (1) memory redundancy; (2) arithmetic intensity collapse; (3) communication overhead from flat collectives. We present Starscream, a rethinking of LLM sharding for the chiplet era. Rather than stretching inter-GPU sharding thinner, Starscream adds intra-GPU sharding along an orthogonal dimension, grounded in three foundational principles: kernel shape preservation, memory redundancy elimination, and communication overhead minimization. These principles form the basis for extending any sharding strategy into the partitioning regime. We demonstrate their effectiveness using tensor parallelism as a case study. On AMD MI300X and MI355X, exploiting partitioning capabilities yields up to 4x speedup for attention and 1.43x for fused GEMMs compared to chiplet-oblivious baselines, demonstrating the effectiveness of chiplet-aware designs. Starscream's principles generalize across sharding strategies and future architectures.

Figures & tables

Explore similar work

CardsList
  1. An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU

    Mar 17, 2026Ruijia Yang, Zeyi WenFine-TuningMemory-Efficient Fine-Tuning

  2. Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets

    Aug 19, 2026Tate Berenbaum, Muthaiah VenkatachalamLLM InferenceOn-Device Language Model Inference

  3. Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs

    Nov 3, 2026Gengyu Rao, Ghadeer Ahmed H Alabandi, Su Ann Chong +28LLM TrainingGPU Kernel Optimization