cs.ARMar 6, 2026

LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis

Authors: Tao Zhang, Rui Ma, Shuotao Xu, Yongqiang Xiong, Peng Cheng

Organizations: Microsoft Research China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

GPU design space exploration (DSE) for modern AI workloads, such as Large-Language Model (LLM) inference, is challenging because of GPUs' vast, multi-modal design spaces, high simulation costs, and complex design optimization objectives (e.g. performance, power and area trade-offs). Existing automated DSE methods are often prohibitively expensive, either requiring an excessive number of exploration samples or depending on intricate, manually crafted analyses of interdependent critical paths guided by human heuristics. We present LUMINA, an LLM-driven GPU architecture exploration framework that leverage AI to enhance the DSE efficiency and efficacy for GPUs. LUMINA extracts architectural knowledge from simulator code and performs sensitivity studies to automatically compose DSE rules,which are auto-corrected during exploration. A core component of LUMINA is a DSE Benchmark that comprehensively evaluates and enhances LLMs' capabilities across three fundamental skills required for architecture optimization, which provides a principled and reproducible basis for model selection and ensuring consistent architectural reasoning. In the design space with 4.7 million possible samples, LUMINA identifies 6 designs of better performance and area than an A100 GPU efficiently, using only 20 steps via LLM-assisted bottleneck analysis. In comparison, LUMINA achieves 17.5x higher than design space exploration efficiency, and 32.9% better designs (i.e. Pareto Hypervolume) than Machine-Learning baselines, showcasing its ability to deliver high-quality design guidance with minimal search cost.

Figures & tables

Figure 1. Design space visulizatin for a GPT3-175B ( Brown et al., 2020 ) inference workload via roofline model ( Williams et al., 2009 ) . Each sampled architecture is embedded via Principal Component Analysis ( Gewers et al., 2018 ) into two dimensions; distributions are capped for visual contrast.
ParameterValue Range#
Interconnect Link Count6, 12, 18, 244
Core Count1, 2, 4, 8, 16, 32, 64, 96, 108, 128, 132, 136, 140, 25614
Sublane Count1, 2, 4, 84
Systolic Array Height and Width4, 8, 16, 32, 64, 1286
Vector Width4, 8, 16, 32, 64, 1286
SRAM Size (KB)32, 64, 128, 192, 256, 512, 10247
Table 1. Example Design Space of a 8 GPU Node
CategoryRepresentative MethodSample LearningSample EfficiencySample Scalibity
HeuristicGrid Search ( Bergstra and Bengio, 2012 )NoRandomHigh
Random Walker ( Xia et al., 2020 )NoRandomHigh
Machine LearningBayesian Optimization (BO) ( Li et al., 2025 )YesLow ( Krishnan et al., 2023 )Low ( Maddox et al., 2021 )
Genetic Algorithms (GA) ( Kao and Krishna, 2020 )YesLow ( Krishnan et al., 2023 )Low ( Martínez-Ballesteros et al., 2015 )
Ant Colony Optimization (ACO) ( Gao and Schafer, 2021 )YesLow ( Krishnan et al., 2023 )Low ( Peake et al., 2019 )
Expertise-DrivenBottleneck-Removal via Critical Path Analysis ( Bai et al., 2023 )NoHighLow
Table 2. Comparison of representative DSE methods.
Figure 2. Lumina Framework Design
Figure 3. Examples of three task in DSE benchmark. For each task, we show the associated evidences from human system prompt, question on the left and LLM’s correct answer on the right.
Benchmark TaskModelAccuracy (Original)Accuracy (Enhanced)
Bottleneck AnalysisPhi-40.700.76
Qwen-30.730.80
Llama-3.10.470.53
Perf/Area PredictionPhi-40.420.61
Qwen-30.590.82
Llama-3.10.230.39
Table 3. Accuracy across tasks and open-source LLMs
Figure 4. Mean PHV vs. Sample Efficiency among DSE Methods.
Figure 5. Distribution of PHV vs. Sample Efficiency among DSE Methods.
Figure 6. Search Pattern Comparison between ACO and Lumina
SpecificationsDesign ADesign BA100
Interconnect Link Count241812
Core Count6496108
Sublane Count444
Vector Width161632
Systolic Array Height ×\times Weight32 ×\times 3232 ×\times 3216 ×\times 16
SRAM Size (KB)128128128
Table 4. Top-2 designs identified by Lumina Compared with NVIDIA A100
More in this topic

Automated Heuristic Design

Automated heuristic design creates or adapts problem-solving rules for optimization, planning, scheduling, and related search tasks.

See all 84 papers

Explore similar work

CardsList
Oct 2, 2026

D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?

GPU kernels generated by large language model (LLM) agents can remain less efficient than expert implementations, but runtime alone does not reveal how the gap relates to design discovery and implementation. We introduce D2K-Bench, a diagnostic benchmark of 26 tasks and 85 workloads that measures how effectively agents translate expert design guidance into efficient GPU kernels. The guidance covers L1: high-level algorithmic insights, L2: dataflow design, and L3: low-level optimization tricks, including dependencies among these levels. Pairwise runs with and without guidance share task descriptions, workloads, tools, hardware, and a 350-turn budget. Complementary assessments examine independently proposed designs and the design properties implemented in generated code. Across five models on NVIDIA B200 GPUs, guidance raises correctness over 130 model-task pairs from 93.1% to 98.5% and increases the Performance Score over all 26 tasks from 1.46 to 1.95. For the three frontier models with correct submissions on all 26 tasks in both runs (GPT-6-Astra, Claude-Opus-4.8, and GPT-5.6-Sol), geometric mean speedup increases from 1.69×1.69\times to 2.49×2.49\times. Across all five models, the mean combined implementation score increases from 57 to 70 out of 100. These results show the value of expert design guidance while identifying design properties that remain unimplemented.

Daifeng Li, Huiqiang Jiang, Chengruidong Zhang +6

HKUST · Alibaba Group · USTC

GPU Kernel GenerationGPU Kernel OptimizationLarge Language Model-Guided OptimizationAI Agent Benchmarks
AskPDF
Jul 20, 2026

Harness Engineering for LLM-Driven GPU Kernel Generation

Large language models (LLMs) can assist GPU kernel generation, but their practical effectiveness depends on whether generated code can be reliably constrained, validated, profiled, and selected. This paper presents a harness-centered system for LLM-driven GPU kernel optimization in the MLSys 2026 FlashInfer AI Kernel Generation Contest on NVIDIA Blackwell B200 GPUs. The system separates an evaluation harness from a profile-backed optimization controller: the harness enforces compilation, correctness, official-aligned timing, and artifact archival, while the controller turns profiler and workload evidence into bounded candidate-generation decisions. Human-authored skills capture operator constraints, references, profiling procedures, and promotion rules, while Codex and Claude Code agents generate candidate kernels inside those constraints. Across five operator definitions, the retained official-aligned artifacts achieved mean-latency speedups over supplied FlashInfer baselines of 1.62x, 18.05x, 29.68x, 1.12x, and 13.70x. The Agent-Assisted kernels outperform the Full-Agent artifacts across the evaluated definitions, indicating that expert-provided optimization directions, high-quality references, and workload context remain critical for reliable AI-driven kernel optimization.

Yue Shui, Chenyu Ma, Hangfei Xu +2

Baidu, Inc.

GPU Kernel OptimizationLarge Language Model-Guided OptimizationGPU Kernel Generation
AskPDF
Jun 20, 2026

AgentDSE: Reasoning-Augmented Architectural Design Space Exploration

Traditional architectural design space exploration (DSE) is highly inefficient, typically requiring tens of thousands of simulator evaluations across various optimization methods. This inefficiency arises because conventional methods treat the simulator as a black-box oracle. In contrast, human architects effectively guide exploration by reasoning through physical constraints, performance bottlenecks, data reuse, and workload structures. To bridge this gap, we introduce AgentDSE, a simulator-in-the-loop methodology driven by a general-purpose large language model (LLM) coding agent. AgentDSE automates this architectural-reasoning loop without requiring model fine-tuning, precomputed design databases, or domain-specific optimizer code. Across deep neural network (DNN) accelerator mapping, hardware/software co-design, and CPU cache-hierarchy optimization, AgentDSE achieves competitive or better design quality with up to two orders of magnitude fewer evaluations. AgentDSE also produces inspectable traces that surface architectural hypotheses, performance cliffs, implicit priors, and simulator artifacts, making every search decision traceable rather than buried in optimizer state.

Chenyu Wang, Jiahe Caroline Shi, David Kong +4

Harvard University · Massachusetts Institute of Technology · Columbia University

Large Language Model-Guided OptimizationElectronic Design AutomationHardware-Software Co-Design
AskPDF