GPU architectural simulation is orders of magnitude slower than native execution, necessitating workload sampling for practical speedups. Existing methods rely on hand-crafted features with limited expressiveness, yielding either aggressive sampling with high errors or conservative sampling with constrained speedups. To address these issues, we propose GCL-Sampler, a sampling framework that leverages Relational Graph Convolutional Networks with contrastive learning to automatically discover high-dimensional kernel similarities from trace graphs. By encoding instruction sequences and data dependencies into graph embeddings, GCL-Sampler captures rich structural and semantic properties of program execution, enabling both high fidelity and substantial speedup. Evaluations on extensive benchmarks show that GCL-Sampler achieves 258.94x average speedup against full workload with 0.37% error, outperforming state-of-the-art methods, PKA (129.23x, 20.90%), Sieve (94.90x, 4.10%) and STEM+ROOT (56.57x, 0.38%).
Figures & tables
Figure 1. GCL-Sampler achieves near-ideal performance (red star) with minimal error and maximum speedup compared to existing methods.
Figure 2. Overview of GCL-Sampler. GCL-Sampler transforms program traces into HRGs and leverages RGCN-based contrastive learning to generate kernel embeddings, followed by K-Means clustering.
Component
Description
tbx , tby , tbz
CTA coordinates in the grid
warp_id
Warp identifier within the CTA
PC
Program counter
mask
32-bit active lane mask
#dests
Number of destination registers
[dest_regs]
Destination register identifiers
Table 1. Trace components for each instruction.
Figure 3. An example of graph construction from traces.
CPU (Intel)
GPU
Architecture
P1
E5-2680 v4
RTX 2080Ti
Turing
P2
i9-12900KF
RTX 3080Ti
Ampere
P3
W5-3435X
RTX 4090
Ada Lovelace
Table 2. Hardware configurations for experiments.
Figure 4. Sampling error of GCL-Sampler, PKA, Sieve and STEM+ROOT. A woman and a girl in white dresses sit in an open car.
Figure 5. Speedup of GCL-Sampler, PKA, Sieve and STEM+ROOT. A woman and a girl in white dresses sit in an open car.
Program
P1
P2
P3
E (%)
SU
E (%)
SU
E (%)
SU
bfs
1.07
3.04 ×
<0.01
3.17 ×
0.05
3.51 ×
lu
0.92
2224.67 ×
0.11
1644.30 ×
<0.01
1658.56 ×
phi-2
0.63
18.41 ×
10.44
13.42 ×
9.74
14.01 ×
qwen1.5
0.38
5.19 ×
2.20
3.76 ×
1.93
3.81 ×
3mm
0.71
3.05 ×
1.61
3.00 ×
0.67
3.03 ×
Table 3. Sampling error and speedup across GPU microarchitecture. P1 denotes Turing, P2 denotes Ampere, and P3 denotes Ada Lovelace. E(%)=Error(in %), SU=Speedup.
Figure 6. Microarchitectural metrics comparison between sampled and full simulation for cfd and pythia workloads.
GPU kernel optimization challenges LLMs beyond standard coding tasks, as it requires an understanding of hardware architecture, parallel computing optimization strategies, and profiling outputs. However, most existing approaches leveraging LLMs for kernel generation apply standard prompting and feedback loops, considering hardware only through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the space of GPU kernels through (1) MAP-Elites quality diversity search with kernel-specific behavioral dimensions to sustain exploration; (2) meta-prompt evolution that co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) a template-based parameter optimization approach to tune kernels to inputs and hardware. We evaluate this framework on Kernel-Bench, robust-kbench and custom tasks, generating SYCL kernels as a cross-platform GPU programming paradigm, and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods and achieves an average speedup of 2.3 on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, allowing quick benchmarking and featuring a flexible user input layer to support kernel generation for a wide range of real use cases beyond benchmarking.
Nina Wiedemann, Quentin Leboutet, Michael Paulitsch +2
High-performance GPU kernels underpin modern deep learning and scientific computing. As workloads become increasingly diverse and GPU hardware evolves rapidly, developing efficient methods for automated GPU kernel generation and optimization has become increasingly important. Existing LLM-based methods typically optimize within a fixed implementation space, limiting either optimization flexibility or search efficiency. We propose HIERA, a hierarchical search-space planning framework for GPU kernel optimization. HIERA constructs contract-augmented task specifications, selects an appropriate implementation space across PyTorch operators, CUDA libraries, and custom CUDA kernels, and uses profiling feedback and expert knowledge to guide structured iterative refinement. Experiments on KernelBench across multiple various workload levels and base LLMs show that HIERA delivers stronger overall implementation validity, sample efficiency, and optimization performance than existing training-free methods, while remaining competitive with the training-based CUDA-L1 without additional model training. A case study on a specialized stencil operator from scientific computing further achieves a 1.53× speedup over cuDNN, demonstrating the potentiality of the general framework beyond standard machine-learning workloads.
Jinghao Wang, Qiqi Gu, Chenpeng Wu +3
Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University
As high-performance computing and AI workloads become increasingly dependent on GPUs, maintaining high performance across rapidly evolving hardware generations has become a major challenge. Developers often spend months tuning scientific applications to fully exploit new architectures, navigating a complex optimization space that spans algorithm design, source implementation, compiler flags and pass sequences, and kernel launch parameters. Existing approaches can effectively search parts of this space in isolation, such as launch configurations or compiler settings, but optimizing across the full space still requires substantial human expertise and iterative manual effort. In this paper, we present Record-Remix-Replay (R^3), a hierarchical optimization framework that combines LLM-driven evolutionary search, Bayesian optimization, and record-replay compilation techniques to efficiently explore GPU kernel optimizations from source-level implementation choices down to compiler pass ordering and runtime configuration. By making candidate evaluation fast and scalable, our approach enables practical end-to-end search over optimization dimensions that are typically treated separately. We show that Record-Remix-Replay can optimize full scientific applications better than traditional approaches over kernel parameters and compiler flags, while also being nearly an order of magnitude faster than modern evolutionary search approaches.
Daniel Nichols, Konstantinos Parasyris, Caetano Melone +3
Lawrence Livermore National Laboratory, Livermore, USA