Record-Remix-Replay: Hierarchical GPU Kernel Optimization using Evolutionary Search

Daniel Nichols; Konstantinos Parasyris; Caetano Melone; Tal Ben-Nun; Giorgis Georgakoudis; Harshitha Menon

cs.DCApr 13, 2026

Record-Remix-Replay: Hierarchical GPU Kernel Optimization using Evolutionary Search

Authors: Daniel Nichols, Konstantinos Parasyris, Caetano Melone, Tal Ben-Nun, Giorgis Georgakoudis, Harshitha Menon

Organizations: Lawrence Livermore National Laboratory, Livermore, USA

Abstract

As high-performance computing and AI workloads become increasingly dependent on GPUs, maintaining high performance across rapidly evolving hardware generations has become a major challenge. Developers often spend months tuning scientific applications to fully exploit new architectures, navigating a complex optimization space that spans algorithm design, source implementation, compiler flags and pass sequences, and kernel launch parameters. Existing approaches can effectively search parts of this space in isolation, such as launch configurations or compiler settings, but optimizing across the full space still requires substantial human expertise and iterative manual effort. In this paper, we present Record-Remix-Replay (R^3), a hierarchical optimization framework that combines LLM-driven evolutionary search, Bayesian optimization, and record-replay compilation techniques to efficiently explore GPU kernel optimizations from source-level implementation choices down to compiler pass ordering and runtime configuration. By making candidate evaluation fast and scalable, our approach enables practical end-to-end search over optimization dimensions that are typically treated separately. We show that Record-Remix-Replay can optimize full scientific applications better than traditional approaches over kernel parameters and compiler flags, while also being nearly an order of magnitude faster than modern evolutionary search approaches.

Figures & tables

Fig. 1: Overview of MAP-Elites evolution as in AlphaEvolve/OpenEvolve. 1 Prompts are constructed from the MAP-Elites population database, 2 sent to an LLM randomly selected from an ensemble of LLMs, and 3 evaluated by compiling and running the code. The population database is then updated according to the MAP-Elites algorithm. Multiple parallel controllers execute this loop concurrently to increase throughput.

	Source-level		Compiler-level		Runtime	System	Eval.
Work	Unstruct. rewrite	DSL / template	Flags / params	Pass order	Launch cfg.	Env. / resources	Required Execution	Cost
Kernel parameter auto-tuners
CLTune; Kernel Tuner [ 4 , 15 ]		$\checkmark$			$\checkmark$	$\circ$	Kernel	Low
Kernel Launcher [ 3 ]		$\checkmark$			$\checkmark$		Kernel	Low
DSL auto-schedulers
TVM / Ansor / MetaSchedule [ 16 , 17 , 18 ]		$\checkmark$	$\circ$		$\checkmark$		Kernel / Graph	Med.

TABLE I: Representative related work in GPU kernel auto-tuning and optimization.

\checkmark

= first-class support;

\circ

= technically possible but indirect / not the primary design point. Evaluation cost is qualitative and reflects the typical inner-loop tuning cost: Low = isolated kernel / replay / JIT evaluation, Med. = operator or graph-level tuning, High = full-application.

Fig. 2: Overview of our Record-Remix-Replay framework. 1 First, the prefix-aware prompt sampler is used to construct prompts that optimize prefix cache hits. 2 An LLM is selected intelligently based on estimated generation time to maximize GPU utilization and is used to optimize the GPU kernel. 3 LLVM IR is generated for the optimized kernel and sent to the replay server, which replays kernels and runs parallel Bayesian optimization to find the best runtime across compiler passes and launch parameters. The population database is then updated according to the MAP-Elites algorithm.

Fig. 3: The breakdown in times spent in generation, compiling, and evaluation during evolution with each progressive optimization used. Results are shown for 1 sequential evolve process using gpt-oss-120b with high reasoning effort on the QUDA application. We see our proposed optimizations have moved evolutionary optimization from evaluation-bound to LLM inference-bound.

Fig. 4: Comparison of time spent generating outputs with and without prefix-aware prompts across a 250 iteration evolution run. We see a reduction in median generation time for both a commercial (gpt-5-mini) and local (gpt-oss-120b) model. Notably, most commercial model providers offer discounts for prefix-cache hits, leading to lower overall costs during evolution.

Fig. 5: Comparison of OpenEvolve’s LLM selection algorithm to

\text{R}^{3}

’s runtime-aware LLM selection. Whereas increasing the number of parallel evolution processes w.r.t. the number of GPUs can increase utilization and throughput, it comes at the cost of increasing stale MAP-Elites database reads and slowing convergence.

\text{R}^{3}

’s runtime-aware LLM selection increases utilization and throughput over OpenEvolve for the same number of processes.

Fig. 6: Comparison of kernel speedups across the approaches and applications on an MI300A. Times are shown for all kernels in an application (Lulesh 15, MiniFE 7, s3d 54, and miniWeather 7) and dots represent outliers.

\text{R}^{3}

yields higher median and max speedups over BO and OpenEvolve.

Fig. 7: Comparison of kernel speedups across the approaches and applications on an H100. Times are shown for all kernels in an application and dots represent outliers.

\text{R}^{3}

yields higher median and max speedups over baselines.

Fig. 8: Comparison of absolute times from kernels in the miniWeather application. The y-axis encodes kernel time after

\text{R}^{3}

optimization and the x-axis encodes kernel time after optimization from record-replay + BO (left plot) and OpenEvolve (right plot). Points under the line are faster with

\text{R}^{3}

Fig. 9: Comparison of final achieved speedup versus the time-to-solution. The vertical axis shows speedup over the original application and the horizontal axis shows time-to-solution in minutes (note log scale). While

\text{R}^{3}

is slower than BO with record-replay, it produces consistently better tuning results. Furthermore,

\text{R}^{3}

is both faster and yields better speedups than OpenEvolve.

GPU Kernel Optimization

GPU kernel optimization improves the speed and memory efficiency of computations by tailoring code to GPU hardware and workloads.

See all 209 papers

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization
Mar 12, 2026Nina Wiedemann, Quentin Leboutet, Michael Paulitsch +2GPU Kernel Optimization GPU Acceleration
Intel Corporation
HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization
Aug 21, 2026Jinghao Wang, Qiqi Gu, Chenpeng Wu +3GPU Kernel Optimization GPU Kernel Generation
Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University
K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
Feb 22, 2026Shiyi Cao, Ziming Mao, Joseph E. Gonzalez +1GPU Kernel Optimization GPU Kernel Generation
UC Berkeley

Mar 12, 2026J/K move · Enter open · S save

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization

Nina Wiedemann, Quentin Leboutet, Michael Paulitsch, Diana Wofk+1

Intel Corporation

GPU Kernel Optimization GPU Acceleration Evolutionary Optimization Large Language Model-Guided Optimization LLM-Based Program Synthesis GPU Kernel Generation+1

Open paper AskPDF arXiv

GPU kernel optimization challenges LLMs beyond standard coding tasks, as it requires an understanding of hardware architecture, parallel computing optimization strategies, and profiling outputs. However, most existing approaches leveraging LLMs for kernel generation apply standard prompting and feedback loops, considering hardware only through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the space of GPU kernels through (1) MAP-Elites quality diversity search with kernel-specific behavioral dimensions to sustain exploration; (2) meta-prompt evolution that co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) a template-based parameter optimization approach to tune kernels to inputs and hardware. We evaluate this framework on Kernel-Bench, robust-kbench and custom tasks, generating SYCL kernels as a cross-platform GPU programming paradigm, and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods and achieves an average speedup of 2.3 on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, allowing quick benchmarking and featuring a flexible user input layer to support kernel generation for a wide range of real use cases beyond benchmarking.

Abstract

GPU Kernel Optimization

Explore similar work

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization

HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization

K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model