We present Kernel-Smith, a framework for high-performance GPU kernel and operator generation that combines a stable evaluation-driven evolutionary agent with an evolution-oriented post-training recipe. On the agent side, Kernel-Smith maintains a population of executable candidates and iteratively improves them using an archive of top-performing and diverse programs together with structured execution feedback on compilation, correctness, and speedup. To make this search reliable, we build backend-specific evaluation services for Triton on NVIDIA GPUs and Maca on MetaX GPUs. On the training side, we convert long-horizon evolution trajectories into step-centric supervision and reinforcement learning signals by retaining correctness-preserving, high-gain revisions, so that the model is optimized as a strong local improver inside the evolutionary loop rather than as a one-shot generator. Under a unified evolutionary protocol, Kernel-Smith-235B-RL achieves state-of-the-art overall performance on KernelBench with Nvidia Triton backend, attaining the best average speedup ratio and outperforming frontier proprietary models including Gemini-3.0-pro and Claude-4.6-opus. We further validate the framework on the MetaX MACA backend, where our Kernel-Smith-MACA-30B surpasses large-scale counterparts such as DeepSeek-V3.2-think and Qwen3-235B-2507-think, highlighting potential for seamless adaptation across heterogeneous platforms. Beyond benchmark results, the same workflow produces upstream contributions to production systems including SGLang and LMDeploy, demonstrating that LLM-driven kernel optimization can transfer from controlled evaluation to practical deployment.
Figures & tables
Figure 1: Comparison of the best program score trajectories across various models over successive evolutionary generations. The program score is formulated as a linear function directly proportional to the speedup, penalizing programs that fail compilation or correctness checks by assigning them appropriately low scores. Throughout the evolution steps, our RL model (Kernel-Smith-235B-RL) exhibited superior efficiency for evolution and peak performance.
Figure 2: An overview of the evolutionary process of our proposed framework and the corresponding model training pipeline.
Models
Level 1 (Easy)
Level 2 (Medium)
Level 3 (Hard)
AVG Level-1/2/3
Corr
Fast1
Avg AMSR
Corr
Fast1
Avg AMSR
Corr
Fast1
Avg AMSR
Corr
Fast1
Avg AMSR
Baselines
Qwen3-235B-2507-think
92
0.57
1.86
96
0.86
3.96
84
0.42
0.76
90.67
0.62
2.20
Qwen3.5-397B-think
89
0.55
2.00
90
0.72
3.86
74
0.38
0.61
84.33
0.55
2.16
Minimax-M2.5
70
0.49
2.39
91
0.77
1.06
56
0.30
0.36
72.33
0.52
1.27
Kimi-K2.5
89
0.55
2.00
90
0.72
3.86
74
0.38
0.61
84.33
0.55
2.16
Table 1: Benchmark Results with Baselines and Our Model. Best results are in bold, and second best are underlined.
Models
Activation
Normalization
Reduction&Aggregation
Loss Function
AVG
Corr
Fast1
Avg AMSR
Corr
Fast1
Avg AMSR
Corr
Fast1
Avg AMSR
Corr
Fast1
Avg AMSR
Corr
Fast1
Avg AMSR
Baselines
GPT-OSS-20B
100
1.00
3.25
100
0.88
29.7
94.1
0.53
3.63
100
0.80
8.56
97.8
0.78
8.69
Qwen3-30B-A3B
100
1.00
9.03
100
1.00
7.03
100
0.71
17.44
100
0.80
5.67
100
0.87
11.48
Qwen3-235B-2507-think
100
1.00
14.55
100
0.75
35.18
100
0.59
1.36
100
1.00
6.12
100
0.80
12.30
DeepSeek-v3.2-think
100
1.00
10.09
100
0.88
6.06
94.1
0.47
8.60
100
0.60
2.89
97.8
0.73
8.01
Table 2: Evaluation results on MACA kernel generation task.
Figure 3: The acceleration curves during the evolutionary process of applying Kernel-Smith to operators across three real-world scenarios.
Table 3: Isolated kernel benchmarking for the SGLang normal_decode_set_metadata fused Triton kernel, LMDeploy fused MoE routing kernel and DeepSeek engram kernel on NV-H200.
Max batch size
Input length
Output length
Baseline latency (ms)
PR latency (ms)
Rel. gain (%)
16
64
64
251.20
249.87
0.53
16
128
128
430.46
427.80
0.62
16
256
256
794.13
787.26
0.87
16
512
512
1579.02
1569.09
0.63
16
1024
1024
3164.77
3147.41
0.55
16
2048
2048
7014.71
6978.49
0.52
Table 4: End-to-end serving latency reported in the SGLang PR on NV-H200 using sglang.bench_serving with meta-llama/Meta-Llama-3.1-8B-Instruct .
Max batch size
Input length
Output length
Input throughput (tokens/s)
Output throughput (tokens/s)
Baseline
Generated kernel
Rel. gain (%)
Baseline
Generated kernel
Rel. gain (%)
512
1K
4K
471.70
485.86
3.00
1886.78
1943.45
3.00
512
1K
8K
177.62
182.08
2.51
1420.97
1456.67
2.51
512
1K
16K
60.92
62.53
2.64
974.70
1000.45
2.64
512
4K
1K
3885.33
3972.52
2.24
971.33
993.13
2.24
512
8K
1K
3986.93
4067.89
2.03
498.37
508.49
2.03
Table 5: End-to-end DeepSeek-v3.2 inference throughput reported in the LMDeploy release note on NV-H200 with tensor parallelism degree 8.
GPU kernel optimization challenges LLMs beyond standard coding tasks, as it requires an understanding of hardware architecture, parallel computing optimization strategies, and profiling outputs. However, most existing approaches leveraging LLMs for kernel generation apply standard prompting and feedback loops, considering hardware only through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the space of GPU kernels through (1) MAP-Elites quality diversity search with kernel-specific behavioral dimensions to sustain exploration; (2) meta-prompt evolution that co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) a template-based parameter optimization approach to tune kernels to inputs and hardware. We evaluate this framework on Kernel-Bench, robust-kbench and custom tasks, generating SYCL kernels as a cross-platform GPU programming paradigm, and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods and achieves an average speedup of 2.3 on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, allowing quick benchmarking and featuring a flexible user input layer to support kernel generation for a wide range of real use cases beyond benchmarking.
Nina Wiedemann, Quentin Leboutet, Michael Paulitsch +2
Optimizing GPU kernels is critical for efficient modern machine learning systems yet remains challenging due to the complex interplay of design factors and rapid hardware evolution. Existing automated approaches typically treat Large Language Models (LLMs) merely as stochastic code generators within heuristic-guided evolutionary loops. These methods often struggle with complex kernels requiring coordinated, multi-step structural transformations, as they lack explicit planning capabilities and frequently discard promising strategies due to inefficient or incorrect intermediate implementations. To address this, we propose Search via Co-Evolving World Model and build K-Search based on this method. By replacing static search heuristics with a co-evolving world model, our framework leverages LLMs' prior domain knowledge to guide the search, actively exploring the optimization space. This approach explicitly decouples high-level algorithmic planning from low-level program instantiation, enabling the system to navigate non-monotonic optimization paths while remaining resilient to temporary implementation defects. We evaluate K-Search on diverse, complex kernels from FlashInfer, including GQA, MLA, and MoE kernels. Our results show that K-Search significantly outperforms state-of-the-art evolutionary search methods, achieving an average 2.10x improvement and up to a 14.3x gain on complex MoE kernels. On the GPUMode TriMul task, K-Search achieves state-of-the-art performance on H100, reaching 1030us and surpassing both prior evolution and human-designed solutions.
Generating high-performance GPU kernels remains challenging due to the need for both correctness and hardware-aware optimization. While large language models (LLMs) show promise in code generation, they often fail to produce kernels that are both correct and efficient. We propose Kernel Foundry, a diagnosis-driven evolutionary framework for automatic GPU kernel optimization. Our method combines expert-guided, retrieval-augmented initialization with a multi-island evolutionary search, where candidate kernels are iteratively refined using structured diagnostic feedback. A centralized experience library accumulates reusable optimization knowledge to guide subsequent evolution, while explicit mechanisms prevent cheating behaviors that bypass kernel-level computation. Experiments on KernelBench show that our method consistently improves both correctness and performance over strong baselines, achieving up to 100% correctness on Level~2.
Zixuan Huang, Da Chen, Kecheng Huang +5
1The Chinese University of Hong Kong · 2Noah’s Ark Lab, Huawei