Organizations: State Key Lab of Processors, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Intelligent Software Research Center, Institute of Software, CAS · Institute of AI for Industries, Chinese Academy of Sciences
High-performance GPU kernels are essential to modern machine learning systems, yet automatically generating kernels that are both correct and efficient remains challenging. Existing LLM-based approaches face two major limitations: the scarcity of high-quality training data aligned with the model's current capabilities, and the inherent trade-off between kernel correctness and performance. To address these challenges, we propose KernelZero, a co-evolution framework that continuously improves GPU kernel generation through two specialized models: a Proposer that generates Torch modules from API sets and a Coder that translates them into CUDA or Triton kernels. KernelZero uses a frontier-driven module generation mechanism to continuously produce capability-aligned training modules based on the Coder's current weaknesses. It further introduces Correctness-Aware Group Relative Policy Optimization (CA-GRPO), which optimizes performance only after correctness becomes sufficiently reliable. By alternating the optimization of the Proposer and Coder, KernelZero forms an automatic curriculum that enables targeted and training-efficient capability improvement. Empirically, KernelZero-7B surpasses Claude-4.5-Sonnet on CUDA and DeepSeek-V4-Pro on Triton. On KernelBench Level 1 and 2, it achieves CUDA pass@1 scores of 75.8% and 69.6%, respectively, with pass@10 reaching 100% and 97%. On Triton, it achieves pass@1 scores of 77.2% and 72.5%, respectively.
Figures & tables
Figure 1: Overview of KernelZero , a co-evolution framework for kernel generation. The system consists of two models with explicit interfaces: a Proposer that takes a set of Torch APIs as input and generates a runnable Torch module, and a Coder that translates the module into an optimized CUDA or Triton kernel.
Figure 2: Overview of KernelZero , a co-evolution framework for kernel generation. The system consists of two models with explicit interfaces: a Proposer that takes a set of Torch APIs as input and generates a runnable Torch module, and a Coder that translates the module into an optimized CUDA or Triton kernel.
Figure 3: Training dynamics of KernelZero. The left panel shows the validity rate, Coder accuracy, and frontier reward during Proposer training. The right panels report pass@1 and fast 1 @1 for CUDA and Triton Coder training. The solid curves denote the complete KernelZero training process, while the lighter dotted curves freeze the Proposer from Step 40.
Figure 4: Ablation of the correctness threshold α . Each bar reports the speedup of the fastest correct kernel among 10 generations. KernelZero-SFT denotes the cold start initialization, while KernelZero-RL uses α=0.5 . The lighter red bars show the results for α=0 , 0.9 , and 1.1 . DeepSeek-V4-Pro and GLM-5.2 are included as external references.
Kernel
Model
Params
Level 1
Level 2
pass@1
pass@5
pass@10
Tok.
pass@1
pass@5
pass@10
Tok.
CUDA
Foundation Models
Claude-3.5-Sonnet
–
26.7
53.42
64
1.9k
5.9
21.27
32
2.6k
Claude-3.7-Sonnet
–
26.9
59.83
70
3.5k
15.6
46.03
61
4.3k
Claude-4.5-Sonnet
–
76.4
95.35
97
2.1k
66.5
90.25
93
3.1k
Gemini-2.5-Flash
–
24.2
62.76
82
14.8k
13.1
44.13
62
21.2k
Table 1: Execution accuracy and token cost of CUDA and Triton kernel generation on KernelBench. The best and second-best available results within each backend are highlighted in bold and underlined , respectively. Lower token cost is better.
Figure 5: Fast@ k results on KernelBench. The left column reports Triton generation and the right column reports CUDA generation. Fast p @ k is the percentage of tasks for which at least one of k generated kernels is correct and achieves more than p× speedup over PyTorch Eager. KernelZero is compared with representative foundation and specialized models.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Overview of Keck , the execution-based validation server for CUDA and Triton kernels. CUDA extensions are compiled in parallel on CPU workers before being dispatched to isolated GPU workers. Triton programs are JIT-compiled on their assigned GPUs. Both paths compare the generated implementation against the PyTorch reference and measure performance only after the correctness check passes.
Figure 7: Torch API co-occurrence statistics and sampled API combinations covering all 442 candidate APIs.
Generating high-performance GPU kernels remains challenging due to the need for both correctness and hardware-aware optimization. While large language models (LLMs) show promise in code generation, they often fail to produce kernels that are both correct and efficient. We propose Kernel Foundry, a diagnosis-driven evolutionary framework for automatic GPU kernel optimization. Our method combines expert-guided, retrieval-augmented initialization with a multi-island evolutionary search, where candidate kernels are iteratively refined using structured diagnostic feedback. A centralized experience library accumulates reusable optimization knowledge to guide subsequent evolution, while explicit mechanisms prevent cheating behaviors that bypass kernel-level computation. Experiments on KernelBench show that our method consistently improves both correctness and performance over strong baselines, achieving up to 100% correctness on Level~2.
Zixuan Huang, Da Chen, Kecheng Huang +5
1The Chinese University of Hong Kong · 2Noah’s Ark Lab, Huawei
Improving GPU kernel efficiency is crucial for efficient AI systems. Recent work has explored leveraging large language models (LLMs) for GPU kernel generation and optimization. However, existing LLM-based kernel optimization pipelines typically rely on opaque, implicitly learned heuristics within the LLMs to determine optimization strategies. This leads to inefficient trial-and-error and weakly interpretable optimizations. Our key insight is to replace implicit heuristics with expert optimization knowledge that is aware of task trajectories. Specifically, we present KernelClaw, a multi-agent framework with a dual-level memory architecture. KernelClaw operates by coordinating agents with long-term memory of reusable expert knowledge and short-term memory to prevent repetitive backtracking. On KernelBench Levels 1-3, KernelClaw achieves a 100% success rate and average speedups of 6.98x, 7.89x, and 1.94x over Torch Eager on Levels 1, 2, and 3, respectively, outperforming prior baselines. Code is available at https://github.com/0satan0/KernelMem/.
Qitong Sun, Jun Han, Tianlin Li +6
School of Computer Science and Engineering, Beihang University, China · Zhejiang Lab, China · Nanyang Technological University, Singapore
Native GPU kernel generation turns high-level tensor programs into executable, efficient low-level code. Existing Large Language Models (LLMs) struggle with this task, while execution-based reinforcement learning suffers from sparse rewards, reward hacking, and training instability. We present MusaCoder, a full-stack training framework for native GPU kernel generation on CUDA and MUSA backends. MusaCoder combines progressive kernel-oriented data synthesis, diversity-preserving rejection fine-tuning, and execution-feedback Reinforcement Learning (RL) through MooreEval, a distributed verifier and reward environment. To stabilize RL, MusaCoder introduces PrimeEcho for first-turn-anchored multi-turn rewards, Buffered Dynamic Retry for recovering signals from all-failed hard samples, and MirrorPop for off-policy sequence filtering. Experiments on KernelBench and a MUSA-ported variant show that MusaCoder outperforms strong open-source and proprietary baselines in both correctness and empirical speedup, with the 9B model matching or exceeding frontier closed-source models and the 27B model establishing a new state of the art. These results demonstrate not only the effectiveness of full-stack execution-feedback training for native kernel generation, but also the capability of Moore Threads GPUs to support the complete LLM post-training stack, providing a practical foundation for large-model training and optimization on emerging accelerators.