GPU Kernel Generation

GPU: Graphics Processing Unit

GPU kernel generation turns operations or workload specifications into device code that runs efficiently on accelerators, where data movement, parallelism, and hardware-specific details strongly affect performance. Approaches include compiler and domain-specific language (DSL) systems with auto-scheduling, as well as LLM-based agents that generate CUDA or Triton code and refine it using profiling, execution feedback, reinforcement learning, or evolutionary search. Verification and benchmarking are central: a kernel must preserve numerical behavior across shapes and inputs while delivering speedups on target hardware. Systems can outperform framework baselines on selected tasks, but matching expert libraries and generalizing across hardware and production workloads remain difficult, and kernel-level gains do not always improve end-to-end runtime.

Latest papers

67 papers3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

God Does Not Play Dice: Enforcing Batch Invariance in Agentic Kernel Generation
Oct 27, 2026Youwei Bao, Chengpeng Wang, Simin ChenGPU Kernel Generation GPU Kernel Optimization
STAR Lab, George Mason University · National University of Singapore
D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?
Oct 2, 2026Daifeng Li, Huiqiang Jiang, Chengruidong Zhang +6GPU Kernel Generation GPU Kernel Optimization
HKUST · Alibaba Group · USTC
AI as a Compiler: Compiling Triton kernels without the Triton compiler
Sep 29, 2026François Costa, Charly Castes, Thomas Bourgeat +1GPU Kernel Optimization Large Language Model-Guided Optimization
Stanford University · EPFL
GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulation
Sep 28, 2026Yuchen Sun, Jinjin He, Sinan Wang +1GPU Kernel Optimization Benchmark Design
Georgia Institute of Technology
KernelZero: Co-Evolving Proposer and Coder for Continuously Improved GPU Kernel Generation
Sep 27, 2026Changxin Ke, Rui Zhang, Zixiang Fang +8GPU Kernel Generation LLM-Based Program Synthesis
State Key Lab of Processors, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Intelligent Software Research Center, Institute of Software, CAS +1
How Much of a Real Workload Can LLM-Generated GPU Kernels Actually Reach?
Sep 17, 2026Gaurav Agarwal, Ashish Garg, Isha SinghalGPU Kernel Generation GPU Kernel Optimization
AMDKernelVault: Large-Scale Datasets and Agentic Training for AMD GPU Kernel Optimization
Sep 11, 2026Ji Liu, Saptarshi Majumder, Yiqing Huang +14GPU Kernel Optimization Large Language Model-Guided Optimization
Advanced Micro Devices, Inc.
Measuring the Checker: Mutation Analysis for GPU-Kernel Benchmark Oracles
Sep 2, 2026Mingzhe Du, Anh Tuan Luu, Dong Huang +1GPU Kernel Correctness Verification Benchmark Auditing
National University of Singapore · Nanyang Technological University · CAIR, VinUniversity
CUDA-Harness: Harnessing Agentic CUDA Kernel Generation and Optimization from Natural Language
Aug 30, 2026Qi Fan, An Zou, Yehan MaGPU Kernel Generation LLM Code Generation
Shanghai Jiao Tong University
DataKernelBench: Can LLMs Optimize Database Queries on GPUs?
Aug 25, 2026Gokul Karthik Kumar, Yotam Perlitz, Corey Lammie +2GPU Kernel Optimization GPU Kernel Generation
IBM Research, Zurich, Switzerland · TU Wien, Vienna, Austria
HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization
Aug 21, 2026Jinghao Wang, Qiqi Gu, Chenpeng Wu +3GPU Kernel Optimization GPU Kernel Generation
Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University
PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX
Aug 18, 2026Genghan Zhang, Yixin Dong, Chengze Fan +4LLM Evaluation GPU Kernel Optimization
Stanford University · Carnegie Mellon University · Independent Researcher +1
KernelArc: A Multi-Agent Framework for GPU Kernel Optimization
Aug 17, 2026Joyjit Kundu, Ben Stoffelen, Kaili Wang +2GPU Kernel Optimization GPU Kernel Generation
AILabs, Interuniversity Microelectronics Centre (IMEC)
A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family
Aug 13, 2026Rishi Shah, Rishav ShresthaAutomated Software Testing Formal Verification
E3A Healthcare
CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution
Aug 12, 2026Zihao Ye, Yingyi Huang, Hongyi Jin +11GPU Kernel Optimization GPU Kernel Generation
1NVIDIA · 2Carnegie Mellon University
RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks
Aug 12, 2026Jinjun Huang, Zhongzhen Wen, Tongtong Xu +3LLM Evaluation GPU Kernel Generation
College of Computer Science and Technology and the State Key Laboratory of Blockchain and Data Security, Zhejiang University · State Key Lab for Novel Software Technology, Nanjing University · Software Engineering Application Technology Laboratory, Huawei +1
SparseDitto: An Agentic Sparse Compilation Framework through Architecture-Aware Synthesis on GPUs
Aug 5, 2026Shiyang Li, Guangyan Sun, Jinwei Tang +3GPU Kernel Optimization High-Performance Computing
University of Minnesota Minneapolis, MN, USA · Northeastern University Boston, MA, USA
Don't Regenerate, Debug: A Domain-Specific Agent for Repairing Near-Miss Accelerator Kernels
Aug 3, 2026Yansong Sun, Shenxiu Wu, Siyuan Chen +6Automated Program Repair GPU Kernel Generation
Department of Computer Science, City University of Hong Kong · Application Software Engineering Lab, Huawei Technologies Ltd.
LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation
Aug 3, 2026Tankun Li, Zhi Chen, Yaohua TangRL for Code Generation GPU Kernel Generation
Moore Threads AI
Meganeura: Portable GPU Training and Inference through Vulkan and Metal
Aug 3, 2026Dzmitry MalyshauEfficient Neural Network Inference GPU Kernel Optimization
Independent Researcher
Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation
Jul 23, 2026Quazi Ishtiaque Mahmud, Nesreen K. Ahmed, Ali JannesariRL for Code Generation GPU Kernel Optimization
Iowa State University, Ames, IA, USA · Cisco AI Research, San Jose, CA, USA
KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?
Jul 22, 2026Peiyu Zang, Jian Tao, Jialing Zhang +4GPU Kernel Optimization Benchmark Design
Beijing Normal University · BAAI · Beijing Jiaotong University +2
Harness Engineering for LLM-Driven GPU Kernel Generation
Jul 20, 2026Yue Shui, Chenyu Ma, Hangfei Xu +2GPU Kernel Optimization Large Language Model-Guided Optimization
Baidu, Inc.
Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
Jul 16, 2026Lingyun Yang, Yuxiao Wang, Shenghao Liang +8GPU Kernel Optimization Large Language Model-Guided Optimization
ATREX Team†, Alibaba Group
Nova: An End-to-End MLIR Compiler for Deep Learning
Jul 15, 2026Adwaid Suresh, Aparna A, Harshini V M +4GPU Kernel Generation
Blubridge AI
EGG: An Expert-Guided Agent Framework for Kernel Generation
Jun 25, 2026Yaochen Han, Ke Fan, Hongxu Jiang +5GPU Kernel Optimization Large Language Model-Guided Optimization
Beihang University, Beijing, China · Shanghai Jiao Tong University, Shanghai, China · Tsinghua University, Beijing, China +1
Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization
Jun 24, 2026Jiading Gai, Shuai Zhang, Kaj Bostrom +6GPU Kernel Optimization High-Performance Computing
Amazon · Siemens · University of Minnesota
The Correctness Illusion in LLM-Generated GPU Kernels
Jun 18, 2026Dipankar SarkarAutomated Software Testing Differential Testing
Arizona State University, USA
daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization
Jun 15, 2026Dayuan Fu, Mohan Jiang, Tongyu Wang +5RL for Code Generation GPU Kernel Optimization
1SII · 2SJTU · 3GAIR
From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generation
Jun 15, 2026Wentao Chen, Jiace Zhu, Xing Zhe Chai +4GPU Kernel Optimization LLM Fine-Tuning
1Shanghai Jiao Tong University · 2Biren Technology

Oct 27, 2026J/K move · Enter open · S save

God Does Not Play Dice: Enforcing Batch Invariance in Agentic Kernel Generation

Youwei Bao, Chengpeng Wang, Simin Chen

STAR Lab, George Mason University · National University of Singapore

GPU Kernel Generation GPU Kernel Optimization GPU Kernel Correctness Verification

Open paper AskPDF arXiv

Large language model (LLM) serving systems dynamically batch user requests for GPU execution, so the same request may be processed alongside different requests across runs. If the GPU kernels lack batch invariance (BI), this changing batch context can alter the numerical output for an unchanged input, undermining reproducibility. Existing kernel generation agents focus on correctness and efficiency but overlook BI. We find 19 confirmed BI violations among 83 batch-applicable released artifacts. To address this limitation, we propose \tool, which combines contract-guided static dependency analysis with dynamic validation of bitwise consistency across batch contexts. \tool is designed as a plug-in component for general kernel agents, allowing their original optimization capabilities to be reused while adding BI enforcement. We integrate it as successive acceptance gates in the optimization loop, rejecting structurally risky or dynamically non-invariant candidates before performance benchmarking and returning feedback to the agent. Experiments across three agent frameworks under baseline and BI-aware generation on real world LLM serving tasks show that \tool improves the BI pass rate from 67.6% to 91.9% while retaining 91.8% of the baseline geometric mean speedup. These results demonstrate that \tool can be incorporated into kernel optimization by agents as an additional reliability objective alongside correctness and speed.