GPU Kernel Generation

GPU: Graphics Processing Unit

GPU kernel generation turns operations or workload specifications into device code that runs efficiently on accelerators, where data movement, parallelism, and hardware-specific details strongly affect performance. Approaches include compiler and domain-specific language (DSL) systems with auto-scheduling, as well as LLM-based agents that generate CUDA or Triton code and refine it using profiling, execution feedback, reinforcement learning, or evolutionary search. Verification and benchmarking are central: a kernel must preserve numerical behavior across shapes and inputs while delivering speedups on target hardware. Systems can outperform framework baselines on selected tasks, but matching expert libraries and generalizing across hardware and production workloads remain difficult, and kernel-level gains do not always improve end-to-end runtime.

Latest papers

67 papers3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning
Mar 3, 2026Shiyang Li, Zijian Zhang, Winson Chen +3Program Synthesis GPU Kernel Generation
University of Minnesota-Twin Cities, Minnesota, USA.
CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation
Feb 27, 2026Weinan Dai, Hanlin Wu, Qiying Yu +13GPU Kernel Generation GPU Kernel Optimization
ByteDance Seed · Institute for AI Industry Research (AIR), Tsinghua University · SIA-Lab of Tsinghua AIR and ByteDance Seed
RedFuser: An Automatic Operator Fusion Framework for Cascaded Reductions on AI Accelerators
Feb 24, 2026Xinsheng Tang, Yangcheng Li, Nan Wang +5GPU Kernel Optimization GPU Kernel Generation
Alibaba Cloud Computing, Shanghai, China · Alibaba Cloud Computing, Shenzhen, China
K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
Feb 22, 2026Shiyi Cao, Ziming Mao, Joseph E. Gonzalez +1GPU Kernel Optimization GPU Kernel Generation
UC Berkeley
KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta
Dec 29, 2025Gang Liao, Hongsen Qin, Ying Wang +36GPU Kernel Optimization GPU Acceleration
KernelEvolve Team, Meta Platforms
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
Dec 2, 2025Songqiao Su, Xiaoya Li, Albert Wang +3Reinforcement Learning GPU Kernel Optimization
DeepReinforce Team
From Legacy Fortran to Portable Kokkos: An Autonomous Agentic AI Workflow
Sep 15, 2025Sparsh Gupta, Kamalavasan Kamalakkannan, Maxim Moraru +2Code Translation Agentic Code Generation
Los Alamos National Laboratory, Los Alamos, NM 87544 USA · Franklin W. Olin College of Engineering, Needham, MA 02492 USA

Mar 3, 2026J/K move · Enter open · S save

StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning

Shiyang Li, Zijian Zhang, Winson Chen, Yuebo Luo+2

University of Minnesota-Twin Cities, Minnesota, USA.

Program Synthesis GPU Kernel Generation Multi-Agent Coding GPU Kernel Optimization

Open paper AskPDF arXiv

Modern machine learning (ML) workloads increasingly rely on GPUs, yet achieving high end-to-end performance remains challenging due to dependencies on both GPU kernel efficiency and host-side settings. Although LLM-based methods show promise on automated GPU kernel generation, prior works mainly focus on single-kernel optimization and do not extend to end-to-end programs, hindering practical deployment. To address the challenge, in this work, we propose StitchCUDA, a multi-agent framework for end-to-end GPU program generation, with three specialized agents: a Planner to orchestrate whole system design, a Coder dedicated to implementing it step-by-step, and a Verifier for correctness check and performance profiling using Nsys/NCU. To fundamentally improve the Coder's ability in end-to-end GPU programming, StitchCUDA integrates rubric-based agentic reinforcement learning over two atomic skills, task-to-code generation and feedback-driven code optimization, with combined rubric reward and rule-based reward from real executions. Therefore, the Coder learns how to implement advanced CUDA programming techniques (e.g., custom kernel fusion, cublas epilogue), and we also effectively prevent Coder's reward hacking (e.g., just copy PyTorch code or hardcoding output) during benchmarking. Experiments on KernelBench show that StitchCUDA achieves nearly 100% success rate on end-to-end GPU programming tasks, with 1.72x better speedup over the multi-agent baseline and 2.73x than the RL model baselines. Code of the STITCHCUDA framework is avalaible at https://github.com/UMN-APEX-Lab/StitchCUDA.