Developing high-performance CUDA kernels demands specialized knowledge in algorithm implementation, correctness validation, and hardware-aware parallel optimization, creating a substantial expertise barrier and making generating CUDA kernels directly from natural language (Text2CUDA) essential. Meanwhile, the general-purpose code generation capability of Large Language Models (LLMs) prompts a series of works exploring LLM-based CUDA kernel generation. They mainly focus on transpilation from high-level frameworks such as PyTorch to CUDA (Torch2CUDA) rather than Text2CUDA, where models must understand the high-level input semantics and handle low-level kernel implementation and validation. Additionally, these methods are vulnerable to reward hacking due to reliance on predefined test inputs. In this paper, we propose CUDA-Harness, a framework for harnessing agentic CUDA kernel generation and optimization from natural language. Specifically, we introduce Intermediate-Structured Generation to connect high-level semantic understanding with low-level kernel generation. To dilute reward hacking in Text2CUDA, we construct Synthesis-Based Verification to provide isolated test data and progressive validation. Furthermore, we propose Feedback-Adaptive Evolution, a kernel evolution strategy that prioritizes correctness while optimizing performance. Finally, through extensive experiments, we demonstrate the effectiveness of CUDA-Harness, with further evaluations illustrating generalization across LLMs, hardware platforms, and to C-to-CUDA transpilation.
Figures & tables
Figure 1: Text2CUDA vs. Torch2CUDA. Torch2CUDA starts from detailed PyTorch code, while Text2CUDA starts from an underspecified natural-language prompt with no available test data.
Figure 3: The overview of the CUDA-Harness.
Method
Overall
Level 1
Level 2
Level 3
Comp.
Func.
RScore
Comp.
Func.
RScore
Comp.
Func.
RScore
Comp.
Func.
RScore
Baseline
90.1
60.9
80.1
89.6
69.0
87.7
92.6
66.2
86.1
88.2
47.6
66.6
OpenCode
92.1
60.6
78.7
93.6
66.6
82.8
92.0
65.8
81.9
90.8
49.4
71.3
Codex
98.3
69.5
86.5
99.2
79.2
91.3
98.6
75.4
94.5
97.2
54.0
73.6
KernelSkill
99.4
63.7
86.1
99.0
72.0
94.4
100.0
70.4
94.3
99.2
48.6
69.7
CudaForge
99.8
67.7
87.9
99.6
75.2
92.7
99.8
73.2
93.7
100.0
54.8
77.4
Table 1: The comparison and ablation results. CUDA-ISG retains only Intermediate-Structured Generation, while CUDA-Harness* further removes Correctness-first Refinement from CUDA-Harness.
Synthesis-based Verification
Benchmark Validation
Pass
Fail
Pass
73.6%
8.7%
Fail
6.9%
10.8%
Table 2: The consistency between Synthesis-Based Verification and benchmark validation.
Method
Metric
Cross-LLMs
Cross-Hardware
Seed 2.0 Lite
DeepSeek V3.2
GLM 5.1
A40
1660 SUPER
Jetson AGX Orin
Baseline
Comp.
90.1
96.5
94.8
90.1
90.5
90.2
Func.
60.9
57.8
60.7
60.9
60.3
59.9
RScore
80.1
79.0
79.7
80.1
52.7
29.9
CUDA-Harness
Comp.
99.7
99.6
99.2
99.7
99.7
99.7
Func.
80.5
76.3
101.8
80.5
79.8
79.4
Table 3: The generalization results of CUDA-Harness across LLMs and across hardware platforms on CUDABench.
Method
Comp.
Func.
Score
Baseline
94.8
90.1
451.6
CUDA-Harness
98.7
97.1
514.8
Table 4: The generalization results of CUDA-Harness to C-to-CUDA transpilation.
GPU kernel optimization is fundamental to modern deep learning but remains a highly specialized task requiring deep hardware expertise. Despite strong performance in general programming, large language models (LLMs) remain uncompetitive with compiler-based systems such as torch.compile for CUDA kernel generation. Existing CUDA code generation approaches either rely on training-free refinement or fine-tune models within fixed multi-turn execution-feedback loops, but both paradigms fail to fundamentally improve the model's intrinsic CUDA optimization ability, resulting in limited performance gains. We present CUDA Agent, a large-scale agentic reinforcement learning system that develops CUDA kernel expertise through three components: a scalable data synthesis pipeline, a skill-augmented CUDA development environment with automated verification and profiling to provide reliable reward signals, and reinforcement learning algorithmic techniques enabling stable training. CUDA Agent achieves state-of-the-art results on KernelBench, delivering 100%, 100%, and 92% faster rate over torch.compile on KernelBench Level-1, Level-2, and Level-3 splits, outperforming the strongest proprietary models such as Claude Opus 4.5 and Gemini 3 Pro by about 40% on the hardest Level-3 setting.
Weinan Dai, Hanlin Wu, Qiying Yu +13
ByteDance Seed · Institute for AI Industry Research (AIR), Tsinghua University · SIA-Lab of Tsinghua AIR and ByteDance Seed
High-performance CUDA kernels are essential for scalable AI systems, while Large Language Models (LLMs) still struggle to generate correct kernels due to strict and implicit execution constraints. Existing LLM-based approaches either rely on costly agentic or reinforcement-learning (RL) pipelines, or adopt supervised fine-tuning (SFT) objectives that fail to explicitly model CUDA sensitivity, namely code tokens or regions tightly coupled with execution constraints. In this work, we investigate CUDA sensitivity from the perspective of token confidence patterns, showing that CUDA sensitivity appears at both token and region levels, where most CUDA-sensitive tokens are predicted with high confidence, while a smaller low-confidence subset forms regions corresponding to execution-critical structures. These findings suggest that effective CUDA kernel generation should both leverage high-confidence CUDA-sensitive tokens and preserve low-confidence CUDA-sensitive regions. Building on these insights, we propose CUDA-Sensitive Instruction Tuning (CuSeT), a low-cost post-training method within a simple SFT framework. CuSeT follows the principle of ``from tokens to regions'' by combining adaptive token-level masking with region-aware sample reweighting. Experiments show that CuSeT consistently improves functional correctness across multiple model families and scales, outperforming standard SFT and advanced SFT variants, while achieving competitive performance against frontier CUDA kernel generation models with substantially lower inference cost.
Wentao Chen, Jiace Zhu, Xing Zhe Chai +4
1Shanghai Jiao Tong University · 2Biren Technology
Large language models show promise for automated CUDA programming, however even the strongest coding models (e.g., Claude-Opus-4.6) may still fall short of expert-level, architecture-aware optimization. We introduce CUDAHercules, a benchmark that evaluates generated CUDA against end-to-end human-expert SOTA systems. It spans single kernels, module-level operators, full applications, and unsolved challenge tasks across Ampere, Hopper, and Blackwell GPUs, with end-to-end tasks gated by domain-specific semantic validators. Evaluating models such as Claude-Opus-4.6 and GPT-5.4 shows a large gap between runnable CUDA and expert CUDA engineering: models often compile and pass tests, but rarely recover the optimization strategies needed to match expert performance. Application semantics further reduce success, and iterative or tool-augmented feedback can improve correctness while drifting toward slow fallback implementations. These results show that automated CUDA programming remains far from fully solved and requires stronger hardware reasoning, better tool use, and training objectives that connect code understanding to hardware architecture-grounded intelligence.