Training large language models to generate efficient CUDA kernels requires supervision spanning diverse computational requirements and iterative code optimization. Constructing this supervision is challenging because of heterogeneous specifications, and useful implementations emerge through iterative complicated testing and refinement. We introduce KernelMosaic, an open dataset pairing diverse CUDA tasks with verified solutions and trajectories. Human-guided review and diversity-aware selection curate tasks expressed as natural-language descriptions, reference programs, mathematical specifications, and their intersections. To build the solutions and trajectories, our agentic framework guides a strong LLM through implementation, correctness repair, and performance optimization with reusable CUDA guidance and execution feedback. Supervised fine-tuning of a small open-source model on KernelMosaic raises correctness from 4%, 0%, and 0% to 88%, 82%, and 16% on KernelBench L1, L2, and L3, respectively, while increasing the fraction of tasks with kernels faster than PyTorch eager at every level. Further training experiments show that supervision from the curated trajectories substantially outperforms direct responses and promotes sustained testing and revision. These results demonstrate that KernelMosaic enables a general-purpose model to achieve substantial gains in CUDA kernel generation.
Figures & tables
Figure 1: Construction of KernelMosaic. (a) Human-guided curation preserves computational intent, reviews quality and testability, and selects for diversity across specification forms. (b) The agentic framework equips a coding agent with reusable CUDA skills and correctness and performance feedback to develop implementations and record attempts and observations. (c) The dataset links tasks and tests with selected CUDA solutions, trajectories, and evaluation evidence for supervised learning.
Dataset
Domain
Scale
Data / supervision
LS-CAT ( Bjertnes et al., 2021 )
CUDA
19,683 kernels
Code and runtimes
KernelBook ( Paliskara and Saroufim, 2025 )
Triton
18,162 pairs
PyTorch–Triton pairs
AutoTriton SFT ( Li et al., 2025b )
Triton
14,102 examples
Code and reasoning
ConCuR ( Kong et al., 2025 )
CUDA
4,892 examples
Code and reasoning
CASS ( Heakl et al., 2026 )
Mixed
60,694 samples
Cross-platform pairs
CUDA-Agent-Ops-6K ( Dai et al., 2026 )
CUDA
6,000 tasks
PyTorch tasks, tests
Table 1: Related datasets for GPU kernel learning. Scale uses each resource’s native unit. Each of the 14,566 tasks in KernelMosaic has an associated development trajectory. Mixed denotes multiple source and assembly languages.
Figure 2: Length and interaction depth of the 14,566 trajectories. (a) Density of complete-trajectory tokens and assistant decisions on logarithmic scales; the dashed line marks the 32,768-token training context. (b) Median decision counts by input category within token-length intervals ( × 1,024). Intervals are right-closed; points with fewer than 20 tasks are omitted. Length includes the complete context and historical reasoning.
Method / model
Inference
L1
L2
L3
C
fast 1
C
fast 1
C
fast 1
Open-weight models
Qwen3-8B ( Li et al., 2025b )
Single-turn
16
9
16
8
14
4
DeepSeek-R1-0528 ( Li et al., 2025b )
Single-turn
35
7
42
28
26
14
Kernel-specialized open-weight models
AutoTriton-8B ( Li et al., 2025b )
Single-turn
36
10
45
17
20
10
Table 2: KernelBench correctness (C) and faster-than-PyTorch-eager coverage (fast 1 ), in percent. The 1k variants each use 1,000 training examples and one epoch. Published results use different evaluation protocols. Rows from Li et al. (2025b) generate Triton; the others generate CUDA. – denotes an unreported result. Appendix C gives protocol details and additional speedup thresholds.
Figure 3: KernelBench performance. (a) Correctness (pale background bars) and correct-and-faster coverage (hatched foreground bars; speedup >1 ) of selected models from Table 2 and full-dataset SFT. Both layers start at zero; N/A denotes unreported results. Published baselines use direct generation or pass@1, while ours use agentic inference; budgets are not matched. (b) Original and SFT task coverage above speedup p . Colors identify L1/L2/L3 (100/100/50 tasks); our candidates are evaluated on H200.
Figure 4: Supervision from agentic trajectories and direct responses. Both variants train Qwen3.5-9B for one epoch on 1,000 independently sampled examples. (a–b) KernelBench correctness and faster-than-eager coverage by level. (c) Fraction of the 250 evaluation trajectories invoking each tool. Trajectory supervision improves L1/L2 performance and supports more frequent evaluation and editing.
Figure 5: Original and full-dataset SFT behavior on KernelBench. (a) Mean within-trajectory tool-use shares; Write creates or replaces files, while Edit modifies existing contents. (b) Correct (solid) and faster-than-eager (dashed) task coverage versus cumulative output tokens ( × 1,024). Each panel includes all tasks at its level.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Split
Trajectories
Windows
Total tokens
Supervised tokens
Training
13,815
37,242
1,002,791,260
416,424,437
Validation
751
2,010
54,137,600
22,460,320
Total
14,566
39,252
1,056,928,860
438,884,757
Appendix
Table 3: Trajectory and token counts for supervised learning. Total tokens include context; supervised tokens contribute to the loss.
Table 5: Training data and optimization for the one-epoch supervision comparison. Tokens are counted before repetition across epochs; total tokens include context.
Model
L1
L2
L3
F1
F2
F1
F2
F1
F2
Qwen3-8B ( Li et al., 2025b )
9
9
8
1
4
2
DeepSeek-R1-0528 ( Li et al., 2025b )
7
1
28
2
14
2
AutoTriton-8B ( Li et al., 2025b )
10
6
17
0
10
4
GPT-4o ( Li et al., 2025b )
3
1
3
0
4
2
Claude Sonnet 4 ( Li et al., 2025b )
11
7
10
1
2
0
Appendix
Table 6: KernelBench speedup coverage (%), with Fp denoting fast p relative to PyTorch eager. Published results use different evaluation protocols. The published rows generate Triton in a single turn; our rows generate CUDA with agentic inference.
Input category
Dataset
Training
Validation
Natural language
354
334
20
PyTorch code
9,964
9,444
520
Mixed language and code
1,772
1,683
89
Algebra/matrix
451
430
21
CUDA programming tasks
2,025
1,924
101
Total
14,566
13,815
751
Appendix
Table 7: Composition of KernelMosaic (14,566 tasks). Each task has one development trajectory; train and validation contain disjoint task identifiers.
Figure 6: Computational vocabulary across the 14,566 dataset task specifications. Word size increases with the square root of the number of tasks mentioning each descriptor; colors identify five thematic groups. Terms include both natural-language expressions and corresponding API names.
Figure 7: Vocabulary across input forms. Circle area and numerical labels encode the percentage of tasks in each input category that mention at least one descriptor from the corresponding group. Groups may share tasks; each task is counted once per group. Column counts include all dataset tasks, including those without a matched descriptor.
Figure 8: Computational concepts within task specifications. (a) Jaccard overlap among the 16 most frequent descriptors, ordered by thematic group and frequency. Values of at least 10% are labeled; the diagonal is omitted. (b) Number of distinct matched descriptors per task within each input category. The fixed lexical rules are applied to the task text, including code and comments; a zero denotes no match to the descriptor vocabulary. All 14,566 tasks are included.
Figure 9: Composition of the 14,566-task dataset. (a) Category proportions in the complete dataset and its training and validation splits. (b) Cumulative task share after independently ranking sources by size in the dataset and training split; markers identify the five largest sources.
Figure 10: Computational profiles of the collected sources. Circle area represents the percentage of tasks within a source matching a descriptor; colors indicate thematic groups. We display the four most frequent descriptors in each group, selected using the complete dataset. Source rows contain 5,219, 2,839, 1,935, 1,817, 1,772, and 984 tasks, respectively. The final row pools the remaining 36 sources. Source names refer to their contributions retained in KernelMosaic.
Figure 11: Empirical trajectory distributions by input category, pooling both data splits. (a) Complete tokenized trajectory length before removal of historical reasoning. The dashed line marks the training context limit. (b) Number of assistant decisions, each comprising an assistant message and any associated tool observations. Curves normalize within each category; horizontal axes are logarithmic.
Figure 12: How trajectories become learning examples. (a) Category shares among trajectories and derived windows. (b) Token roles across 1.057 billion tokens in both splits; only assistant targets contribute to the loss. Tool categories include message wrappers; assistant categories include message bodies and end tokens. (c) Token totals with historical reasoning restored or removed, holding windows and history truncation fixed. Token totals are measured per dataset copy.
Figure 13: Sensitivity of algebra coverage to the similarity cutoff, with the previously selected collection held fixed. (a) Six measured selection settings. (b) Retained specifications at the default and category-specific cutoffs, grouped by existing review annotations. Unassessed tasks lack this annotation. Counts refer to the pre-generation algebra candidate pool.
Figure 14: Feedback and recovery under the common evaluation budget. Top: feedback proportions, normalized within each task and then averaged. Output mismatches include shape, value, and finiteness checks; build/execution errors include candidate construction, loading, and runtime failures; implementation-rule failures concern the CUDA execution requirement. Inconclusive checks remain a separate category. Tasks with no eligible evaluation are included explicitly. Bottom: correctness achieved at the first evaluation or after a revision, with all scheduled tasks in the denominator. Evaluations include recorded cache responses and repeated checks.
Model
Level
Calls
Repeated writes
Profiling
Original
L1
178.5
66.5%
6%
Original
L2
99
53.1%
4%
Original
L3
75
46.7%
2%
KernelMosaic SFT
L1
56
2.7%
87%
KernelMosaic SFT
L2
71.5
2.2%
85%
KernelMosaic SFT
L3
74
3.8%
30%
Appendix
Table 8: Trajectory behavior by model and KernelBench level. Calls is the median number of tool calls per task. Repeated writes is the percentage of completed writes reproducing an earlier file path and content. Profiling is the percentage of tasks invoking the profiler.
Figure 15: Action transitions in original and SFT trajectories. Rows denote the current tool and columns the next tool; cells show percentages averaged over tasks after row normalization within each task. All archived tool calls are included, ordered by recorded start time. Each row averages only tasks with at least one outgoing transition from that tool. A dash denotes an absent context. Repetition here refers to tool type, whereas Table 8 measures exact repeated write contents.
First candidate
First correct candidate
Model
Level
Correct (%)
Solved
Median speedup
fast 1 (%)
Original
L1
0
4
0.19 ×
0
Original
L2
0
0
—
0
Original
L3
0
0
—
0
KernelMosaic SFT
L1
40
88
1.04 ×
49
KernelMosaic SFT
L2
23
82
1.17 ×
46
Appendix
Table 9: First-candidate correctness and first-correct-candidate performance on KernelBench. Correctness and fast 1 use all tasks at each level (100/100/50). Solved counts tasks with a passing candidate within budget. Median speedup is computed over each row’s solved tasks, using their first passing implementation; a dash denotes no solved tasks.
Figure 16: Optimization after the first correct implementation. Each point compares a task’s first timed correct candidate with its best measured candidate within the trajectory budget. Both axes show speedup over PyTorch eager on logarithmic scales; diagonal guides mark equal performance. The shaded region contains tasks that cross from slower-than-eager to faster-than-eager implementations. All 178 solved SFT tasks and four solved original-model tasks are included.
Figure 17: SFT progress under joint resource cutoffs. Cells show the percentage of all tasks with a correct candidate (top) or a trusted faster-than-eager candidate (bottom) recorded before both cutoffs. Horizontal ticks denote 1,024 output tokens; active time excludes inference and infrastructure waits. All panels share the color scale.
Figure 18: Development activity after one epoch of SFT on 1,000 examples. Curves show cumulative task shares by the number of tool calls, including errored or interrupted invocations. Horizontal ticks show call counts on a log(1+calls) scale. Each level includes all scheduled tasks.
Input category
Example
Source / construction origin
PyTorch code
Affine layer normalization
KernelBook ( Paliskara and Saroufim, 2025 )
Mixed language and code
Floating-point histogram
Nemotron SFT CUDA v1
Natural language
Integer matrix transpose
fhrozen Stack CUDA prompts
Other CUDA code
Missing transform kernel
llama.cpp CUDA backend
Algebra / matrix
Batched low-rank operator
Synthetic algebra construction
Appendix
Table 10: Illustrative prompts spanning the five curation categories. Input form is distinct from the requested implementation and from the evaluation interface.
Candidate / evaluation
Correctness
Candidate ( μ s)
Eager ( μ s)
Speedup
Round 0 / public
3/3 cases
16.77
72.28
4.31 ×
Round 1 / public
3/3 cases
10.76
48.94
4.55 ×
Selected round 1 / hidden
5/5 cases
11.83
48.95
4.14 ×
Appendix
Table 11: LayerNorm implementation revision. Latencies are task-level aggregates from the adapter, in microseconds. Public rounds use the same public cases; the last row uses separate hidden cases.