Hardware kernel optimization requires repeated compilation, correctness testing, profiling, and revision. LLM agents can automate parts of this process, and stronger foundation models, longer context windows, and longer execution horizons have improved optimization within individual tasks. These advances alone do not enable an agent to learn from completed optimization runs. Existing kernel-optimization agents seldom preserve a decision, its observed execution feedback, and the later decisions that use that evidence. Retaining every prior trajectory is also impractical because an expanding history competes with the current task for context. We present KOPE, an experience-driven framework for hardware kernel optimization. KOPE records optimization trajectories with correctness and performance feedback in Experience Graph Memory, then uses Active Context Management and Injection to retrieve relevant experience under a fixed token budget. The graph retains decision order, observed outcomes, and alternative branches, allowing evidence collected on the target hardware to inform later optimization steps and tasks. Under the same GLM-5.2 setting, the geometric mean of KOPE's per-operator speedups is 1.54× that of CANNBot, the strongest competing baseline. In a complete 53-operator ablation, Active Context Management and Injection raises pass rate from 60.0% to 84.6%, increases the evaluator-reported positive-field geometric mean from 0.0382 to 0.0661, and reduces optimization token consumption from 15.9B to 1.113B tokens relative to passive agent-led context construction. Enabling Experience Graph Memory raises full-suite pass rate from 55.2% to 84.6% and yields a 1.43× geometric-mean speedup on valid timing comparisons. These results support continual optimization through external experience while the foundation model remains fixed.
Figures & tables
Figure 1: KOPE’s experience-driven workflow. With model parameters fixed, Active Context Management and Injection constructs a bounded prompt from required task state (hot), retrieved experience (warm), and broader knowledge (cold). Target-side execution returns compiler, correctness, and performance outcomes, which are recorded in a persistent exploration journal and Decision-to-Outcome Experience Graph. Retrieval informs the next decision within the current task and carries experience across tasks.
Model
System
Op. Pass
Case Pass
Pass Rate
Score
GLM-5.2
KOPE
52/53
897/1,060
84.6%
2004.49
CANNBot
37/53
613/1,060
57.8%
1465.93
CUDA-Agent
13/53
156/1,060
14.7%
312.00
Deepseek- V4-Pro
KOPE
47/53
783/1,060
73.9%
1242.90
CANNBot
25/53
417/1,060
39.3%
1015.38
CUDA-Agent
8/53
96/1,060
9.1%
192.00
Table 1: Best full-suite result for each model and system configuration. Op. Pass reports operators with at least one passing case over the 53-operator suite. Case Pass reports accepted cases over all 1,060 cases, with missing operators counted as failures.
Figure 2: Model and workflow comparison on CANN Bench. (a) Full-suite pass rate for KOPE, CANNBot, and CUDA-Agent over 53 operators and 1,060 cases. Cases from unreported operators count as failures. (b) Positive GLM-5.2 operator-speedup fields grouped by CANN Bench difficulty and workflow. Boxes show the interquartile range, black lines show the median, whiskers extend to 1.5× the interquartile range, and points are individual measurements. The logarithmic axis marks parity at 1× . KOPE, CANNBot, and CUDA-Agent contribute 52, 37, and 13 measurements, respectively. CUDA-Agent contributes 8/3/2/0 measurements from L1 to L4, all below parity. The plot omits missing and nonpositive fields and does not include Deepseek-V4-Pro speedups.
Context Policy
Case Pass
Pass Rate
Score
Speedup
Tokens
Passive
636/1,060
60.0%
636.00
0.0382 ×
15.9B
Active
897/1,060
84.6%
2004.49
0.0661 ×
1.113B
Effect
+261
+24.6 pp
+1368.49
1.73×
−93.0%
Table 2: Active versus passive context construction. Speedup reports the geometric mean over positive fields; Tokens reports aggregate use.
GraphMemory Configuration
Case Pass
Pass Rate
Speedup
L1
L2
L3
L4
With
897/1,060
84.6%
0.659 ×
0.0866 ×
0.0344 ×
0.0182 ×
Without
585/1,060
55.2%
0.664 ×
0.0595 ×
0.0229 ×
0.00199 ×
Effect
+312
+29.4 pp
0.993×
1.455×
1.501×
9.175×
Table 3: Experience Graph Memory ablation. Case Pass and Pass Rate use the full 1,060-case suite. With reports actual per-level geometric means of positive per-operator speedup fields; Without is derived using matched positive-timing ratios. Effect reports With relative to Without.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Measured outcomes for four KOPE knowledge settings on K3. Panel (a) reports pass coverage over the common 204-case universe. Panel (b) reports geometric-mean speedup over the fixed 123-case intersection passed by every setting, preventing coverage differences from changing the performance denominator. The dashed line marks PyTorch parity. Error bars are omitted because the archive contains per-case point estimates but not the underlying repeated timing samples.
Item
Value
Hardware
SpacemiT K3 (8 × X100, riscv64, RVV 1.0)
Memory
16 GB
System
Bianbu 4.0.4 (resolute)
Python
3.12.13
PyTorch
2.8.0+spacemit.0
Triton
3.6.0+spacemit.a5
Appendix
Table 4: RISC-V experimental environment.
Setting
Passed
Rate
GM pass
GM ∩
Cold start
123/204
60.3%
0.388 ×
0.388 ×
Ascend memory
170/204
83.3%
0.142 ×
0.145 ×
K3 memory
182/204
89.2%
0.508 ×
0.514 ×
Ascend + K3
188/204
92.2%
0.758 ×
0.786 ×
Appendix
Table 5: Measured KOPE outcomes on the K3 case universe.
Figure 4: Measured passed cases by operator and KOPE knowledge setting. Each cell shows passed/total cases; color encodes the corresponding pass rate. Rows are grouped as matmul, activation, elementwise, and reduction families. The archived pass sets are nested across the four settings; this is an observed property of these runs, not an assumption used by the analysis.
LLM-based agents are increasingly used to generate GPU kernels, but they often struggle to determine when an optimization is sound because its required code state and dependencies are implicit in expert implementations. We introduce KLineage, which learns this missing "when" knowledge from expert kernels: instead of relying on forward rollouts, KLineage walks expert implementations backward through validation-gated simplifications and reverses each accepted step into a reusable optimization skill. Each skill records not only the optimization intent, but also when to apply the optimization technique, including where it applies in code, what conditions made it valid, what effect it has, and what failures its assumptions avoid. A downstream LLM materializes these skills on new code surfaces under the same compile/correctness/profile gate. This guidance on when to apply each optimization can help downstream models to generate higher-performance kernels. On five expert workloads across two NVIDIA architectures, these lineage-derived skills serve as an effective optimization curriculum, exceeding recent memory-based LLM-kernel baselines in both final kernel quality and optimization efficiency under the same fixed budget. We also demonstrate that the KLineage framework extends beyond NVIDIA GPUs to Ascend NPUs. Our code is publicly available at https://github.com/ict-agent/klineage.
Shuoming Zhang, Qiuchu Yu, Ruiyuan Xu +11
SKLP, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of New South Wales
LLM discovery and optimization systems are increasingly applied across domains, implementing a common propose-evaluate-revise loop. Such optimization or discovery progresses via context conditioning on received feedback from an environment. However, as modern LLM agents are increasingly complex in their structure, it is difficult to evaluate which components contribute the most, and when and how this exploration may fail. We answer these questions through three controlled experiments. Our findings: (1) In pure black-box optimization, LLMs act as greedy optimizers. (2) In zero-shot kernel generation, providing explicit input-size information has no measurable effect, models converge to the same kernel parameters regardless of size or temperature, as though the size instruction were invisible. Moreover, when tasked to perform kernel optimization for uncommon kernel sizes, performance sharply degrades regardless of the language used. (3) In feedback-loop kernel optimization, CUDA improves monotonically under iterative feedback, while TVM IR actively degrades, which demonstrates that kernel optimization degrades when models operate with low-density language. Our results conclude that LLMs in code optimization tasks highly depend on pretrained priors rather than provided feedback or agentic structure.
Dmitry Redko, Albert Fazlyev, Konstantin Sozykin +3
Applied AI Institute · AI Talent Hub, ITMO University · YSDA +1
Recent large language model (LLM) agents have shown promise in using execution feedback for test-time adaptation. However, robust self-improvement remains far from solved: most approaches still treat each problem instance independently, without accumulating reusable knowledge. This limitation is particularly pronounced in domain-specific languages such as Triton, which are underrepresented in LLM pretraining data. Their strict constraints and non-linear optimization landscape further make naive generation and local refinement unreliable. We propose AdaExplore, an agent framework that enables self-improvement via accumulated execution feedback for performance-critical kernel code generation through two complementary stages: failure-driven adaptation and diversity-preserving search, jointly improving correctness and optimization performance without additional fine-tuning or external knowledge. In the adaptation stage, the agent synthesizes tasks and converts recurring failures into a reusable memory of validity rules, helping subsequent generations remain within the feasible set. In the search stage, the agent organizes candidate kernels as a tree and alternates between small local refinements and larger structural regeneration, allowing it to explore the optimization landscape beyond local optima. Experiments on kernel runtime optimization benchmarks validate these gains: AdaExplore achieves 3.12x and 1.72x speedups on KernelBench Level-2 and Level-3, respectively, within 100 steps, and continues to improve with additional computation.
Weihua Du, Jingming Zhuo, Yixin Dong +9
1Carnegie Mellon University · University of Washington · 3Arm Ltd.