Efficient custom operators are essential for modern deep learning workloads, yet developing high-performance kernels for Ascend NPUs remains slow, expert-intensive, and difficult to automate. Although AI agents now approach expert performance on GPU kernel benchmarks, current models remain less capable of generating AscendC kernels for NPUs, potentially reflecting the relative scarcity of relevant training data. We thus present CAKE, a CANN-native system for deployable custom operators that combines adaptive multi-DSL routing, separate Generation and Evolution modes, and reviewed knowledge reuse through CAKE Book and a Self-Evolving Agent. On 53 CANN-Bench v1.1 operators on Ascend 950PR, CAKE's aggregated results achieve 100% case pass rate on the standard set (1,060/1,060) and 98.28% on the hidden set (4,167/4,240), with an overall mean score of 78.63 out of 100 and an average performance speedup of 1.334× over the baseline. Evolution then pushes correct operators further: EngramGateFusion cuts aggregate device time by 46.6% over 20 rounds and roughly halves the mean roofline gap. These results offer a practical path toward closing Ascend's AI-for-kernel capability gap.
Figures & tables
Figure 1: CAKE architecture. Four stacked layers cooperate to turn an operator request into a deployable, benchmark-validated Ascend operator. CAKE-Agents provides the controllers—generation, the evolution main and sub-agents, and automatic benchmark adaptation. The CAKE-Router gates each request on track–chip–operator-type feasibility, then scores the surviving code-generation tracks against the operator’s eight-dimensional structural profile, while the Skills harness supplies typed lifecycle tools for description, lowering, optimization, evaluation, and debugging. The CAKE Book is a cross-user encyclopedia maintained as a shared wiki, and CAKE-Bench is a user-defined evaluation suite—operator task specifications, test cases, precision requirements, and scoring—so that evaluation feedback is not tied to any single benchmark. A Self-Evolving Agent observes all layers: it traces, judges, and edits skills, analyses evolution runs, writes verified experience back to CAKE Book, and closes the loop with measured operator results.
Constrained variant evolution with Good/Medium/Poor sampling; CAKE Book wiki across runs and agents
Table 1: Closest Ascend-native operator systems versus CAKE. Each prior system targets one slice of operator engineering—training a model, transcompiling a DSL, optimizing an existing kernel, or supplying curated knowledge to a search—whereas CAKE delivers a deployable CANN operator and reuses measured experience across runs. AgenticCANN is the closest: it also treats the scarce AscendC corpus as the central obstacle, but its knowledge is authored by experts ahead of time rather than accumulated from measured runs. Entries are drawn from the cited papers.
Figure 2: The routing decision. L1 reduces the contract to an eight-dimensional feature vector and a difficulty; L0 removes infeasible tracks before any scoring and abstains rather than guessing when nothing is feasible; L2 scores the surviving tracks under Equation 3 and decides between exploiting the best estimate and measuring an eligible contender; L3 records the outcome, which moves the ability estimates that L2 will use next time.
Operator
φ
b
pA
pB
pC
Chosen
Decided by
gelu
(0,0,1,0,2,0,0,0)
−2.18
0.694
0.463
0.613
DSL-A
baseline ability
gather
(0,0,1,0,1,2,0,0)
−1.40
0.239
0.252
0.366
DSL-C
M (data movement)
conv_2d
(1,0,3,0,2,0,0,2)
+1.47
0.165
0.314
0.218
DSL-B
B (blocking)
mha
(3,3,3,3,2,0,1,2)
+7.57
0.257
0.168
0.101
DSL-A
S (scheduling depth)
Table 2: Four routing decisions on the 910B under Equation 3 (FrontierModel-A backbone, pure exploitation). φ is printed in dimension order (T,S,B,R,D,M,F,C) and b is centered on the corpus mean. Columns pA , pB , pC are the predicted normalized scores of the three DSL tracks. The last column names the dimension whose ability coefficients decide the ordering.
Figure 3: Fitted per-dimension abilities θd for the three DSL tracks under one backbone. Axes are the eight feature dimensions of Section 3 . The tracks are nearly indistinguishable on tiling ( T ), reduction ( R ), and dtype spread ( D ), and separate on blocking ( B , where DSL-B reaches +2.83 ), data movement ( M , where DSL-A is −1.69 against DSL-C’s −0.31 ), and scheduling depth ( S , where DSL-A is +2.45 against DSL-C’s +0.91 ). These gaps are what make the routing ordering operator-dependent rather than fixed.
Figure 4: Roofline ceiling model (Eq. 4 ). Each case is placed by its arithmetic intensity (AI); the binding pipeline gives the roof Tmin , and the headroom (gap from the measured point up to the roof) bounds how much evolution can still gain. A variant that reaches its roof ( headroom≈1 ) is converged.
Figure 5: The CAKE-evo loop. The analysis layer (blue) sets the ceiling target and bottleneck-driven variant assignment; the evolutionary stages (green) spawn, measure, and tier variants; AIC injects profiling-grounded guidance; the roofline headroom provides the stop test; PMR compounds knowledge into rules.
Multiple TilingKeys with a specialized kernel per shape regime
Table 4: Bottleneck-signature → strategy routing across the classes in Table 3 . The roofline verdict and fusion analysis select which optimization family each variant explores.
Figure 7: CAKE’s aggregated results by difficulty level on Ascend 950PR. Levels 1–4 contain 8, 16, 21, and 8 operators. Left: mean operator score, with a maximum of 100. Right: case pass rate, computed over 20 standard or 80 hidden cases per operator. The same operator grouping is used for both sets.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Membase
Regbase
Residency
LocalTensor in UB
RegTensor in VREG
API
whole-tensor AscendC::*
AscendC::Reg::* / MicroAPI
Addressing
block-strided aligned
can be unaligned
Masking
count argument
explicit MaskReg predicate
Movement
DataCopy (TPipe queue)
Load(Un)Align / Store(Un)Align
Appendix
Table 5: Membase and Regbase operand models: data residency, addressing, masking, and transfer operations.
Figure 8: Target and path routing. check_device.sh reports the chip generation, which admits the 910 lane or the two 950-specific lanes. On the 950, operators with data-dependent non-contiguous access take SIMT; the remainder use Regbase when the register-level DSL covers the kernel and fall back to membase when it does not. The SIMT model-selection questions Q1–Q6 are given in Appendix A.5 .
Figure 9: @vf Regbase kernel skeleton. In this pattern, narrow I/O (bf16/int8) is widened to RegTensor<float> on load via LoadDist and packed back on store; computation uses float registers.
Figure 10: SIMT model selection. Questions Q1–Q2 gate entry to the SIMT path; Q3–Q6 choose among three programming models. Model A is a CUDA-like global kernel; Model B bridges SIMT threads into the AscendC entry via asc_vf_call ; Model C adds a simd_vf stage that offloads vectorizable arithmetic to the SIMD pipe after SIMT has performed the irregular access.
Ascend C operator optimization is critical for NPU (Neural Processing Unit) inference performance but requires deep hardware expertise. While large language models (LLMs) have shown promise in automated CUDA kernel generation, the fundamentally different programming model of Ascend C introduces unique challenges that remain unexplored. In this paper, we propose AgenticCANN, a knowledge-augmented agentic evolution framework specifically tailored for automated Ascend C operator synthesis in low-corpus NPU environments. To overcome the severe platform knowledge deficit on unfamiliar hardware, AgenticCANN incorporates a knowledge-orchestrated generation system that delivers structured, multi-level domain insights across the development lifecycle to resolve the upstream feasibility bottleneck. Building on this foundation, it features a stage-adaptive agentic evolution strategy that dynamically aligns LLM interaction modes with specific generation and evolution phases, balancing high-exploration candidate discovery with high-convergence performance tuning. Extensive experiments on Huawei Ascend 910B across six operators spanning five pattern categories demonstrate that our method achieves 90 to 100 percent feasibility on elementwise and normalization operators, 56% on fusion operators, and up to 6.65× speedup on 1B Pangu model inference kernels. Further analysis reveals that knowledge injection monotonically improves feasibility from 57% to 86% on elementwise operators, demonstrating its general rather than operator-specific benefit.
Junhao Qiu, Zidong Wang, Yansong Sun +3
Department of Computer Science, City University of Hong Kong · Hong Kong Yuwin Technology Co., Ltd
LLM agents have become capable kernel engineers on mainstream GPU stacks, but they struggle on emerging accelerators such as Ascend NPUs, whose programming model is barely represented in pretraining data. Recent NPU agents address this gap largely through experience: they accumulate memory from their own generation and optimization trials and reuse it on later tasks. We argue that experience alone is not enough for kernel generation. An agent can only remember what it has tried, and it rarely tries a mechanism it does not know. As a result, such memory offers little help when a new operator calls for a different mechanism, and even familiar operators raise unseen situations as optimization proceeds. We present ASCENDEVOLVE, which grounds every step of an evolutionary kernel search in a large external knowledge base built from platform documentation, toolkit headers, and operator repositories. Because this corpus far exceeds any context window, ASCENDEVOLVE indexes it and routes each agent to a small, task-specific slice of evidence instead of the corpus itself. The retrieved knowledge seeds diverse implementation strategies, guides repair when a candidate fails, and constrains how strong candidates are recombined. On 53 Ascend C operators from CANN-Bench spanning four difficulty levels, ASCENDEVOLVE achieves an overall score of 71.39, improving over Reflexion and OpenEvolve by 11.6% and 17.2%.
Xi Chen, Ming Shen, Haotian Yan +2
Shenzhen Loop Area Institute (SLAI) · Guangzhou University · Shanghai Jiao Tong University +2
Optimizing AscendC (Ascend C) operators for Ascend NPUs is difficult for two reasons. First, unlike CUDA, the ecosystem offers few public kernels to learn from. Second, performance depends on a coupled two-part implementation: a host-side tiling program that controls data movement and a kernel program that schedules and pipelines computation. We present AscendOptimizer, an episodic agent that builds missing optimization knowledge from execution itself. For kernel optimization, AscendOptimizer rewinds strong implementations by removing optimizations in a controlled way, then keeps the changes whose removal measurably hurts performance as reusable experience for later rewriting. For host-side optimization, it runs profiling-in-the-loop evolutionary search to find valid, fast tiling and data-movement configurations directly from hardware feedback. This combination lets the agent improve kernel structure and host-side scheduling together. On a benchmark of 101 real AscendC operators, AscendOptimizer achieves a 1.21x geometric-mean speedup over the open-source baseline, and 53.47% of operators run faster than their references. Given a same budget of evaluations per operator, AscendOptimizer consistently outperforms Best-of-N sampling and OpenEvolve in terms of geometric mean speedup, fast_p tail speedup ratios, and overall optimization progress across varying budgets.
Jiehao Wu, Zixiao Huang, Wenhao Li +3
School of Computer Science and Technology, East China Normal University · School of Computer Science and Technology, Tongji University · Shanghai University of International Business and Economics +3