cs.AIOct 22, 2026
SaveCANNBot-CAKE: Collaborative Agentic Kernel Evolution for Ascend NPU
Organizations: Huawei
Abstract
Efficient custom operators are essential for modern deep learning workloads, yet developing high-performance kernels for Ascend NPUs remains slow, expert-intensive, and difficult to automate. Although AI agents now approach expert performance on GPU kernel benchmarks, current models remain less capable of generating AscendC kernels for NPUs, potentially reflecting the relative scarcity of relevant training data. We thus present CAKE, a CANN-native system for deployable custom operators that combines adaptive multi-DSL routing, separate Generation and Evolution modes, and reviewed knowledge reuse through CAKE Book and a Self-Evolving Agent. On 53 CANN-Bench v1.1 operators on Ascend 950PR, CAKE's aggregated results achieve case pass rate on the standard set (1,060/1,060) and on the hidden set (4,167/4,240), with an overall mean score of 78.63 out of 100 and an average performance speedup of over the baseline. Evolution then pushes correct operators further: EngramGateFusion cuts aggregate device time by over 20 rounds and roughly halves the mean roofline gap. These results offer a practical path toward closing Ascend's AI-for-kernel capability gap.
Figures & tables
Figure 1: CAKE architecture. Four stacked layers cooperate to turn an operator request into a deployable, benchmark-validated Ascend operator. CAKE-Agents provides the controllers—generation, the evolution main and sub-agents, and automatic benchmark adaptation. The CAKE-Router gates each request on track–chip–operator-type feasibility, then scores the surviving code-generation tracks against the operator’s eight-dimensional structural profile, while the Skills harness supplies typed lifecycle tools for description, lowering, optimization, evaluation, and debugging. The CAKE Book is a cross-user encyclopedia maintained as a shared wiki, and CAKE-Bench is a user-defined evaluation suite—operator task specifications, test cases, precision requirements, and scoring—so that evaluation feedback is not tied to any single benchmark. A Self-Evolving Agent observes all layers: it traces, judges, and edits skills, analyses evolution runs, writes verified experience back to CAKE Book, and closes the loop with measured operator results.
| System | Core approach | Input | Delivered unit | Search & cross-run knowledge |
|---|---|---|---|---|
| AscendKernelGen [ 1 ] | Train a domain model (Ascend-CoT data, SFT/RL) | NL requirement | AscendC kernel scored on a benchmark | None; experience fixed in model weights |
| AscendCraft [ 7 ] | DSL generation then constrained lowering passes | NL requirement | AscendC kernel | None; reuse via category expert examples |
| AscendOptimizer [ 3 ] | Episodic agent: optimization rewind + tiling search | Existing correct AscendC operator | Optimized kernel and host tiling | Host-tiling evolution; per-benchmark experience bank |
| AgenticCANN [ 4 ] | Six-level expert knowledge taxonomy injected by phase; stage-adaptive agent scheduling | NL requirement | Compiled and numerically verified AscendC kernel | Candidate evolution; knowledge authored once, not written back from runs |
| CAKE (ours) | Agentic workflow over staged, inspectable artifacts | NL requirement or AscendC baseline | Deployable CANN operator (build, install, bind, evaluate) | Constrained variant evolution with Good/Medium/Poor sampling; CAKE Book wiki across runs and agents |
Table 1: Closest Ascend-native operator systems versus CAKE. Each prior system targets one slice of operator engineering—training a model, transcompiling a DSL, optimizing an existing kernel, or supplying curated knowledge to a search—whereas CAKE delivers a deployable CANN operator and reuses measured experience across runs. AgenticCANN is the closest: it also treats the scarce AscendC corpus as the central obstacle, but its knowledge is authored by experts ahead of time rather than accumulated from measured runs. Entries are drawn from the cited papers.
Figure 2: The routing decision. L1 reduces the contract to an eight-dimensional feature vector and a difficulty; L0 removes infeasible tracks before any scoring and abstains rather than guessing when nothing is feasible; L2 scores the surviving tracks under Equation 3 and decides between exploiting the best estimate and measuring an eligible contender; L3 records the outcome, which moves the ability estimates that L2 will use next time.
| Operator | Chosen | Decided by | |||||
|---|---|---|---|---|---|---|---|
| gelu | (0,0,1,0,2,0,0,0) | 0.694 | 0.463 | 0.613 | DSL-A | baseline ability | |
| gather | (0,0,1,0,1,2,0,0) | 0.239 | 0.252 | 0.366 | DSL-C | (data movement) | |
| conv_2d | (1,0,3,0,2,0,0,2) | 0.165 | 0.314 | 0.218 | DSL-B | (blocking) | |
| mha | (3,3,3,3,2,0,1,2) | 0.257 | 0.168 | 0.101 | DSL-A | (scheduling depth) |
Table 2: Four routing decisions on the 910B under Equation 3 (FrontierModel-A backbone, pure exploitation). is printed in dimension order and is centered on the corpus mean. Columns , , are the predicted normalized scores of the three DSL tracks. The last column names the dimension whose ability coefficients decide the ordering.
Figure 3: Fitted per-dimension abilities for the three DSL tracks under one backbone. Axes are the eight feature dimensions of Section 3 . The tracks are nearly indistinguishable on tiling ( ), reduction ( ), and dtype spread ( ), and separate on blocking ( , where DSL-B reaches ), data movement ( , where DSL-A is against DSL-C’s ), and scheduling depth ( , where DSL-A is against DSL-C’s ). These gaps are what make the routing ordering operator-dependent rather than fixed.
Figure 4: Roofline ceiling model (Eq. 4 ). Each case is placed by its arithmetic intensity (AI); the binding pipeline gives the roof , and the headroom (gap from the measured point up to the roof) bounds how much evolution can still gain. A variant that reaches its roof ( ) is converged.
Figure 5: The CAKE-evo loop. The analysis layer (blue) sets the ceiling target and bottleneck-driven variant assignment; the evolutionary stages (green) spawn, measure, and tier variants; AIC injects profiling-grounded guidance; the roofline headroom provides the stop test; PMR compounds knowledge into rules.
| Class | Scope | Representative strategies |
|---|---|---|
| D Multi-dtype | kernel | mixed precision; template / TilingKey dtype dispatch |
| P Performance | kernel | double buffering; adaptive tiling; pipeline sync; 32B alignment; vectorized copy; matmul scheduling; L1/L2 cache hierarchy |
| A Accuracy | kernel | FP32 intermediate; Welford mean/var; rounding mode; softmax stability; high-precision rsqrt |
| F Fusion | cross-op | elementwise-chain anti-spill; norm epilogue; matmul epilogue (CV) |
| H Host-level | host | TilingKey priority-chain dispatch; multi-core load balance; L2/L1 memory tiering |
Table 3: Strategy classes. D/P/A are kernel-level; F and H raise the altitude to cross-operator fusion and host-side, shape-driven dispatch.
| Bottleneck signature | Routed strategies |
|---|---|
| memory-bound fusable chain | Fusion to keep intermediates off GMEM; double buffering; alignment; vectorized copy; memory-tier selection |
| compute-bound (vector) | Adaptive tiling; small-dimension row merging; algorithm selection; FP32 intermediates where precision requires them |
| compute-bound (cube) | Matmul scheduling (SplitK/StreamK); L1/L2 cache optimization; matmul-epilogue fusion |
| instruction / scalar-bound | Algorithm redesign; specialized AscendC APIs; narrow synchronization |
| load imbalance (straggler core) | Host-side dispatch and multi-core load balancing |
| shape regimes diverge across cases | Multiple TilingKeys with a specialized kernel per shape regime |
Table 4: Bottleneck-signature strategy routing across the classes in Table 3 . The roofline verdict and fusion analysis select which optimization family each variant explores.
Figure 7: CAKE’s aggregated results by difficulty level on Ascend 950PR. Levels 1–4 contain 8, 16, 21, and 8 operators. Left: mean operator score, with a maximum of 100. Right: case pass rate, computed over 20 standard or 80 hidden cases per operator. The same operator grouping is used for both sets.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Membase | Regbase | |
| Residency | LocalTensor in UB | RegTensor in VREG |
| API | whole-tensor AscendC::* | AscendC::Reg::* / MicroAPI |
| Addressing | block-strided aligned | can be unaligned |
| Masking | count argument | explicit MaskReg predicate |
| Movement | DataCopy (TPipe queue) | Load(Un)Align / Store(Un)Align |
Table 5: Membase and Regbase operand models: data residency, addressing, masking, and transfer operations.
Figure 8: Target and path routing. check_device.sh reports the chip generation, which admits the 910 lane or the two 950-specific lanes. On the 950, operators with data-dependent non-contiguous access take SIMT; the remainder use Regbase when the register-level DSL covers the kernel and fall back to membase when it does not. The SIMT model-selection questions Q1–Q6 are given in Appendix A.5 .
Figure 9: @vf Regbase kernel skeleton. In this pattern, narrow I/O (bf16/int8) is widened to RegTensor<float> on load via LoadDist and packed back on store; computation uses float registers.
Figure 10: SIMT model selection. Questions Q1–Q2 gate entry to the SIMT path; Q3–Q6 choose among three programming models. Model A is a CUDA-like global kernel; Model B bridges SIMT threads into the AscendC entry via asc_vf_call ; Model C adds a simd_vf stage that offloads vectorizable arithmetic to the SIMD pipe after SIMT has performed the irregular access.
More in this topic
Agentic Code Generation
Agentic code generation uses AI agents to plan, write, test, and revise executable programs with tools and feedback.
See all 125 papers