We present CompoundingKernel, a self-improving agent system that writes GPU kernels by search. As of 2026-11-11, its generated kernels rank #1 on the public L1, L2 and Quantization boards of SOL-ExecBench (NVIDIA B200). They have held #1 on L1 since 2026-10-27, albeit a short loss on 2026-11-07. All 235 problems have a site-verified correct kernel; on the 209 public-board problems they are 4.40x faster than the benchmark's reference baseline. The search starts from zero: agents get each problem's PyTorch definition but no reference kernels or official scores. A per-problem archive outside the model holds every correct kernel and its lineage. An LLM controller reads it, picks a parent or a new design, and gives each parallel child agent one direction. Checks outside the model decide correctness. At equal wall time, it beat a coded UCB style tree on 35 of 38 problems by making 2.4x as many correct attempts. Over a thousand agents ran at once, and one warm restart fix raised their evaluation rate 3.6x. We find that the rate of reliable measurements, not the number of agents, limits how far such a search scales. Our code and kernels are available on URL.
Figures & tables
Figure 1: Four loops, one search per problem: \circled 1 a child writes one change, \circled 2 the LLM controller picks what next, \circled 3 agents fix the harness at warm restarts, \circled 4 people decide sends. Official scores never enter the search; people use the standings outside it, to choose sends and to place compute.
Figure 2: The archive is the agents’ only shared memory: they read it with read-only SQL, and only evaluation appends a correct kernel. Most tables hang off one scored kernel, whose parent link is the tree edge.
Table 4: Official results by board. Ranks and public SOL means come from the public boards at 2026-11-11, and per-problem counts from the per-problem boards at the same date. Board screenshots are in Appendix .