AsmEvo: Agentic Assembly-Level Optimization of AMD GPU Kernels with Functional Equivalence Verification

Ji Liu; Puyuan Yang; Rongzhang Zheng; Fan Wang; Jinglin Wang; Muhammad A. Awad; Mortis Huang; Andy Chang

cs.CLAug 21, 2026

AsmEvo: Agentic Assembly-Level Optimization of AMD GPU Kernels with Functional Equivalence Verification

Authors: Ji Liu, Puyuan Yang, Rongzhang Zheng, Fan Wang, Jinglin Wang, Muhammad A. Awad, Mortis Huang, Andy Chang,

Organizations: Advanced Micro Devices, Inc. (AMD) · Southern University of Science and Technology, Shenzhen, China

Abstract

High-performance ML systems increasingly rely on GPU kernels whose editable source is unavailable, generated, or too distant from final machine code to expose remaining optimizations. Existing LLM kernel optimizers and autotuners mainly operate on CUDA, Triton, HIP, or tensor-program source and validate against reference implementations. We study a stricter setting: optimizing an already compiled AMDGPU code object, where the deployed binary is the only behavioral oracle. We present AsmEvo, an agentic assembly-level optimizer for AMD GPU kernels. Given an AMDGPU code object K0, AsmEvo reconstructs a reassemblable representation, proposes low-level edits with a long-horizon agent, rebuilds an ABI-preserving optimized object, and accepts candidates only after differential verification against K0 under identical launches. AsmEvo combines code-object recovery, metadata-aware rebuilding, profiling-guided hot-window editing, correctness-gated timing, and conservative in-place patch fallback. We conduct extensive experiments with AsmEvo on various AMD GPU kernels. On MI308X, AsmEvo improves 29 of 30 selected KernelBench kernels, reaching 1.35x geometric-mean and 3.88x maximum speedup. On MI300X production workloads, it improves all evaluated AITer binaries and vLLM/SGLang Triton assembly kernels, reaching 1.09x/1.31x and 1.18x/1.34x geometric-mean/maximum speedups, respectively, while preserving functional equivalence.

Figures & tables

Figure 1: AsmEvo combines faithful code-object recovery, metadata-aware ABI-preserving rebuild, and correctness-before-performance evaluation against the original $K_{0}$ . A modular LLM driver proposes edits through multi-start search, verified composition, and anti-stall redirection, while only verified, latency-improving candidates enter the lineage.

Source	Size	Coverage
KernelBench L1	15	activations, losses, reductions/norms, depthwise conv, matmul, pooling
KernelBench L2	15	fused GEMM/matmul and conv pipelines, normalization, activations, pooling, reductions
AITer	4	code-object MoE, FP8 FMHA, BF16 FMHA, BF16 FMHA backward
Triton JIT	4	vLLM/SGLang A8W8 GEMM, MoE, MoE LoRA, AWQ GEMM

Table 1: Kernel sources, counts, and operator coverage.

Split	Kernels	Improved	Geo.	Max	Attempts/K	Commits/K	GPU	Best case
KernelBench L1	15	15 (100%)	1.56	3.88	119.3	7.2	MI308X	Depthwise Conv2D
KernelBench L2	15	14 (93%)	1.17	2.55	156.4	6.8	MI308X	Matmul+Swish+Sum+GroupNorm
All KernelBench	30	29 (97%)	1.35	3.88	137.8	7.0	MI308X	Depthwise Conv2D

Table 2: KernelBench performance under the default AsmEvo configuration. Speedups are measured against the original compiled code object and reported only after functional-equivalence verification.

Source	Kernel	Speedup
AITer	aiter_fmoe_fp8_blockscale subgu256	1.310
AITer	aiter_fmha_hd128_fp8_causal	1.007
AITer	new_aiter_fmha_hd192x128 bf16_causal	1.025
AITer	new_aiter_fmha_bwd_hd128 bf16_causal_a32	1.049
vLLM	gemm_a8w8_blockscale	1.234
SGLang	sglang_fused_moe lora_kernel	1.025

Table 3: Verified per-kernel speedups over the original production binaries on MI300X. Only binary-level AITer and assembly-level Triton results are included. The family geometric means are

1.09\times

for AITer and

1.18\times

for Triton JIT. Triton JIT HSACOs are collected from vLLM and SGLang and evaluated using captured production dispatches.

Figure 2: Verified AsmEvo speedups over the original code objects (

1.00\times

) for 30 selected KernelBench kernels, ranked by speedup. The lower panel shows each kernel’s operator composition. All reported candidates pass functional-equivalence verification.

GPU Kernel Optimization

GPU kernel optimization improves the speed and memory efficiency of computations by tailoring code to GPU hardware and workloads.

See all 209 papers

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization
Mar 12, 2026Nina Wiedemann, Quentin Leboutet, Michael Paulitsch +2GPU Kernel Optimization GPU Acceleration
Intel Corporation
AMDKernelVault: Large-Scale Datasets and Agentic Training for AMD GPU Kernel Optimization
Sep 11, 2026Ji Liu, Saptarshi Majumder, Yiqing Huang +14GPU Kernel Optimization Large Language Model-Guided Optimization
Advanced Micro Devices, Inc.
ARGUS: Agentic GPU Optimization Guided by Data-Flow Invariants
Apr 16, 2026Haohui Mai, Xiaoyan Guo, Xiangyun Ding +7GPU Kernel Optimization GPU Acceleration
UC Riverside5 · UCAS4 · Tsinghua University2 +2

Mar 12, 2026J/K move · Enter open · S save

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization

Nina Wiedemann, Quentin Leboutet, Michael Paulitsch, Diana Wofk+1

Intel Corporation

GPU Kernel Optimization GPU Acceleration Evolutionary Optimization Large Language Model-Guided Optimization LLM-Based Program Synthesis GPU Kernel Generation+1

Open paper AskPDF arXiv

GPU kernel optimization challenges LLMs beyond standard coding tasks, as it requires an understanding of hardware architecture, parallel computing optimization strategies, and profiling outputs. However, most existing approaches leveraging LLMs for kernel generation apply standard prompting and feedback loops, considering hardware only through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the space of GPU kernels through (1) MAP-Elites quality diversity search with kernel-specific behavioral dimensions to sustain exploration; (2) meta-prompt evolution that co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) a template-based parameter optimization approach to tune kernels to inputs and hardware. We evaluate this framework on Kernel-Bench, robust-kbench and custom tasks, generating SYCL kernels as a cross-platform GPU programming paradigm, and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods and achieves an average speedup of 2.3 on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, allowing quick benchmarking and featuring a flexible user input layer to support kernel generation for a wide range of real use cases beyond benchmarking.

Abstract

GPU Kernel Optimization

Explore similar work

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization

AMDKernelVault: Large-Scale Datasets and Agentic Training for AMD GPU Kernel Optimization

ARGUS: Agentic GPU Optimization Guided by Data-Flow Invariants