GPU Kernel Optimization

GPU: Graphics Processing Unit

Ask about this topic

Overview

GPU kernel optimization studies how to implement individual operations—such as matrix multiplication, attention, reductions, and sparse computations—so they use GPU memory and compute units efficiently. Common approaches include tiling and fusion, low-precision arithmetic, compiler scheduling, and domain-specific languages such as CUDA, Triton, and CuTe. Autotuners and LLM-based agents increasingly search these design spaces using profiling, execution feedback, evolutionary search, or reinforcement learning, with benchmarks such as KernelBench measuring correctness and speed. A central challenge is that gains often depend on the workload and GPU architecture: generated kernels may pass correctness checks yet lose to vendor libraries, regress on unseen shapes, or yield little end-to-end improvement.

Latest papers

CardsList

211 papers28 papers in the last four weeks, up 56% on the four weeks before. 0.2% of all new papers.

  1. The Endgame of FSDP Communication: Layout-Aware Collectives on Copy Engines, RDMA, and NVSwitch

    Oct 28, 2026Zejian Wang, Yifan Zhao, Kaihui Ren +11Communication-Efficient Distributed TrainingDistributed Training

    KlingAI Research, Shanghai, China · KlingAI Research, Beijing, China

  2. Evolving Agent Teams

    Oct 27, 2026Shiyi Cao, Ziming Mao, Dacheng Li +2Multi-Agent CodingSelf-Evolving Agents

    University of California, Berkeley

  3. God Does Not Play Dice: Enforcing Batch Invariance in Agentic Kernel Generation

    Oct 27, 2026Youwei Bao, Chengpeng Wang, Simin ChenGPU Kernel GenerationGPU Kernel Optimization

    STAR Lab, George Mason University · National University of Singapore

  4. Less is more: learning what to profile for GPU kernel optimization

    Oct 27, 2026Yihan Wang, Peiyu Liu, Runyu Chen +1GPU Kernel OptimizationLarge Language Model-Guided Optimization

    Renmin University of China · University of International Business and Economics

  5. One Megakernel is Not Enough: Predicting Where to Cut for Streaming Inference

    Oct 26, 2026Aditya Chatterjee, Anand Joseph X M, Sanjay Singh ChauhanEfficient Neural Network InferenceInference-Time Optimization

    NVIDIA, Bangalore, India · NVIDIA, Hyderabad, India · NVIDIA, Pune, India

  6. Evaluating the Applicability of LLM Inference Optimizations on Apple Silicon

    Oct 22, 2026Kyrylo Yemets, Andrii Yaroshevych, Victor Muryn +5LLM InferenceOn-Device Language Model Inference

    MacPaw AI&Research, Kyiv, Ukraine · Department of Artificial Intelligence, Lviv Polytechnic National University, Lviv, Ukraine · Institute of Mathematics of NAS of Ukraine, Kyiv, Ukraine

  7. GPU-Accelerated Construction of the Gabriel Graph: Parallelization Strategies and Applications to Large-Margin Classification and Graph-Based Clustering

    Oct 20, 2026Diego Sanches Nere dos Santos, Luiz Carlos Bambirra TorresGraph ConstructionGPU Acceleration

    Department of Computing and Systems, Institute of Exact and Applied Sciences, Federal University of Ouro Preto (UFOP), João Monlevade, MG, Brazil · Department of Computing, Institute of Exact and Biological Sciences, Federal University of Ouro Preto (UFOP), Ouro Preto, MG, Brazil

  8. CusADi2: Latency-Optimized Batch Computation of Robot Kinematics, Jacobians, and Hessians on the GPU

    Oct 18, 2026John Viljoen, Lasse Peters, Masayoshi Tomizuka +1GPU AccelerationRobot Kinematics

    Department of Mechanical Engineering, University of California, Berkeley, CA, USA.

  9. Conjugate Compilation: Self-Improving Agentic Co-Compilation for Speculative Decoding

    Oct 18, 2026Annabelle Sujun Tang, Rohan Mahapatra, Christopher Priebe +4LLM Inference AccelerationSpeculative Decoding

    University of California, San Diego · Massachusetts Institute of Technology · Systalyze +2

  10. FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper

    Oct 16, 2026Wei WangEfficient Transformer InferenceGPU Kernel Optimization

  11. Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs

    Oct 15, 2026Haoyu Zhang, Arshad Rahman, Adnan Aziz +55Hardware-Software Co-DesignHeterogeneous Computing

  12. How to Train Your Little Dragon: Teaching a Small Language Model to Construct and Refine GPU Optimization Scripts

    Oct 14, 2026Annabelle Sujun Tang, Wei Tang, Songyao Jiang +7Compiler OptimizationCode Optimization

    University of California, San Diego · Amazon Annapurna Labs

  13. LunaTP: Enabling Efficient and Scalable Tensor Product for Equivariant MLIPs

    Oct 13, 2026Yuanchang Zhou, Mingzhen Li, Chen Wang +3Machine Learning Interatomic PotentialsGPU Kernel Optimization

    State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences

  14. FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention

    Oct 13, 2026Guoxia Wang, Qianyue He, Siming Wu +6Context ParallelismAttention Mechanisms

    Baidu Inc.

  15. GPU Goldmines: Tunable WebGPU Kernels for Edge LLM Inference

    Oct 12, 2026Rithik Sharma, Reese Levine, Zheyuan Chen +3GPU Kernel OptimizationLLM Inference Acceleration

    UC Santa Cruz · EPFL · Microsoft Research

  16. PageWeaver: KV-Guided Query Unions for Sparse Attention

    Oct 8, 2026Zhiyuan Li, Zihan Li, Zefang Yuan +2Sparse AttentionGPU Kernel Optimization

    SenseTime

  17. QUILT: Rethinking Sparse-Attention Prefill through Shared Query Execution

    Oct 8, 2026Zhenduo Zhao, Qihui Zhou, Mingcong Song +7LLM Inference AccelerationSparse Attention

    Huawei Technologies Co., Ltd. China

  18. Cleave: Scaling Tensor Program Optimization via Decoupled Algebraic Search and Operator Scheduling

    Oct 6, 2026David Pissarra, Jinkun Lin, Haitian Jiang +2Compiler OptimizationGPU Kernel Optimization

    New York University New York, USA

  19. FlashCart: Fast Cartesian Tensor Products for Equivariant Interatomic Potentials

    Oct 5, 2026Viktor Zaverkin, Payman Goodarzi, Sergey V. Sukhomlinov +4Machine Learning Interatomic PotentialsGPU Kernel Optimization

    INM – Leibniz Institute for New Materials · Saarland University · German Research Center for Artificial Intelligence (DFKI) +2

  20. T-CCL: Resource Efficient and Performant Collective Communication using Tensor Memory Accelerator

    Oct 5, 2026Keyvan Dadashzadeh, Yuehong Zhou, Minyu Cui +1GPU Kernel OptimizationLLM Inference Acceleration

    Chalmers University of Technology and University of Gothenburg Gothenburg, Sweden · Linnaeus University Växjö, Sweden

  21. TurboPairFormer: Fast and Stable Protein Folding Model Training with an Optimized Triangle Attention Kernel

    Oct 5, 2026Yide Ran, Chelsea Lowman, Jan Domański +6Protein Structure PredictionGPU Kernel Optimization

    Stevens Institute of Technology · Lambda · Open Molecular Software Foundation

  22. MetaKernelBench: Measuring GPU Kernel Knowledge Transfer Beyond Code

    Oct 4, 2026Xueyi Chen, Shiyu Liu, Xin Jin +5GPU Kernel OptimizationAI Agent Benchmarks

    Westlake University · Shanghai Jiao Tong University · Zhejiang University

  23. Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU

    Oct 2, 2026Jafar Badour, Maurice van Keulen, Elena MocanuDynamic Sparse TrainingSparse Neural Networks

    University of Twente

  24. D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?

    Oct 2, 2026Daifeng Li, Huiqiang Jiang, Chengruidong Zhang +6GPU Kernel GenerationGPU Kernel Optimization

    HKUST · Alibaba Group · USTC

  25. FlashSinkhorn 2: Block-Sparse Entropic Optimal Transport

    Oct 1, 2026Felix X. -F. Ye, Yu Chin Fabian Lim, Naigang Wang +1Optimal TransportGPU Kernel Optimization

    Department of Mathematics & Statistics, University at Albany, Albany, NY, USA · IBM T. J. Watson Research Center, Yorktown Heights, NY, USA

  26. AI as a Compiler: Compiling Triton kernels without the Triton compiler

    Sep 29, 2026François Costa, Charly Castes, Thomas Bourgeat +1GPU Kernel OptimizationLarge Language Model-Guided Optimization

    Stanford University · EPFL

  27. GEM-KMeans: Memory-Efficient and Accurate Clustering on Massive Scale with GPU Optimization

    Sep 28, 2026Peng Xu, Nihar Koganti, Volodymyr Kindratenko +1ClusteringNon-Negative Matrix Factorization

    Department of Statistics, University of Illinois Urbana-Champaign · Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign · National Center for Supercomputing Applications, University of Illinois Urbana-Champaign +3

  28. Mixture-of-Kittens: MoE Megakernel for NVL72s

    Sep 28, 2026Stuart H. Sul, Nash Brown, Henry Wildermuth +3Distributed TrainingMixture-of-Experts Models

    Department of Computer Science, Stanford University · Cursor Research

  29. GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulation

    Sep 28, 2026Yuchen Sun, Jinjin He, Sinan Wang +1GPU Kernel OptimizationBenchmark Design

    Georgia Institute of Technology

  30. Hardware-Aware Features for CUTLASS Kernel Selection

    Sep 28, 2026Shriram Chandran, Dominic Rinderer, Yakup Budanaz +3GPU Kernel OptimizationLearning to Rank

    ETH Zurich Zurich, Switzerland