GPU Kernel Optimization

GPU: Graphics Processing Unit

Overview

GPU kernel optimization studies how to implement individual operations—such as matrix multiplication, attention, reductions, and sparse computations—so they use GPU memory and compute units efficiently. Common approaches include tiling and fusion, low-precision arithmetic, compiler scheduling, and domain-specific languages such as CUDA, Triton, and CuTe. Autotuners and LLM-based agents increasingly search these design spaces using profiling, execution feedback, evolutionary search, or reinforcement learning, with benchmarks such as KernelBench measuring correctness and speed. A central challenge is that gains often depend on the workload and GPU architecture: generated kernels may pass correctness checks yet lose to vendor libraries, regress on unseen shapes, or yield little end-to-end improvement.

Latest papers

CardsList

239 papers33 papers in the last four weeks, up 3% on the four weeks before. 0.3% of all new papers.

  1. A Fast, Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication

    Nov 18, 2026Jiakun Yan, Shiksha Patel, David Sidler +3Computation OffloadingGPU Kernel Optimization

  2. Compiling Specialized GPU-Initiated Networking

    Nov 13, 2026Javid Baydamirli, Ismayil Ismayilov, Kaan Oktay +1GPU Kernel Optimization

  3. AscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs

    Nov 13, 2026AI Training Platform Team, Bingwei Zhang, Jing Li +24NPU AccelerationGPU Kernel Optimization

  4. MatrixSplat: Matrix-Parallel and Feature-Scalable 3D Gaussian Splatting Rendering on GPUs

    Nov 11, 2026Yangming Zhang, Siyi Wu, Jian Wang +53DGS Acceleration3D Gaussian Splatting

  5. KernelMosaic: An Open Dataset and Agentic Framework for CUDA Kernel Generation

    Nov 10, 2026Ningyuan Xi, Yuhang Lai, Ning MiaoGPU Kernel GenerationGPU Kernel Optimization

  6. FAME-QP: A GPU-Accelerated Quadratic-Programming Solver for Real-Time Model Predictive Control

    Nov 10, 2026Santosh Mohan Rajkumar, Debdipta Goswami, Stephanie StockarQuadratic ProgrammingModel Predictive Control

  7. CAFIN: CUDA-accelerated Adaptive Fisher Information Matrix Network Optimization via GPU-Native Kernel Fusion

    Nov 10, 2026Maryam Arshad, Muhammad Khuram ShahzadNeural Network OptimizationGPU Kernel Optimization

  8. Mitigating SM-Level Load Imbalance in 3D Gaussian Splatting on GPUs

    Nov 9, 2026Junha Ryu, Hoi-Jun Yoo3D Gaussian Splatting3DGS Acceleration

  9. Correct First, Fast Second: What a Local 7B Code Model Can and Cannot Do for GPU Kernel Optimization

    Nov 9, 2026Jui-Chien Tsou, Po-Yi WuGPU Kernel GenerationLarge Language Model-Guided Optimization

  10. FastGRT: Accelerating Training of 3D Gaussian Ray Tracing

    Nov 8, 2026Meet Banthia, Xiaoteng Liu, Hexu Zhao +2Neural RenderingDifferentiable Ray Tracing

  11. FOSCAR: Accelerating Decode Attention over 2- and 4-Bit KV Caches through Affine Folding

    Nov 6, 2026Donglin Zhuang, Pragaash Ponnusamy, Xiaoxia Wu +2LLM Inference AccelerationGPU Kernel Optimization

  12. Fused-Softmax for efficient quantized attention

    Nov 6, 2026Tianxing Wu, Juuso Korhonen, Jesus Carabano BravoSoftmax AttentionGPU Kernel Optimization

  13. Dynamic GPU Voxel Grids and Voxel-sequential Processing in NanoVDB

    Nov 5, 2026Eftychios Sifakis, Francis Williams, Christopher Horvath +4GPU Kernel Optimization

  14. KOSMOS: Self-scheduling Persistent Kernels for Communication-Compute Overlap

    Nov 5, 2026Alexander Magro, Aristotle Martin, Ye Wang +1GPU Kernel OptimizationDistributed Training

  15. Where Is VLA Inference Time Spent? Bottlenecks and Optimization Interactions Across GPUs

    Nov 4, 2026Muhammad Danish Waseem, Ahmed Ali-Eldin HassanEfficient VLA InferenceInference-Time Optimization

  16. Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs

    Nov 3, 2026Gengyu Rao, Ghadeer Ahmed H Alabandi, Su Ann Chong +28LLM TrainingGPU Kernel Optimization

  17. Accelerating FastDEC Clustering Algorithm with GPUs

    Nov 2, 2026Mi Li, Eibe Frank, Bernhard PfahringerDensity-Based ClusteringGPU Acceleration

  18. EpiLoRA: Fusing Many LoRA Adapters into Low-Precision GEMMs for Serving and RL

    Nov 2, 2026Hieu Tran, Hong YuMulti-LoRA InferenceLLM Serving

  19. Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization

    Nov 2, 2026Gabriele Oliaro, Jaeseong Lee, Zhaoyuan Su +3GPU Kernel OptimizationGPU Kernel Generation

  20. Turbo-GMM: FlashAttention-Inspired GPU-Accelerated Gaussian Mixture Model

    Nov 2, 2026Momin Abbas, Sadia Asif, Asad Ul Haq +1Gaussian Mixture ModelsGPU Kernel Optimization

  21. Weights That Stay: On-Chip Megakernels for Serial Heads in Speculative Decoding

    Oct 31, 2026Xinwei Qiang, Ziyi Xu, Zaifeng Pan +2LLM Inference AccelerationParallel Speculative Decoding

  22. The Endgame of FSDP Communication: Layout-Aware Collectives on Copy Engines, RDMA, and NVSwitch

    Oct 28, 2026Zejian Wang, Yifan Zhao, Kaihui Ren +11Communication-Efficient Distributed TrainingDistributed Training

  23. Evolving Agent Teams

    Oct 27, 2026Shiyi Cao, Ziming Mao, Dacheng Li +2Multi-Agent CodingSelf-Evolving Agents

  24. God Does Not Play Dice: Enforcing Batch Invariance in Agentic Kernel Generation

    Oct 27, 2026Youwei Bao, Chengpeng Wang, Simin ChenGPU Kernel GenerationGPU Kernel Optimization