GPU Kernel Optimization

GPU: Graphics Processing Unit

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 126

All topics

Meganeura: Portable GPU Training and Inference through Vulkan and Metal
Aug 3, 2026Dzmitry MalyshauEfficient Neural Network Inference GPU Kernel Optimization
Independent Researcher
Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference
Aug 2, 2026Ruokai Yin, Priyadarshini PandaGPU Kernel Optimization LLM Inference
Yale University New Haven, USA · University of Southern California Los Angeles, USA
DrainSinkhorn: Safe Elimination for Batched Entropic Optimal Transport
Jul 27, 2026Xinyang WenGPU Kernel Optimization Optimal Transport
X-Stage: Modeling Post-Issue Backpressure in GPU Communication--Computation Fusion
Jul 25, 2026Jianwen Xian, Zhiyuan Xu, Yuchen Li +9GPU Kernel Optimization
1KlingAI Research · 2Tsinghua University · 3NVIDIA
Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization
Jul 25, 2026Dongjie Chen, Ping Zhao, Bohua Zhan +11GPU Kernel Optimization Large Language Model-Guided Optimization
Huawei Technologies Co., Ltd.
Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation
Jul 23, 2026Quazi Ishtiaque Mahmud, Nesreen K. Ahmed, Ali JannesariRL for Code Generation GPU Kernel Optimization
Iowa State University, Ames, IA, USA · Cisco AI Research, San Jose, CA, USA
KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?
Jul 22, 2026Peiyu Zang, Jian Tao, Jialing Zhang +4GPU Kernel Optimization Benchmark Design
Beijing Normal University · BAAI · Beijing Jiaotong University +2
Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts
Jul 21, 2026Minyu Cui, Anna Wingkvist, Morgan EricssonGPU Kernel Optimization Communication-Efficient Distributed Training
Linnaeus University · Växjö, Sweden
MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel
Jul 21, 2026Lenore Mulin, Gaetan HainsEfficient Transformer Inference Grouped-Query Attention
University at Albany, SUNY · LACL, Universit´e Paris-Est Cr´eteil
Sobek: Streaming Equivariant Tensor Product Convolutions
Jul 20, 2026Vladimir Chorošajev, Cédric BényGPU Kernel Optimization GPU Acceleration
Cortex Discovery
FlashPDE: A Drop-In Fused Triton Operator Library for Neural PDE Solvers
Jul 20, 2026Peiyu Zang, Bosen Xie, Ruoxiang Xu +1Automatic Differentiation Neural PDE Solvers
School of Mathematical Sciences, Beijing Normal University · 2Beijing University of Posts and Telecommunications
Harness Engineering for LLM-Driven GPU Kernel Generation
Jul 20, 2026Yue Shui, Chenyu Ma, Hangfei Xu +2GPU Kernel Optimization Large Language Model-Guided Optimization
Baidu, Inc.
A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment
Jul 20, 2026Nikola Pižurica, Matteo Risso, Nikola Milović +4GPU Kernel Optimization Bayesian Inference
Computer Science Center, University of Montenegro · Fain Tech, Podgorica, Montenegro · Department of Control and Computer Engineering, Politecnico di Torino +2
CaT-GS: Efficient 3DGS Rendering for Large Scale Scenes via Inter-frame Caching and Tile Scheduling
Jul 20, 2026Tingjia Zhang, Bo Chen, Shengzhong Liu +23D Gaussian Splatting 3DGS Acceleration
Shanghai Jiao Tong University · University of Illinois Urbana-Champaign
A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi
Jul 16, 2026A. C. Opus, J. Q. LuEfficient Transformer Inference GPU Kernel Optimization
Department of Physics, University of Puerto Rico, Mayaguez, PR 00680, USA
Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
Jul 16, 2026Lingyun Yang, Yuxiao Wang, Shenghao Liang +8GPU Kernel Optimization Large Language Model-Guided Optimization
ATREX Team†, Alibaba Group
WarpMPC: Large-Batch MPC on GPU via ADMM with Unrolled $LDL^\top$ Factorization
Jul 13, 2026Henrik Hose, Se Hwan Jeon, Charles Khazoom +2GPU Kernel Optimization GPU Acceleration
Institute for Data Science in Mechanical Engineering (DSME), RWTH Aachen University, Germany · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge MA, USA.
Decomposing Runtime, Kernel, and Quantization Speedups via a Matched FP16 Intermediate: A Hardware-Conditioned Case Study on Four NVIDIA RTX A5000 GPUs
Jul 13, 2026Weijia Han, Lisha QuLLM Quantization GPU Kernel Optimization
University of Washington
LiteTopK: Exploiting the Curse of Dimensionality for a Fused Indexer-TopK Kernel in Long-Context Sparse Attention
Jul 13, 2026Ziqi Yin, Jianyang Gao, Peiqi Yin +2GPU Kernel Optimization LLM Inference Acceleration
Nanyang Technological University · ETH Zurich · The Chinese University of Hong Kong
FlashBEV: Fast and Memory-Efficient Exact BEV Transformation with IO-Awareness
Jul 11, 2026Shunsuke Yokokawa, Hironori KasaharaGPU Kernel Optimization Autonomous Driving Perception
Waseda University, Tokyo, Japan · T2, Inc., Tokyo, Japan
FlashTrie: A GPU-Accelerated Constrained Beam Search for Generative Retrieval
Jul 10, 2026Dakshitha Anandakumar, Anurag Mukkara, Wenxiang Hu +5Constrained Decoding GPU Kernel Optimization
Microsoft, Redmond, WA, USA · Nvidia, Santa Clara, CA, USA
CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems
Jul 8, 2026Tingkai Liu, Muralidhar Andoorveedu, Sanjoy Das +2GPU Kernel Optimization High-Performance Computing
University of Illinois Urbana-Champaign · NVIDIA Corporation
FAST: A Holistic Framework for Optimizing Memory-I/O, Computation, and Sampling in Temporal GNN Training
Jul 6, 2026Yushu Cai, Qingrui Zhu, Lei Liu +3Temporal GNNs Graph Neural Networks
Xidian University Guangzhou, Guangdong, China · Hunan University Changsha, Hunan, China
WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution
Jul 2, 2026Wan Song, Wei Zhou, Rui Wang +4GPU Kernel Optimization Convolutional Neural Networks
Hefei University of Technology, Hefei, China · Lingyang Industrial Internet Co., Ltd., Hefei, China · Kochi University of Technology, Kochi, Japan.
GPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Study
Jun 29, 2026Rania Zitouni, Nadine Bousdjira, Sarah Hasnaoui +2GPU Kernel Optimization GPU Acceleration
E´ cole Nationale Supe´rieure d’Informatique (ESI) · High Performance Computing (HPC)
EGG: An Expert-Guided Agent Framework for Kernel Generation
Jun 25, 2026Yaochen Han, Ke Fan, Hongxu Jiang +5GPU Kernel Optimization Large Language Model-Guided Optimization
Beihang University, Beijing, China · Shanghai Jiao Tong University, Shanghai, China · Tsinghua University, Beijing, China +1
PersistentKV: Page-Aware Decode Scheduling for Long-Context LLM Serving on Commodity GPUs
Jun 25, 2026Muhammad AhmedLLM Serving GPU Kernel Optimization
Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization
Jun 24, 2026Jiading Gai, Shuai Zhang, Kaj Bostrom +6GPU Kernel Optimization High-Performance Computing
Amazon · Siemens · University of Minnesota
SOLAR: AI-Powered Speed-of-Light Performance Analysis
Jun 24, 2026Qijing Huang, Sana Damani, Zhifan Ye +9GPU Kernel Optimization High-Performance Computing
NVIDIA
StreamKL: Fast and Memory-Efficient KL Divergence for Boosting Attention Distillation
Jun 18, 2026Guangda Liu, Yiquan Wang, Chengwei Li +6GPU Kernel Optimization Knowledge Distillation
Shanghai Jiao Tong University · Huawei · Fudan University

Aug 3, 2026, cs.LGJ/K move · Enter open · S save

Meganeura: Portable GPU Training and Inference through Vulkan and Metal

Dzmitry Malyshau

Independent Researcher

Efficient Neural Network Inference GPU Kernel Optimization GPU Acceleration GPU Kernel Generation

Open paper AskPDF arXiv

Training and deployed inference often cross export, conversion, and platform-specific runtime boundaries. Meganeura asks whether one compact native compiler can span both phases on consumer GPUs. Its typed static graph, automatic differentiation, optimizer, checkpoint, memory planner, and runtime lower specialized programs through Vulkan and Metal. We compare five matched workloads with PyTorch on NVIDIA and AMD discrete GPUs, an AMD APU, Apple silicon, and an Intel iGPU. The protocol separates strict f32 from validated fast paths and gates forward and backward independently. Forty-eight of 50 device-workload-mode cells pass both gates; the other two share one unresolved backward-reference disagreement on a newly supported APU. In strict f32, Meganeura wins 12 of 20 GPU-referenced minimal-latency cells and has a median valid training gap of 1.8x. On the discrete AMD GPU, four of five inference workloads are within 1.10x of compiled ROCm PyTorch and three training workloads are faster. Under accelerated contracts, the worst training gap is 4.6x. Compilation takes 0.1-2.4 seconds versus 6-96 seconds for torch.compile on supported GPU paths; the stripped binary is 13 MiB. Dispatch profiles localize the largest gaps to convolution derivatives and attention backward. A physical Android XR case study transfers a Meganeura-trained decoder into an Adreno/OpenXR application sharing the graphics queue. The results show that general consumer graphics APIs can support a compact shared train-to-deploy stack at useful, sometimes vendor-competitive performance. The measured gaps point to kernel coverage, scheduling, and arithmetic policy rather than an identified API limitation.

Momentum

Related topics

Latest papers 126