GPU Acceleration

GPU: Graphics Processing Unit

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 125

All topics

Mixed-Precision Communication-Avoiding SGD for Generalized Linear Models on GPUs
Jun 16, 2026Aditya Devarakonda, Irene Simó Muñoz, Giulia GuidiGPU Acceleration Communication-Efficient Distributed Training
Department of Computer Science, Wake Forest University, Winston-Salem, NC, USA · Department of Computer Science, Cornell University, Ithaca, NY, USA
Local-GS: Accelerating 3D Gaussian Splatting via Tile-Local Warp Coherence
Jun 15, 2026Yang Luo, Yan Gong, Yongsheng Gao +33D Gaussian Splatting 3DGS Acceleration
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, the School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
Rigel: Reverse-Engineering the Metal 4.1 Tensor Compute Path on the Apple M4 Max GPU
Jun 11, 2026Ramchand KumaresanGPU Kernel Optimization GPU Acceleration
Murai Labs
A Scalable PyTorch Abstraction for Multi-GPU Gaussian Splatting
Jun 9, 2026Matthew Cong, Francis Williams, Jonathan Swartz +33D Gaussian Splatting GPU Acceleration
1NVIDIA · University of Toronto · 3Vector Institute
Flash-GMM: A Memory-Efficient Kernel for Scalable Soft Clustering
Jun 9, 2026Gal Bloch, Ariel Gera, Matan Orbach +2GPU Kernel Optimization GPU Acceleration
IBM Research
TorchKM: A GPU-Oriented Library for Kernel Learning and Model Selection
Jun 4, 2026Yikai Zhang, Gaoxiang Jia, Jie Ding +1Model Selection Kernel Regression
Department of Statistics and Actuarial Science, University of Iowa, Iowa City, IA 52242, USA · Individual Researcher · AIScientists, Inc. (MorphMind), 245 Main Street, Cambridge, MA 02142, USA
Accelerated Fourier SAT (AFSAT): Fully Realising a GPU-based Symmetric Pseudo-Boolean SAT Solver
Jun 4, 2026Cody J Christopher, Charles GrettonGPU Acceleration High-Performance Computing
School of Computing, Australian National University, Canberra, Australia.
RadiusFPS: Efficient Farthest Point Sampling on CPUs and GPUs via Spherical Voxel Pruning
Jun 4, 2026Ziyang Yu, Xiang Li, Qiong Chang +1GPU Acceleration Point Cloud Learning
School of Computing Institute of Science Tokyo Tokyo
AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference
Jun 4, 2026Xuanzhe Li, Ziyan Weng, Zhiyu Zhu +1Efficient Transformer Inference GPU Acceleration
City University of Hong Kong (Dongguan) · City University of Hong Kong
GS-NFS: Bandwidth-adaptive Streaming of Dynamic Gaussian Splats and Point Clouds
Jun 4, 2026Rajrup Ghosh, Haodong Wang, Haoran Hong +63DGS Compression 3D Gaussian Splatting
University of Southern California
MelT: A Portable, Single-GEMM Mel Audio Frontend via Non-Uniform DFT with Measured Latency and Energy Gains on GPUs
May 31, 2026Augusto Camargo, Marcelo FingerGPU Acceleration Speech Processing
Institute of Mathematics and Statistics (IME), University of São Paulo, São Paulo, Brazil
Threshold-Based Exclusive Batching for LLM Inference
May 30, 2026Weifang Zhang, Yuzhou Nie, Bowen Pang +2GPU Acceleration LLM Inference
The Hong Kong Polytechnic University · UC Santa Barbara · Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences +1
HiGS: A Hierarchical Rendering Architecture for Real-Time 3D Gaussian Splatting
May 29, 2026Dawid Pająk, Martin Bisson, Rodolfo Lima3D Gaussian Splatting 3DGS Acceleration
NVIDIA
On Efficient Scaling of GNNs via IO-Aware Layers Implementations
May 29, 2026Daria Fomina, Daniil Krasylnikov, Alexey Boykov +3Graph Attention Networks GPU Kernel Optimization
1Yandex · 2HSE University · 3ITMO University
GP-GOMEA with GPU-Based Fitness Evaluations: Design and Performance Analysis
May 29, 2026Jasper Post, Johannes Koch, Anton Bouter +2GPU Acceleration Evolutionary Optimization
Centrum Wiskunde & Informatica, Amsterdam, The Netherlands · Delft University of Technology, Delft, The Netherlands · Leiden University Medical Center, Leiden, The Netherlands
Caspar: CUDA Accelerator for Symbolic Programming with Adaptive Reordering
May 28, 2026Emil Martens, Aaron Miller, Matias Varnum +1Bundle Adjustment GPU Kernel Optimization
Norwegian University of Science and Technology · Skydio
Memory-Bound but Not Bandwidth-Limited: The Physical AI Inference Gap in Batch-1 LLM Decode
May 28, 2026Josef ChenGPU Kernel Optimization GPU Acceleration
KAIKAKU
HTAM: Hierarchical Transition-Attended Memory for Operator Optimization
May 28, 2026Yining Zhang, Mingyang Yi, Chen Wang +5GPU Kernel Optimization Memory-Augmented Language Models
School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy +1
CA-AC-MPC: CUDA-Accelerated Actor-Critic Model Predictive Control
May 27, 2026Antoonio Buo, Vittorio Cammarota, Michele Avagnale +3GPU Acceleration Model Predictive Control
PRISMA Lab and CREATE Consortium, Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy
KLineage: Recovering the Missing When of Kernel Optimization by Deoptimizing Experts
May 27, 2026Shuoming Zhang, Qiuchu Yu, Ruiyuan Xu +11GPU Kernel Optimization GPU Acceleration
SKLP, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of New South Wales
FastKernels: Benchmarking GPU Kernel Generation in Production
May 22, 2026Gabriele Oliaro, Yichao Fu, May Jiang +5GPU Kernel Optimization GPU Acceleration
1Snowflake AI Research · 2CMU · 3UCSD +1
ModeSwitch-LLM: A Lightweight Phase-Aware Controller for Cross-Mode LLM Inference on a Single GPU
May 21, 2026Aman Sunesh, Ali Alshehhi, Hivansh DhakneLLM Inference Efficiency LLM Serving
Department of Computer Engineering NYU Abu Dhabi Abu Dhabi, United Arab Emirates · Courant Institute of Mathematical Sciences New York University New York, United States · Department of Computer Engineering New York University New York, United States
From Sequential Nodes to GPU Batches: Parallel Branch and Bound for Optimal $k$ -Sparse GLMs
May 21, 2026Jiachang Liu, Andrea LodiBranch and Bound GPU Kernel Optimization
∗Center for Data Science for Enterprise and Society, Cornell University, Ithaca, USA · †School of Operations Research and Information Engineering, Cornell University, Ithaca, USA · ‡Jacobs Technion-Cornell Institute, Cornell Tech and Technion–IIT, New York, USA
Instant GPU Efficiency Visibility at Fleet Scale
May 20, 2026Connor Pedersen, Dong H. Ahn, Michel Migdal +2GPU Acceleration High-Performance Computing
NVIDIA
Llamas on the Web: Memory-Efficient, Performance-Portable, and Multi-Precision LLM Inference with WebGPU
May 20, 2026Reese Levine, Rithik Sharma, Nikhil Jain +5GPU Acceleration LLM Inference
UC Santa Cruz · Microsoft Research
Real-Time Parallel Counterfactual Regret Minimization
May 19, 2026Boning Li, Longbo HuangGame-Playing Agents GPU Acceleration
IIIS, Tsinghua University
CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
May 19, 2026Han Guo, Jack Zhang, Arjun Menon +4GPU Kernel Optimization GPU Acceleration
1Massachusetts Institute of Technology · *Part of this work was done at Together AI. · 2Princeton University +2
LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation
May 18, 2026Yukang Chen, Luozhou Wang, Wei Huang +13GPU Acceleration Long Video Generation
NVIDIA
Lightweight Gaussian Process Inference in C++ on Metal and CUDA
May 18, 2026Yu-Hsueh FangGPU Kernel Optimization GPU Acceleration
Department of Information Management, National Taiwan University · 2H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology
CoX-MoE: Coalesced Expert Execution for High-Throughput MoE Inference with AMX-Enabled CPU-GPU Co-Execution
May 18, 2026Muyoung Son, Yi Chen, Seungjae Yoo +2Expert Offloading GPU Acceleration
KAIST · Daejeon, Republic of Korea

Jun 16, 2026, cs.DCJ/K move · Enter open · S save

Mixed-Precision Communication-Avoiding SGD for Generalized Linear Models on GPUs

Aditya Devarakonda, Irene Simó Muñoz, Giulia Guidi

Department of Computer Science, Wake Forest University, Winston-Salem, NC, USA · Department of Computer Science, Cornell University, Ithaca, NY, USA

GPU Acceleration Communication-Efficient Distributed Training High-Performance Computing Generalized Linear Models Stochastic Gradient Descent

Open paper AskPDF arXiv

Distributed stochastic gradient descent (SGD) is limited by communication rather than computation, since each iteration requires an AllReduce across processes. Communication-avoiding SGD (CA-SGD) amortizes communication over

s

iterations by replacing

s

consecutive AllReduces with a single AllReduce of an

sb\times sb

Gram matrix, trading more computation and bandwidth for fewer synchronization points. Modern GPUs with matrix hardware and reduced-precision formats offset this by accelerating the Gram GEMM and shrinking BF16 traffic. We study mixed-precision CA-SGD for generalized linear models on NVIDIA GPUs. Our finite-precision analysis decomposes the local rounding error of one CA-SGD outer iteration into nine independent precision choices, depending on the hardware only through its low-precision unit roundoffs, so the resulting recipes transfer in principle across GPU generations. The recipe stores the input matrix and margin vector in low precision, computes the Gram matrix from low-precision inputs with high-precision accumulation, communicates it in high precision, and performs the inner recurrence and weight updates in high precision. On NERSC Perlmutter A100 GPUs, mixed-precision CA-SGD matches FP32 SGD loss within

0.5\%

on logistic, linear, and Poisson problems and reaches

5.1

6.8\times

speedup over FP32 SGD on epsilon, SUSY, HIGGS, synth, and Poisson-synth. Our software is available at https://doi.org/10.5281/zenodo.20448273

Momentum

Related topics

Latest papers 125