Sparse Matrix Multiplication

Follow this topic Ask about this topic

Latest papers

10 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

SSR: Sparse Segment Reduction for Ternary GEMM Acceleration
Oct 6, 2026Adeline Pittet, Shien Zhu, Valérie Verdan +1Ternary Quantization LLM Inference Acceleration
Systems Group, ETH Zurich, Switzerland
Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU
Oct 2, 2026Jafar Badour, Maurice van Keulen, Elena MocanuDynamic Sparse Training Sparse Neural Networks
University of Twente
Reinforcement Learning-Guided Graph Transformations for SpTRSV Optimization
Sep 30, 2026Buse YılmazHigh-Performance Computing Sparse Matrix Multiplication
Department of Computer Engineering, MEF University, Huzur, Maslak Ayazağ̆a Cd., İstanbul, Turkey.
Spectral Analysis for Sparse Matrix Computation: Insights and Potential
Aug 29, 2026Ruifeng Zhang, Xipeng ShenSparse Matrix Multiplication High-Performance Computing
North Carolina State University Raleigh, North Carolina, USA
SparseDitto: An Agentic Sparse Compilation Framework through Architecture-Aware Synthesis on GPUs
Aug 5, 2026Shiyang Li, Guangyan Sun, Jinwei Tang +3GPU Kernel Optimization High-Performance Computing
University of Minnesota Minneapolis, MN, USA · Northeastern University Boston, MA, USA
Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference
Aug 2, 2026Ruokai Yin, Priyadarshini PandaGPU Kernel Optimization LLM Inference
Yale University New Haven, USA · University of Southern California Los Angeles, USA
Scalable Optimal Transport Algorithm for Network Alignment
Jul 11, 2026Elaheh Hassani, Durga Mandarapu, Qi Yu +2Graph Matching Sparse Matrix Multiplication
Lawrence Berkeley National Laboratory CA, USA · University of Illinois at Urbana-Champaign IL, USA
Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices
Jun 13, 2026Tao Lu, Haoyu Wang, Zonghui Wang +3LLM Pruning GPU Kernel Optimization
National University of Singapore · Zhejiang University
GPU Acceleration of Sparse Fully Homomorphic Encrypted DNNs
Apr 13, 2026Lara D'Agata, Carlos Agulló-Domingo, Óscar Vera-López +7Sparse Matrix Multiplication Homomorphic Encryption
University of Glasgow, UK · University of Murcia, Spain · AMD +1
Making LLMs Optimize Multi-Scenario CUDA Kernels Like Experts
Mar 7, 2026Yuxuan Han, Meng-Hao Guo, Zhengning Liu +2Sparse Matrix Multiplication GPU Kernel Optimization
Tsinghua University · Proxseer Inc.

Oct 6, 2026J/K move · Enter open · S save

SSR: Sparse Segment Reduction for Ternary GEMM Acceleration

Adeline Pittet, Shien Zhu, Valérie Verdan, Gustavo Alonso

Systems Group, ETH Zurich, Switzerland

Ternary Quantization LLM Inference Acceleration Sparse Matrix Multiplication

Open paper AskPDF arXiv

Large Language Models (LLMs) require substantial computational resources, limiting their deployment on resource-constrained hardware. Ternary LLMs mitigate these demands through weight quantization via ternary values, achieving significant compression often with 50-90% sparsity. However, existing approaches have limitations: methods optimized for ternary weights, such as BitNet, redundant segment reduction (RSR), and its improved version RSR++, do not exploit sparsity structures, while conventional sparse formats neglect ternary characteristics, foregoing dual optimization opportunities. In this paper, we introduce Sparse Segment Reduction (SSR), a ternary matrix multiplication method designed to accelerate the inference of ternary LLMs and general Ternary Weight Networks (TWNs). SSR has a dedicated optimized ternary data format and an algorithm that systematically exploits sparsity patterns through computation trees that scale with the sparsity. SSR provides theoretical gains with asymptotically faster inference than RSR++ for sparsity above 50%, while practical evaluations reveal performance improvements across all sparsity levels. Evaluation results show that SSR achieves 2.1-11.3x speedup over RSR++ on ternary GEMM with 45-95% sparsity. Furthermore, SSR achieves 3.5-6.3x end-to-end speedup and 4.9% of memory saving over RSR++ on the Llama-3 1B model inference.