Overview

Structured sparsity studies how to restrict parameters or computations to organized patterns rather than removing elements independently, often to improve efficiency, interpretability, or statistical reliability. In statistical learning and signal recovery, group penalties, block-sparse models, and structured factorization encode relationships among variables; in neural networks, pruning and sparse training target channels, attention heads, tiles, or N:M weight patterns. Dynamic routing and input-dependent masks extend these ideas to language models, vision, and other settings, with evaluations spanning predictive benchmarks, reconstruction tasks, and deployment on accelerators. A central challenge is balancing accuracy against practical speed and memory gains: hardware-friendly patterns and optimized kernels matter, and the best sparsity structure or allocation often varies across layers and workloads.

Latest papers

CardsList

71 papers14 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. SoU-RHS: Bayesian Structured Sparsity for Compact Architecture Recovery in CNNs

    Nov 18, 2026Rafael Mouallem Rosa, Julyan ArbelStructured Neural Network CompressionNeural Network Pruning

  2. Cross-Layer TopK: Dynamically Allocating Sparsity Across Layers in Transcoders

    Nov 6, 2026Anyuan Zhuo, Shaohua Zhang, Tianle Li +4LLM InterpretabilityStructured Sparsity

  3. Rethinking Optimization Methods for Hierarchical Feature Selection

    Nov 6, 2026Xinyuan Song, Zekun Cai, Liang ZhaoFeature SelectionProximal Optimization

  4. A Parametric Linear Conic Projection for Structured Sparse Attention

    Nov 3, 2026Michel Barlaud, Clement Genninasca, Maxime Zanella +2Structured SparsitySparse Attention

  5. Structured Channel Network Pruning with Bi-Sparse Matrix Factorization

    Oct 31, 2026Jingjing Liu, Ziyang Wang, Xianchao Xiu +1Channel PruningNeural Network Pruning

  6. Breaking Information Islands in Sparse Tuning via Small-World Connectivity

    Oct 30, 2026Xianchao Guan, Zijun Xiong, Yifeng Wang +5Sparse Fine-TuningFine-Tuning Language Models

  7. SparseGDN: Chunk Sparse in Linear Attention

    Oct 30, 2026Jiedong Zhuang, Peng Zhao, Jialin Tang +1LLM Inference AccelerationGated Delta Networks

  8. Two-Stage Clipping: Linear-Time Approximate and Exact Mixed-Norm Projections for Transformers

    Oct 28, 2026Michel Barlaud, Jean-Paul MarmoratStructured SparsityTransformer Pruning

  9. On the Sparsity of Direct Preference Optimization: Weight Disentanglement in the NTK Regime

    Oct 27, 2026Kensuke Sasaki, Issei SatoDirect Preference OptimizationLLM Alignment

  10. A knowledge-driven nonconvex square-root regularization framework for joint group sparsity and structured smoothness

    Oct 26, 2026Wanling Xie, Hu YangStructured SparsitySparse Recovery

  11. Learning Structured Sparsity for Large Models: Bridging Model Compressibility and Efficient Inference

    Oct 22, 2026Tianxiang Xia, Kai Lion, Niao He +3LLM CompressionLLM Inference Acceleration

  12. BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility

    Oct 16, 2026Rida Assalouh, Tom Jacobs, Rebekka Burkholz +2Muon OptimizerStructured Sparsity

  13. Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too

    Oct 13, 2026Mehdi Makni, Natalia Ponomareva, Rahul MazumderLLM PruningNeural Network Pruning

  14. Heterogeneous Structured Pruning Under Resource Constraints for Hardware-Efficient Large Language Models

    Oct 11, 2026Cameron Barker, Andreas Paraskeva, Jan N. van Rijn +2LLM PruningNeural Network Pruning

  15. SARATHI: Decoupled Data-Free Probing and Adaptive Reconstruction for Structured LLM Pruning

    Oct 11, 2026Sankar Behera, Dhruv Singh, Satyadev Ahlawat +1LLM PruningStructured Pruning

  16. asdex: Automatic Sparse Differentiation in JAX

    Oct 8, 2026Adrian Hill, Guillaume DalleAutomatic DifferentiationStructured Sparsity

  17. SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference

    Oct 8, 2026Qitong Wang, Xinwei Niu, Mingluo Su +3LLM PruningLLM Inference Acceleration

  18. CausalBind: Causal Modeling and Learning for Protein-Molecule Virtual Screening

    Oct 5, 2026Loka Li, Jin Tian, Kun ZhangCausal Representation LearningStructured Sparsity

  19. Low-Rank and Structured Sparse Tensor Decomposition for Anomaly Detection in Multivariate Functional Data

    Oct 3, 2026Mohammad N. Bisheh, Che-Yi Liao, Kamran PaynabarAnomaly DetectionIndustrial Anomaly Detection

  20. CASS: Contribution-Aware Structured Sparsity for Model Merging

    Sep 28, 2026Yan Li, Guiping Cao, Meng Xu +5Multi-Task LearningStructured Sparsity

  21. GroupMask: Layer-Adaptive Group-wise Sparsity for Semi-Structured LLM Pruning

    Sep 27, 2026Zhengao Li, Shuoqiu Li, Xiaofang Zhang +7LLM PruningLLM Compression

  22. DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inference

    Sep 26, 2026Yifei Cai, Zhuoran Li, Xiaozuo Shen +2Transformer InferencePrivacy-Preserving ML

  23. Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers

    Sep 17, 2026Charles Kulick, Armenak Petrosyan, Sui TangStructured SparsityNeural Network Pruning