Structured Sparsity

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 30

All topics

CausalBind: Causal Modeling and Learning for Protein-Molecule Virtual Screening
Oct 5, 2026Loka Li, Jin Tian, Kun ZhangCausal Representation Learning Structured Sparsity
Mohamed bin Zayed University of Artificial Intelligence · Carnegie Mellon University
CASS: Contribution-Aware Structured Sparsity for Model Merging
Sep 28, 2026Yan Li, Guiping Cao, Meng Xu +5Multi-Task Learning Structured Sparsity
Pengcheng Laboratory, Shenzhen, China · City University of Hong Kong, Hong Kong, China · Southern University of Science and Technology, Shenzhen, China +2
GroupMask: Layer-Adaptive Group-wise Sparsity for Semi-Structured LLM Pruning
Sep 27, 2026Zhengao Li, Shuoqiu Li, Xiaofang Zhang +7LLM Pruning LLM Compression
Florida State University · University of Connecticut · College of William & Mary +1
Calendar-Structured Sparse Principal Component Analysis for Interpretable Multi-Periodic Electricity Consumption Profiles
Sep 5, 2026Carlos Quesada-Granja, Tony Castillo-Calzadilla, Carlos Rizo-MaestrePrincipal Component Analysis Structured Sparsity
Dept. of Computer Science and Artificial Intelligence, University of Alicante. Ctra. de San Vicente del Raspeig s/n, 03690 San Vicente del Raspeig, Alicante, Spain. · Dept. of Industrial Engineering, University of La Laguna. Camino San Francisco de Paula s/n, 38200 San Cristóbal de La Laguna, Santa Cruz de Tenerife, Spain. · Dept. of Architectural Constructions, University of Alicante. Ctra. de San Vicente del Raspeig s/n, 03690 San Vicente del Raspeig, Alicante, Spain.
Learning Sparse Decision Trees via Transformer Variational Auto-Encoders
Sep 1, 2026Giacomo Fidone, Alessio Cascione, Riccardo GuidottiVariational Autoencoders Structured Sparsity
University of Pisa · ISTI-CNR
Behaviorally Effective LoRA Writes Are Sparse and Structured
Sep 1, 2026Haruto Sato, Yuki Tanaka, Ren Nakamura +2Structured Sparsity Low-Rank Adaptation
Independent Researchers
Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs
Jul 30, 2026Jinyi Liu, Wei Chen, Pengyu Chen +4Activation Sparsity Structured Sparsity
Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · School of Computer Science, Nanjing University of Posts and Telecommunications
Sparse by Command: Task-Conditional Compute Skipping for Multi-Task Inference Accelerators
Jul 24, 2026Afzal Ahmad, Gaoyu Mao, Shoubo Hu +4Structured Sparsity FPGA-Based Neural Network Acceleration
The Hong Kong University of Science and Technology · Huawei Noah’s Ark Lab · The Hong Kong University of Science and Technology (Guangzhou)
BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator
Jul 20, 2026Varun Manjunath, Ruokai Yin, Donghyun Lee +2Efficient Neural Network Inference AI Accelerator Inference
University of Southern California Los Angeles, USA · Yale University New Haven, USA
SpikeDS: Dual Sparsity Spikformer for Perineural Invasion Prediction in 3D MRI
Jul 13, 2026Induk Um, Youngung Han, Kyeonghun Kim +14Spiking Transformers 3D Medical Imaging
Chung-Ang University, Seoul, Republic of Korea · Seoul National University, Seoul, Republic of Korea · OUTTA, Seoul, Republic of Korea +5
It Takes a MAESTRO To Prune Bad Experts
Jul 9, 2026Palaash Goel, Ayush Maheshwari, Tanmoy ChakrabortyMixture-of-Experts Pruning LLM Pruning
Indian Institute of Technology Delhi, India · NVIDIA, India
Deep Multitask Learning for Mixed-Type Outcomes with Shared Sparsity
Jul 1, 2026Huichao Li, Tong Wang, Sanguo Zhang +1Multi-Task Learning Structured Sparsity
School of Mathematical Sciences, University of Chinese Academy of Sciences · School of Statistics and Data Science, Southeast University, Nanjing, Jiangsu, China · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China +2
Depth-Staggered Fibonacci Spacing for Sparse Attention: Static Schedules Beat Learned Dilation and Extrapolate Where Dense Attention Fails
Jun 26, 2026Chad A. CappsStructured Sparsity Long-Context Modeling
Independent Researcher
End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inference
Jun 26, 2026Yuhang Chen, Jinhao Duan, Ruichen Zhang +11LLM Inference Structured Sparsity
University of North Carolina at Chapel Hill · Meta AI
Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Jun 23, 2026Thomas Fel, Matthew Kowal, Mozes Jacobs +22Neural Representation Geometry Activation Sparsity
•Goodfire · aHarvard University · bStanford University +1
Scaling Audio Models Efficiently: Joint Optimization of Scale, Resolution, Adaptation, Precision, and Sparsity
Jun 22, 2026Vyom Agarwal, Mokshda Gangrade, Siddharth Pal +1Model Compression Structured Sparsity
Computer, Mathematical, and Natural Sciences, University of Maryland, College Park, MD, USA · RTX BBN Technologies, Columbia, MD, USA · Electrical and Computer Engineering, University of Maryland, College Park, MD, USA
On the Sparsity-Storage-Accuracy Tradeoff in Parsimoniously Activated Dictionary Learning
Jun 21, 2026Zihui Zhao, Yuanbo Tang, Yang LiStructured Sparsity Dictionary Learning
Tsinghua University, Institute of Data and Information Shenzhen Key Laboratory of Ubiquitous Data Enabling
Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMs
Jun 9, 2026Ruixuan Huang, Jinyuan Shi, Hantao Huang +5Continual Learning for LLMs LLM Compression
1HKUST · 2Moffett AI · 3Independent Contributor
SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference
Jun 9, 2026Jaeseong Lee, Seung-won Hwang, Samyam RajbhandariLLM Pruning High-Performance Computing
Snowflake AI Research* · Seoul National University†
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
Jun 3, 2026Jiangyang He, Shaolin Zhu, Deyi XiongMixture-of-Experts Pruning LLM Pruning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China
HASTE: Hardware-Aware Dynamic Sparse Training for Large Output Spaces
May 31, 2026Nasib Ullah, Jinbin Zhang, Jean Lucien Randrianantenaina +2Multi-Label Classification Structured Sparsity
Department of Computer Science, Aalto University, Espoo, Finland · Department of Computer Science, University of Bath, Bath, UK. · IST Austria.
PrunePath: Towards Highly Structured Sparse Language Models
May 27, 2026Zhexuan Gu, Zixun Fu, Yancheng YuanMixture-of-Experts Pruning LLM Pruning
Department of Applied Mathematics, The Hong Kong Polytechnic University
Inference Time Context Sparsity: Illusion or Opportunity?
May 22, 2026Sahil Joshi, Prithvi Dixit, Agniva Chowdhury +5LLM Inference Efficiency Structured Sparsity
α Rice University · β UC Berkeley · γ IIT Bombay
Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity
May 21, 2026Hangyue Zhao, Paul Caillon, Erwan Fagnou +1Self-Attention Structured Sparsity
ESPCI PSL, Paris, France · LAMSADE, Université Paris Dauphine - PSL, Paris, France
Two-Valued Symmetric Circulant Matrices: Applications in Deep Learning
May 15, 2026Jayakrishna Amathi, Venkata Prasanth Yanambaka, Saraju P. Mohanty +1Structured Sparsity Edge ML
Department of Computer Science and Engineering University of North Texas Denton, TX, USA. · Division of Computer Science Texas Woman’s University Denton, TX, USA.
Compositional Sparsity as an Inductive Bias for Neural Architecture Design
May 14, 2026Hongyu Lin, Antonio Briola, Yuanrong Wang +1Hierarchical Representation Learning Structured Sparsity
Department of Computer Science, University College London, London, United Kingdom
Relative Kinetic Utility: Calibrating Cross-Layer Credit for Global Structured LLM Pruning
May 9, 2026Tianhao Qian, Guilin Qi, Jiayu ChenLLM Pruning Structured Sparsity
School of Mathematics, Southeast University Nanjing 210096, China
SparseForge: Efficient Semi-Structured LLM Sparsification via Annealing of Hessian-Guided Soft-Mask
May 7, 2026Liu Hanzuo, Chaofan Lin, Weixuan Sun +4LLM Pruning Sparse Recovery
†Tsinghua University · ‡Tencent
Proximal Projection for Doubly Sparse Regularized Models
May 6, 2026Jia Wei He, R. Ayesha Ali, Gerarda DarlingtonStructured Sparsity Linear Regression
Department of Mathematics & Statistics , University of Guelph
PATCH: Learnable Tile-level Hybrid Sparsity for LLMs
Sep 27, 2025Younes Hourri, Mohammad Mozaffari, Maryam Mehri DehnaviLLM Pruning LLM Compression
Department of Computer Science University of Toronto

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

CausalBind: Causal Modeling and Learning for Protein-Molecule Virtual Screening

Loka Li, Jin Tian, Kun Zhang

Mohamed bin Zayed University of Artificial Intelligence · Carnegie Mellon University

Causal Representation Learning Structured Sparsity Structure-Based Drug Design Causal Identifiability Molecular Representation Learning

Open paper AskPDF arXiv

Protein-molecule virtual screening is increasingly cast as a problem of representation learning in a shared embedding space. Existing methods rely on dense holistic alignment, entangling invariant binding determinants with nuisance correlations and limiting transfer to new targets. It has been noted that binding in protein-molecule systems involves sparse cross-modality interactions: binding is governed by a small contact interface and a few decisive local interactions (e.g., hydrogen bonds, hydrophobic contacts, and salt bridges) rather than the global structures of the protein and molecule. We hypothesize that uncovering and leveraging sparse interaction patterns is critical for generalization beyond the training data, as these patterns are reusable and expected to improve performance across different scenarios. In this paper, we aim to identify and leverage sparse interaction patterns, and verify our hypothesis. Since the training data contain only observed binding pairs, we formalize this prior via a V-structure causal model under Heckman-style selection, and establish three theoretical results: (i) the latent concepts of interacting proteins and molecules are not identifiable without appropriate sparsity constraints; (ii) these concepts and their sparse interactions are component-wise identifiable under structural sparsity conditions; and (iii) a low-rank relaxation of these conditions yields subspace identifiability of the concepts and interactions. Inspired by these principles, we propose CausalBind with three implementation variants. Extensive experiments on DUD-E and LIT-PCBA benchmarks show that all variants consistently outperform strong retrieval baselines, with the largest gains on LIT-PCBA early enrichment, and further generalize to target- and scaffold-level out-of-distribution splits. Code is available at https://github.com/lokali/CausalBind.

Momentum

Related topics

Latest papers 30