Structured Neural Network Compression

Latest papers

8 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

MINE: Entropy-coded Memory-efficient Inference for Large Language Models
Nov 6, 2026Emil Schätzle, Markus PüschelStructured Neural Network Compression LLM Inference Acceleration
Department of Computer Science, ETH Zurich, Zurich, Switzerland
NeuralZip: Reusable Setup for Fast Lossless Compression
Oct 7, 2026Martín Bravo, Samuel Horváth, Gonzalo Navarro +1Structured Neural Network Compression Lossless Data Compression
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Computer Science, University of Chile, Santiago, Chile · National Center for Artificial Intelligence (CENIA), Santiago, Chile
HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networks
Jun 29, 2026Lukas Meiner, Jens Mehnert, Alexandru Paul ConduracheEfficient Neural Network Inference Structured Neural Network Compression
Robert Bosch GmbH, Leonberg, Germany. · Universität zu Lübeck, Lübeck, Germany.
Field-level weak lensing cosmology with $60$ simulations using multifidelity simulation-based inference
Jun 22, 2026Alex A. Saoulis, Kiyam Lin, Niall Jeffrey +5Simulation-Based Inference Structured Neural Network Compression
Department of Physics & Astronomy, University College London, Gower Street, London, WC1E 6BT, United Kingdom · Department of Earth Sciences, University College London, 5 Gower Place, London, WC1E 6BS, United Kingdom · Department of Physics & King’s Institute for Artificial Intelligence, King’s College London, Strand, London WC2R 2LS, United Kingdom +3
OFA-Diffusion Compression: Compressing Diffusion Model in One-Shot Manner
Apr 14, 2026Haoyang Jiang, Zekun Wang, Mingyang Yi +6Structured Neural Network Compression Diffusion Model Compression
Renmin University of China Beijing, China · Alibaba Inc. Hangzhou, China · Independent researcher +1
ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
Mar 28, 2026Jinwu Yang, Jiaan Wu, Zedong Liu +17Structured Neural Network Compression Neural Network Compression
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Huawei Technologies Co., Ltd., Shenzhen, Guangdong, China
Avey-B
Feb 17, 2026Devang Acharya, Mohammad HammoudStructured Neural Network Compression Language Modeling
Avey AI
On the Interaction of Compressibility and Adversarial Robustness
Jul 23, 2025Melih Barsbey, Antônio H. Ribeiro, Umut Şimşekli +1Neural Network Robustness Adversarial Robustness
Department of Computing, Imperial College London, UK · Department of Information Technology, Uppsala University, Sweden · INRIA, CNRS, Département d’Informatique de l’Ecole Normale Supérieure / PSL, France

Nov 6, 2026J/K move · Enter open · S save

MINE: Entropy-coded Memory-efficient Inference for Large Language Models

Emil Schätzle, Markus Püschel

Department of Computer Science, ETH Zurich, Zurich, Switzerland

Structured Neural Network Compression LLM Inference Acceleration Memory-Efficient Inference Entropy Coding LLM Compression Neural Network Compression

Open paper AskPDF arXiv

We present MINE (Memory-efficient INference Engine), a lossless entropy coding approach that accelerates single-stream inference of Large Language Models (LLMs), where existing lossless compression methods yield speedups only for specific data types such as BF16 and under additional assumptions. Single-stream decoding is strongly memory-bound, since every weight must be streamed from memory once per generated token and requests cannot be batched in local or low-latency deployments. At the core of MINE is dtANS-LLM, a GPU entropy coding scheme adapted from the prior dtANS to the needs of LLM inference. It exploits the non-uniform distribution of weights, which persists even after quantization, and exposes hyper-parameters that trade compression ratio against decoding cost. We auto-tune these parameters for each tensor and GPU to obtain the Pareto front between model size and decoding latency. MINE integrates into vLLM as a plugin and selects the best kernel configuration for a given memory budget. On Llama~~3.1~~8B~~Instruct with a 2.1-bit EntQuant checkpoint, MINE achieves speedups of 2.52 $\times$ and 1.48 $\times$ over the fastest baseline on an RTX~~4060~~Ti and an RTX~~5090, at effective bitrates of 2.24 and 3.43~bits per weight. MINE is available as open source.