Neural Network Compression

Ask about this topic

Overview

Neural network compression studies how to represent, train, or run models with fewer parameters and less memory or computation, making them practical on devices with limited resources. Common approaches include pruning weights or neurons, quantization, low-rank and tensor factorization, parameter sharing, and knowledge distillation; some methods also compress activations exchanged during distributed training. Work spans small edge models and microcontrollers as well as large language, vision, and diffusion models, evaluated on tasks such as image classification, language modeling, and video coding. A central challenge is that smaller parameter counts or fewer theoretical operations do not always mean faster inference, and aggressive compression can harm accuracy, robustness, or task-specific behavior.

Latest papers

CardsList

218 papers28 papers in the last four weeks, up 8% on the four weeks before. 0.2% of all new papers.

Mar 28, 2026

ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs

The rapid scaling of Large Language Models presents significant challenges for their deployment and inference, particularly on resource-constrained specialized AI hardware accelerators such as Huawei's Ascend NPUs, where weight data transfer has become a critical performance bottleneck. While lossless compression can preserve model accuracy and reduce data volume, existing lossless compression algorithms exhibit extremely low throughput when ported to the Ascend NPU architecture. In this paper, we propose ENEC, a novel lossless compression method specifically customized for AI model weights and optimized for Ascend Neural Processing Units. ENEC adopts a block-based fixed-length encoding scheme and incorporates a series of NPU-specific optimizations: bit-width quantization with hierarchical halving bit-packing, vectorized branch-free integer transformation, and dependency-decoupled intra-segment scan for efficient prefix-sum computation. Experimental results demonstrate that ENEC outperforms existing state-of-the-art NPU compressors in both compression ratio and throughput. Compared to leading GPU solutions, ENEC achieves a 3.43X higher throughput than DietGPU and a 1.12X better compression ratio than nvCOMP. By reducing weight transmission overhead, ENEC significantly improves end-to-end inference performance, achieving up to a 6.3X speedup. On Ascend NPUs, ENEC is the first open-source lossless compression algorithm for model weights that achieves performance comparable to state-of-the-art GPU compressors, offering an effective solution for deploying large-scale AI models.

Jinwu Yang, Jiaan Wu, Zedong Liu +17

Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Huawei Technologies Co., Ltd., Shenzhen, Guangdong, China

Neural Network CompressionEfficient Neural Network InferenceAI Accelerator Inference
AskPDF
Mar 27, 2026

PQuantML: A Tool for End-to-End Hardware-aware Model Compression

PQuantML is a new open-source, hardware-aware neural network model compression library tailored to end-to-end workflows. Motivated by the need to deploy performant models to environments with strict latency constraints, PQuantML simplifies training of compressed models by providing a unified interface to apply pruning and quantization, either jointly or individually. The library implements multiple pruning methods with different granularities, as well as fixed-point quantization with support for High-Granularity Quantization. We evaluate PQuantML on representative tasks such as the jet substructure classification, so-called jet tagging, an on-edge problem related to real-time LHC data processing. Using various pruning methods with fixed-point quantization, PQuantML achieves substantial parameter and bit-width reductions while maintaining accuracy. The resulting compression is further compared against existing tools, such as QKeras and HGQ.

Roope Niemi, Anastasiia Petrovych, Arghya Ranjan Das +9

European Center for Nuclear Research (CERN), CH-1211 Geneva, Switzerland · Purdue University, West Lafayette, IN 47907, USA · California Institute of Technology, Pasadena, CA 91125, United States +4

Model CompressionNeural Network CompressionNeural Network QuantizationHardware-Aware QuantizationNeural Network Pruning
AskPDF
Mar 26, 2026

Once-for-All Channel Mixers (HYPERTINYPW): Generative Compression for TinyML

Deploying neural networks on microcontrollers is constrained by kilobytes of flash and SRAM, where 1x1 pointwise (PW) mixers often dominate memory even after INT8 quantization across vision, audio, and wearable sensing. We present HYPER-TINYPW, a compression-as-generation approach that replaces most stored PW weights with generated weights: a shared micro-MLP synthesizes PW kernels once at load time from tiny per-layer codes, caches them, and executes them with standard integer operators. This preserves commodity MCU runtimes and adds only a one-off synthesis cost; steady-state latency and energy match INT8 separable CNN baselines. Enforcing a shared latent basis across layers removes cross-layer redundancy, while keeping PW1 in INT8 stabilizes early, morphology-sensitive mixing. We contribute (i) TinyML-faithful packed-byte accounting covering generator, heads/factorization, codes, kept PW1, and backbone; (ii) a unified evaluation with validation-tuned t* and bootstrap confidence intervals; and (iii) a deployability analysis covering integer-only inference and boot versus lazy synthesis. On three ECG benchmarks (Apnea-ECG, PTB-XL, MIT-BIH), HYPER-TINYPW shifts the macro-F1 versus flash Pareto frontier: at about 225 kB it matches a roughly 1.4 MB CNN while being 6.31x smaller (84.15% fewer bytes), retaining at least 95% of large-model macro-F1. Under 32-64 kB budgets it sustains balanced detection where compact baselines degrade. The mechanism applies broadly to other 1D biosignals, on-device speech, and embedded sensing tasks where per-layer redundancy dominates, indicating a wider role for compression-as-generation in resource-constrained ML systems. Beyond ECG, HYPER-TINYPW transfers to TinyML audio: on Speech Commands it reaches 96.2% test accuracy (98.2% best validation), supporting broader applicability to embedded sensing workloads where repeated linear mixers dominate memory.

Yassien Shaalan

Neural Network CompressionTiny MLHypernetworks
AskPDF
Mar 20, 2026

TinyML Enhances CubeSat Mission Capabilities

Earth observation (EO) missions traditionally rely on transmitting raw or minimally processed imagery from satellites to ground stations for computationally intensive analysis. This paradigm is infeasible for CubeSat systems due to stringent constraints on the onboard embedded processors, energy availability, and communication bandwidth. To overcome these limitations, the paper presents a TinyML-based Convolutional Neural Networks (ConvNets) model optimization and deployment pipeline for onboard image classification, enabling accurate, energy-efficient, and hardware-aware inference under CubeSat-class constraints. Our pipeline integrates structured iterative pruning, post-training INT8 quantization, and hardware-aware operator mapping to compress models and align them with the heterogeneous compute architecture of the STM32N6 microcontroller from STMicroelectronics. This Microcontroller Unit (MCU) integrates a novel Arm Cortex-M55 core and a Neural-ART Neural Processing Unit (NPU), providing a realistic proxy for CubeSat onboard computers. The paper evaluates the proposed approach on three EO benchmark datasets (i.e., EuroSAT, RS_C11, MEDIC) and four models (i.e., SqueezeNet, MobileNetV3, EfficientNet, MCUNetV1). We demonstrate an average reduction in RAM usage of 89.55% and Flash memory of 70.09% for the optimized models, significantly decreasing downlink bandwidth requirements while maintaining task-acceptable accuracy (with a drop ranging from 0.4 to 8.6 percentage points compared to the Float32 baseline). The energy consumption per inference ranges from 0.68 mJ to 6.45 mJ, with latency spanning from 3.22 ms to 30.38 ms. These results fully satisfy the stringent energy budgets and real-time constraints required for efficient onboard EO processing.

Luigi Capogrosso, Michele Magno

Interdisciplinary Transformation University of Austria · ETH Zurich

On-Device InferenceEnergy-Efficient MLTiny MLNeural Network CompressionRemote Sensing Image Understanding
AskPDF
Mar 20, 2026

Growing Networks with Autonomous Pruning

This paper introduces Growing Networks with Autonomous Pruning (GNAP) for image classification. Unlike traditional convolutional neural networks, GNAP change their size, as well as the number of parameters they are using, during training, in order to best fit the data while trying to use as few parameters as possible. This is achieved through two complementary mechanisms: growth and pruning. GNAP start with few parameters, but their size is expanded periodically during training to add more expressive power each time the network has converged to a saturation point. Between these growing phases, model parameters are trained for classification and pruned simultaneously, with complete autonomy by gradient descent. Growing phases allow GNAP to improve their classification performance, while autonomous pruning allows them to keep as few parameters as possible. Experimental results on several image classification benchmarks show that our approach can train extremely sparse neural networks with high accuracy. For example, on MNIST, we achieved 99.44% accuracy with as few as 6.2k parameters, while on CIFAR10, we achieved 92.2\ accuracy with 157.8k parameters.

Charles De Lambilly, Stefan Duffner

INSA Lyon, CNRS, Ecole Centrale de Lyon, Universite Claude Bernard Lyon 1, Université Lumière Lyon 2, LIRIS, UMR5205, , Villeurbanne, 69621, France

Dynamic Sparse TrainingNeural Network GrowthSparse Neural NetworksNeural Network PruningNeural Network CompressionImage Classification
AskPDF
Mar 17, 2026

Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs

Deploying deep neural networks (DNNs) on edge devices requires strong compression with minimal accuracy loss. This paper introduces Mix-and-Match Pruning, a globally guided, layer-wise sparsification framework that leverages sensitivity scores and simple architectural rules to generate diverse, high-quality pruning configurations. The framework addresses a key limitation that different layers and architectures respond differently to pruning, making single-strategy approaches suboptimal. Mix-and-Match derives architecture-aware sparsity ranges, e.g., preserving normalization layers while pruning classifiers more aggressively, and systematically samples these ranges to produce ten strategies per sensitivity signal (magnitude, gradient, or their combination). This eliminates repeated pruning runs while offering deployment-ready accuracy-sparsity trade-offs. Experiments on CNNs and Vision Transformers demonstrate Pareto-optimal results, with Mix-and-Match reducing accuracy degradation on Swin-Tiny by 40% relative to standard single-criterion pruning. These findings show that coordinating existing pruning signals enables more reliable and efficient compressed models than introducing new criteria.

Danial Monachan, Samira Nazari, Mahdi Taheri +4

Brandenburg Technical University, Cottbus, Germany · University of Zanjan, Iran · Tallinn University of Technology, Estonia +1

Neural Network CompressionNeural Network Pruning
AskPDF
Mar 16, 2026

SimCert: Probabilistic Certification for Behavioral Similarity in Deep Neural Network Compression

Deploying Deep Neural Networks (DNNs) on resource-constrained embedded systems requires aggressive model compression techniques like quantization and pruning. However, ensuring that the compressed model preserves the behavioral fidelity of the original design is a critical challenge in the safety-critical system design flow. Existing verification methods often lack scalability or fail to handle the architectural heterogeneity introduced by pruning. In this work, we propose SimCert, a probabilistic certification framework for verifying the behavioral similarity of compressed neural networks. Unlike worst-case analysis, SimCert provides quantitative safety guarantees with adjustable confidence levels. Our framework features: (1) A dual-network symbolic propagation method supporting both quantization and pruning; (2) A variance-aware bounding technique using Bernstein's inequality to tighten safety certificates; and (3) An automated verification toolchain. Experimental results on ACAS Xu and computer vision benchmarks demonstrate that SimCert outperforms state-of-the-art baselines.

Jingyang Li, Fu Song, Guoqiang Li

Shanghai Jiao Tong University, Shanghai 200240, China · Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

Neural Network VerificationNeural Network Compression
AskPDF
Mar 15, 2026

High-Fidelity Compression of Seismic Velocity Models via SIREN Auto-Decoders

Implicit Neural Representations (INRs) have emerged as a powerful paradigm for representing continuous signals independently of grid resolution. In this paper, we propose a high-fidelity neural compression framework based on a SIREN (Sinusoidal Representation Networks) auto-decoder to represent multi-structural seismic velocity models from the OpenFWI benchmark. Our method compresses each 70x70 velocity map (4,900 points) into a compact 256-dimensional latent vector, achieving a compression ratio of 19:1. We evaluate the framework on 1,000 samples across five diverse geological families: FlatVel, CurveVel, FlatFault, CurveFault, and Style. Experimental results demonstrate an average PSNR of 32.47 dB and SSIM of 0.956, indicating high-quality reconstruction. Furthermore, we showcase two key advantages of our implicit representation: (1) smooth latent space interpolation that generates plausible intermediate velocity structures, and (2) zero-shot super-resolution capability that reconstructs velocity fields at arbitrary resolutions up to 280x280 without additional training. The results highlight the potential of INR-based auto-decoders for efficient storage, multi-scale analysis, and downstream geophysical applications such as full waveform inversion.

Caiyun Liu, Xiaoxue Luo, Jie Xiong

School of Information and Mathematics, Yangtze University, Jingzhou, Hubei 434023, P.R. China · School of Electronic Information and Electrical Engineering, Yangtze University, Jingzhou, Hubei 434023, P.R. China

Sinusoidal Representation NetworksModel CompressionNeural Network CompressionImplicit Neural Representations
AskPDF
Mar 9, 2026

Adaptive MLP Pruning for Large Vision Transformers

Large vision transformers present impressive scalability, as their performance can be well improved with increased model capacity. Nevertheless, their cumbersome parameters results in exorbitant computational and memory demands. By analyzing prevalent transformer structures, we find that multilayer perceptron (MLP) modules constitute the largest share of the model's parameters. In this paper, we propose an Adaptive MLP Pruning (AMP) method to substantially reduce the parameters of large vision transformers without obvious performance degradation. First, we adopt Taylor based method to evaluate neuron importance of MLP. However, the importance computation using one-hot cross entropy loss ignores the potential predictions on other categories, thus degrading the quality of the evaluated importance scores. To address this issue, we introduce label-free information entropy criterion to fully model the predictions of the original model for more accurate importance evaluation. Second, we rank the hidden neurons of MLP by the above importance scores and apply binary search algorithm to adaptively prune the ranked neurons according to the redundancy of different MLP modules, thereby avoiding the predefined compression ratio. Experimental results on several state-of-the-art large vision transformers, including CLIP and DINOv2, demonstrate that our method achieves roughly 40% parameter and FLOPs reduction in a near lossless manner. Moreover, when the models are not finetuned after pruning, our method outperforms other pruning methods by significantly large margin. The source code and trained weights are available at https://github.com/visresearch/AMP.

Chengchao Shen

Central South University

Neural Network CompressionNeural Network PruningEfficient ViTsVision Transformer
AskPDF
Mar 5, 2026

Structured Multidimensional Representation Learning for Large Language Models

Transformer architectures achieve state-of-the-art performance across a wide range of pattern recognition and natural language processing tasks, but their scaling is accompanied by substantial parameter growth and redundancy in the embedding dimension. In this work, we introduce a structured spectral factorization of the embedding space based on the L-product for third-order tensors. By reshaping token representations into spectral tensor slices and performing attention and feed-forward operations in the transform domain, we obtain a Tensor Transformer architecture that decomposes the encoder into p independent spectral sub-transformers while preserving standard Transformer semantics. We prove that the proposed L-Transformer is spectrally equivalent to p parallel Transformers operating on reduceddimensional embeddings, which yields approximately 1/p reduction (up to lower-order terms such as biases and normalization parameters) in encoder parameters under fixed total embedding size. When instantiated with a real-valued Discrete Cosine Transform (DCT), the method remains fully differentiable and compatible with existing training pipelines. Beyond compression, the spectral decomposition introduces an inductive bias over embedding frequencies, enabling slice-dependent frequency scaling that improves generalization. Experiments on IMDB and AGNews show that the proposed model can substantially reduce encoder parameters (up to 75% for p=4) while maintaining competitive accuracy. On IMDB, the tensorized encoder matches or improves upon the standard baseline under compression, whereas on AGNews at moderate width we observe a small accuracy decrease in exchange for a 4 times encoder reduction; at BERT-base width (d=768), performance returns to parity.

Alaa El Ichi, Khalide Jbilou, Mohamed El Guide +1

Universit´e du Littoral Cote d’Opale, LMPA, 50 rue F. Buisson, 62228 Calais-Cedex, France. · FGSES, University Mohammed VI Polytechnic, Rabat, Morocco · LISIC, 50 rue F. Buisson, Universit´e du Littoral Cote d’Opale, 62228 Calais-Cedex, France.

TransformerEmbedding CompressionNeural Network CompressionTensor Decomposition
AskPDF
Mar 5, 2026

A Benchmark Study of Neural Network Compression Methods for Hyperspectral Image Classification

Deep neural networks have achieved strong performance in image classification tasks due to their ability to learn complex patterns from high-dimensional data. However, their large computational and memory requirements often limit deployment on resource-constrained platforms such as remote sensing devices and edge systems. Network compression techniques have therefore been proposed to reduce model size and computational cost while maintaining predictive performance. In this study, we conduct a systematic evaluation of neural network compression methods for a remote sensing application, namely hyperspectral land cover classification. Specifically, we examine three widely used compression strategies for convolutional neural networks: pruning, quantization, and knowledge distillation. Experiments are conducted on two benchmark hyperspectral datasets, considering classification accuracy, memory consumption, and inference efficiency. Our results demonstrate that compressed models can significantly reduce model size and computational cost while maintaining competitive classification performance. These findings provide insights into the trade-offs between compression ratio, efficiency, and accuracy, and highlight the potential of compression techniques for enabling efficient deep learning deployment in remote sensing applications.

Sai Shi

Department of Computer and Information Science, Temple University, Philadelphia, USA

Neural Network CompressionNeural Network QuantizationNeural Network PruningKnowledge DistillationHyperspectral Image Classification
AskPDF
Feb 27, 2026

Causal Mechanism Reduction: Mechanism Replacement for Neural Network Pruning and Abstraction

Which internal mechanisms of a neural network can be replaced while preserving the computation it performs? Structured pruning asks for smaller deployable networks; causal abstraction asks for high-level models that commute with interventions. We introduce causal mechanism reduction (CMR), a framework that treats a trained network as a deterministic structural causal model and replaces selected internal variables by constants or affine functions of retained variables. These replacements compile exactly into smaller dense networks by bias and weight folding, and induce reduced causal models testable with interchange interventions. We derive a unified second-order replacement-risk objective whose special cases recover mean replacement, variance-based pruning (VBP), logit-distortion scoring, and affine neuron merging, together with a margin-based certificate linking logit distortion to interchange-intervention agreement. The framework also exposes a basic invariance requirement: functionally identical ReLU networks should induce the same reduction. Under exact positive-scaling reparameterizations, VBP's kept set collapses to chance-level overlap while the logit-distortion score is exactly invariant. Empirically, CMR variants are competitive with VBP under matched fine-tuning of DeiT-Tiny on ImageNet-100; the clearer separation appears in the invariance and interchange tests, where the logit-distortion score preserves kept sets and consistently improves distributional fidelity. CMR thus gives pruning, compilation, and causal-abstraction verification a common object to optimize and verify.

Amir Asiaee

Vanderbilt University Medical Center Nashville, TN

Structural Causal ModelsCausal AbstractionNeural Network CompressionNeural Network Pruning
AskPDF
Feb 27, 2026

GRAIL: Post-hoc Compensation by Linear Reconstruction for Compressed Networks

Structured deep model compression methods are hardware-friendly and substantially reduce memory and inference costs. However, under aggressive compression, the resulting accuracy degradation often necessitates post-compression finetuning, which can be impractical due to missing labeled data or high training cost. We propose post-hoc blockwise compensation, called GRAIL, a simple zero-finetuning step applied after model compression that restores each block's input-output behavior using a small calibration set. The method summarizes hidden activations via a Gram matrix and applies ridge regression to linearly reconstruct the original hidden representation from the reduced one. The resulting reconstruction map is absorbed into the downstream projection weights, while the upstream layer is compressed. The approach is selector-agnostic (Magnitude, Wanda, Gram-based selection, or folding), data-aware (requiring only a few forward passes without gradients or labels), and recovers classic pruning or folding when the Gram matrix is near identity, indicating weak inter-channel correlations. Across ResNets, ViTs, and decoder-only LLMs, GRAIL consistently improves accuracy or perplexity over data-free and data-aware pruning or folding baselines in practical compression regimes, with manageable overhead and no backpropagation. The code is available at https://github.com/TWWinde/GRAIL_Compensation.

Wenwu Tang, Dong Wang, Lothar Thiele +1

Graz University of Technology · ETH Zurich, Switzerland · Complexity Science Hub, Austria

Neural Network CompressionNeural Network Pruning
AskPDF
Feb 26, 2026

Isotropic Activation Functions Enable Deindividuated Neurons and Adaptive Topologies

Varying the number of neurons while preserving function is difficult because neural networks' dense connectivity creates strong computational interdependence between units. Elementwise activation functions, particularly their limited equivariance properties, are identified as a central hurdle to achieving this. We propose activation functions designed around chosen equivariances, enabling dense neural networks to adapt via singular value decomposition of their affine maps. This diagonalises layers into one-to-one, ordered connections, making it easier to assess the impact of individual connections on function. Low-impact neurons can be removed (neurodegeneration), and a thresholded buffer of largely inactive scaffold' neurons is maintained (neurogenesis). These symmetry-led diagonalisation and structural changes prioritise function-invariance: neurogenesis exactly preserves function, neurodegeneration is asymptotically function-preserving with the pruned singular-value threshold. Moreover, this construction enables asymptotic 50\% parameter sparsification of dense networks while preserving function analytically. Thus, the approach enables real-time restructuring of the architecture in response to task demands, task appending, removal or changes. Conceptually, this motivates broader applications of symmetry prescriptions at the small scale, even when no data-driven model-scale symmetry adherence is required. Following this, we build activations around symmetry, not from classical assumptions about individual neurons. Unlike typical elementwise forms, these alternatives feature explicit basis independence and do not individuate neurons. We leverage this to decompose and reinterpret neurons across sequential layers, simplifying structure and providing a natural route to adaptive topologies. Additional developments include a new tunable parameter, the intrinsic length'.

George Bird

Department of Computer Science, University of Manchester, UK

Dynamic Neural NetworksNeural Network Activation FunctionsNeural Network PruningNeural Network Compression
AskPDF
Feb 25, 2026

Compact Circulant Layers with Spectral Priors

Critical applications in areas such as medicine, robotics and autonomous systems require compact (i.e., memory efficient), uncertainty-aware neural networks suitable for edge and other resource-constrained deployments. We study compact spectral circulant and block-circulant-with-circulant-blocks (BCCB) layers: FFT-diagonalizable circular convolutions whose weights live directly in the real FFT (RFFT) half (1D) or half-plane (2D). Parameterizing filters in the frequency domain lets us impose simple spectral structure, perform structured variational inference in a low-dimensional weight space, and calculate exact layer spectral norms, enabling inexpensive global Lipschitz bounds and margin-based robustness diagnostics. By placing independent complex Gaussians on the Hermitian support we obtain a discrete instance of the spectral representation of stationary kernels, inducing an exact stationary Gaussian-process prior over filters on the discrete circle/torus. We exploit this to define a practical spectral prior and a Hermitian-aware low-rank-plus-diagonal variational posterior in real coordinates. Empirically, spectral circulant/BCCB layers are effective compact building blocks in both (variational) Bayesian and point estimate regimes: compact Bayesian neural networks on MNIST->Fashion-MNIST, variational heads on frozen CIFAR-10 features, and deterministic ViT projections on CIFAR-10/Tiny ImageNet; spectral layers match strong baselines while using substantially fewer parameters and with tighter Lipschitz certificates.

Joseph Margaryan, Thomas Hamelryck

Department of Computer Science (DIKU), University of Copenhagen, Copenhagen, Denmark · Department of Biology, University of Copenhagen, Copenhagen, Denmark

Neural Network CompressionBayesian Neural Networks
AskPDF
Feb 24, 2026

Sensitivity-Guided Framework for Pruned and Quantized Reservoir Computing Accelerators

This paper presents a compression framework for Reservoir Computing that enables systematic design-space exploration of trade-offs among quantization levels, pruning rates, model accuracy, and hardware efficiency. The proposed approach leverages a sensitivity-based pruning mechanism to identify and remove less critical quantized weights with minimal impact on model accuracy, thereby reducing computational overhead while preserving accuracy. We perform an extensive trade-off analysis to validate the effectiveness of the proposed framework and the impact of pruning and quantization on model performance and hardware parameters. For this evaluation, we employ three time-series datasets, including both classification and regression tasks. Experimental results across selected benchmarks demonstrate that our proposed approach maintains high accuracy while substantially improving computational and resource efficiency in FPGA-based implementations, with variations observed across different configurations and time series applications. For instance, for the MELBOEN dataset, an accelerator quantized to 4-bit at a 15% pruning rate reduces resource utilization by 1.2% and the Power Delay Product (PDP) by 50.8% compared to an unpruned model, without any noticeable degradation in accuracy.

Atousa Jafari, Mahdi Taheri, Hassan Ghasemzadeh Mohammadi +2

Paderborn University, Paderborn, Germany · Brandenburg Technical University, Cottbus, Germany · Tallinn University of Technology, Tallinn, Estonia +1

Reservoir ComputingNeural Network CompressionNeural Network QuantizationFPGA-Based Neural Network AccelerationNeural Network Pruning
AskPDF
Feb 24, 2026

Elimination-compensation pruning for fully-connected neural networks

The unmatched ability of Deep Neural Networks in capturing complex patterns in large and noisy datasets is often associated with their large hypothesis space, and consequently to the vast amount of parameters that characterize model architectures. Pruning techniques affirmed themselves as valid tools to extract sparse representations of neural networks parameters, carefully balancing between compression and preservation of information. However, a fundamental assumption behind pruning is that expendable weights should have small impact on the error of the network, while highly important weights should tend to have a larger influence on the inference. We argue that this idea could be generalized; what if a weight is not simply removed but also compensated with a perturbation of the adjacent bias, which does not contribute to the network sparsity? Our work introduces a novel pruning method in which the importance measure of each weight is computed considering the output behavior after an optimal perturbation of its adjacent bias, efficiently computable by automatic differentiation. These perturbations can be then applied directly after the removal of each weight, independently of each other. After deriving analytical expressions for the aforementioned quantities, numerical experiments are conducted to benchmark this technique against some of the most popular pruning strategies, demonstrating an intrinsic efficiency of the proposed approach in very diverse machine learning scenarios. Finally, our findings are discussed and the theoretical implications of our results are presented.

Enrico Ballini, Luca Muscarnera, Alessio Fumagalli +2

MOX, Department of Mathematics, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, Italy

Neural Network PruningNeural Network Compression
AskPDF
Feb 22, 2026

Mapping Networks

The escalating parameter counts in modern deep learning models pose a fundamental challenge to efficient training and resolution of overfitting. We address this by introducing the Mapping Networks which replace the high dimensional weight space by a compact, trainable latent vector based on the hypothesis that the trained parameters of large networks reside on smooth, low-dimensional manifolds. Henceforth, the Mapping Theorem enforced by a dedicated Mapping Loss, shows the existence of a mapping from this latent space to the target weight space both theoretically and in practice. Mapping Networks significantly reduce overfitting and achieve comparable to better performance than target network across complex vision and sequence tasks, including Image Classification, Deepfake Detection etc, with 99.5%\mathbf{99.5\%}, i.e., around 500×500\times reduction in trainable parameters.

Lord Sen, Shyamapada Mukherjee

National Institute of Technology Rourkela, Odisha, India

Weight-Space LearningNeural Network CompressionNeural Network GeneralizationHypernetworks
AskPDF
Feb 20, 2026

Cut Less, Fold More: Model Compression through the Lens of Projection Geometry

Compressing neural networks without retraining is vital for deployment at scale. We study calibration-free compression through the lens of projection geometry: structured pruning is an axis-aligned projection, whereas model folding performs a low-rank projection via weight clustering. We formalize both as orthogonal operators and show that, within a rank distance of one, folding provably yields smaller parameter reconstruction error, and under mild smoothness assumptions, smaller functional perturbations than pruning. At scale, we evaluate >1000 checkpoints spanning ResNet18, PreActResNet18, ViT-B/32, and CLIP ViT-B/32 on CIFAR-10 and ImageNet-1K, covering diverse training hyperparameters (optimizers, learning rates, augmentations, regularization, sharpness-aware training), as well as multiple LLaMA-family 60M and 130M parameter models trained on C4. We show that folding typically achieves higher post-compression accuracy, with the largest gains at moderate-high compression. The gap narrows and occasionally reverses at specific training setups. Our results position folding as a geometry-aware, calibration-free alternative to pruning that is often superior in practice and principled in theory.

Olga Saukh, Dong Wang, Haris Šikić +2

Graz University of Technology · Complexity Science Hub · Swiss Data Science Center +1

Model CompressionNeural Network CompressionNeural Network Pruning
AskPDF
Feb 19, 2026

Investigating Target Class Influence on Neural Network Compressibility for Energy-Autonomous Avian Monitoring

Biodiversity loss poses a significant threat to humanity, making wildlife monitoring essential for assessing ecosystem health. Avian species are ideal subjects for this due to their popularity and the ease of identifying them through their distinctive songs. Traditionalavian monitoring methods require manual counting and are therefore costly and inefficient. In passive acoustic monitoring, soundscapes are recorded over long periods of time. The recordings are analyzed to identify bird species afterwards. Machine learning methods have greatly expedited this process in a wide range of species and environments, however, existing solutions require complex models and substantial computational resources. Instead, we propose running machine learning models on inexpensive microcontroller units (MCUs) directly in the field. Due to the resulting hardware and energy constraints, efficient artificial intelligence (AI) architecture is required. In this paper, we present our method for avian monitoring on MCUs. We trained and compressed models for various numbers of target classes to assess the detection of multiple bird species on edge devices and evaluate the influence of the number of species on the compressibility of neural networks. Our results demonstrate significant compression rates with minimal performance loss. We also provide benchmarking results for different hardware platforms and evaluate the feasibility of deploying energy-autonomous devices.

Nina Brolich, Simon Geis, Maximilian Kasper +3

Fraunhofer Institute for Integrated Circuits IIS, Germany · Fachhochschule Erfurt, Germany · Universität Erfurt, Germany +2

Bird Vocalization ClassificationEnergy-Efficient MLTiny MLNeural Network CompressionAudio ClassificationPassive Acoustic Monitoring
AskPDF
Feb 19, 2026

Sign Lock-In: Randomly Initialized Weight Signs Persist and Bottleneck Sub-Bit Model Compression

Sub-bit model compression targets storage below one bit per weight; as magnitudes are aggressively compressed, the sign bit becomes a fixed-cost bottleneck. Across Transformers, CNNs, and MLPs, learned sign matrices resist low-rank approximation and are spectrally indistinguishable from an i.i.d. Rademacher baseline. This randomness gives rise to the lower bound of sub-bit model compression -- the one-bit wall. Despite this apparent randomness, most weights retain their initialization signs; flips primarily occur via rare near-zero boundary crossings, suggesting that sign-pattern randomness is largely inherited from initialization. We formalize this behavior with sign lock-in theory, a stopping-time analysis of sign flips under SGD noise. Under bounded updates and a rare re-entry condition into a small neighborhood of zero, the number of effective sign flips exhibits a geometric tail. Building on this mechanism, we introduce a from-scratch low-rank sign-template training method that prevents the emergence of this one-bit wall.

Akira Sakai, Yuma Ichikawa

Fujitsu Limited · Tokai University · Riken Center for AIP

Model CompressionNeural Network CompressionNeural Network Initialization
AskPDF
Feb 19, 2026

Amber-Image: Efficient Compression of Large-Scale Diffusion Transformers

Diffusion Transformer (DiT) architectures have significantly advanced Text-to-Image (T2I) generation but suffer from prohibitive computational costs and deployment barriers. To address these challenges, we propose an efficient compression framework that transforms the 60-layer dual-stream MMDiT-based Qwen-Image into lightweight models without training from scratch. Leveraging this framework, we introduce Amber-Image, a series of streamlined T2I models. We first derive Amber-Image-10B using a timestep-sensitive depth pruning strategy, where retained layers are reinitialized via local weight averaging and optimized through layer-wise distillation and full-parameter fine-tuning. Building on this, we develop Amber-Image-6B by introducing a hybrid-stream architecture that converts deep-layer dual streams into a single stream initialized from the image branch, further refined via progressive distillation and lightweight fine-tuning. Our approach reduces parameters by 70% and eliminates the need for large-scale data engineering. Notably, the entire compression and training pipeline-from the 10B to the 6B variant-requires fewer than 2,000 GPU hours, demonstrating exceptional cost-efficiency compared to training from scratch. Extensive evaluations on benchmarks like DPG-Bench and LongText-Bench show that Amber-Image achieves high-fidelity synthesis and superior text rendering, matching much larger models.

Chaojie Yang, Tian Li, Yue Zhang +1

Computational Intelligence Dept, HelloGroup Inc.

Diffusion Model CompressionT2I Diffusion ModelsDiffusion TransformerNeural Network CompressionDiffusion Model DistillationStructured Pruning
AskPDF
Feb 18, 2026

LegoNet: Memory Footprint Reduction Through Block Weight Clustering

As the need for neural network-based applications to become more accurate and powerful grows, so too does their size and memory footprint. With embedded devices, whose cache and RAM are limited, this growth hinders their ability to leverage state-of-the-art neural network architectures. In this work, we propose LegoNet, a compression technique that constructs blocks of weights of the entire model regardless of layer type and clusters these induced blocks. Using blocks instead of individual values to cluster the weights, we were able to compress ResNet-50 trained for Cifar-10 and ImageNet with only 32 4x4 blocks, compressing the memory footprint by over a factor of 64x without having to remove any weights or changing the architecture and no loss to accuracy, nor retraining or any data, and show how to find an arrangement of 16 4x4 blocks that gives a compression ratio of 128x with less than 3% accuracy loss. This was all achieved with no need for (re)training or fine-tuning.

Joseph Bingham, Noah Green, Saman Zonouz

Dept of Electrical and Computer Engineering, Rutgers University, New Brunswick, New Jersey

Neural Network Compression
AskPDF
Feb 16, 2026

COMPOT: Calibration-Optimized Matrix Procrustes Orthogonalization for Transformers Compression

Post-training compression of transformer models commonly relies on truncated singular value decomposition (SVD) strategies leading to low-rank approximations of the original model weights. However, the underlying single shared subspace modeling can degrade accuracy even at moderate compression. Sparse dictionary learning provides a more flexible union-of-subspaces representation, but existing approaches often suffer from costly iterative dictionary and coefficient updates. We propose COMPOT\textbf{COMPOT} (C\textbf{C}alibration-O\textbf{O}ptimized M\textbf{M}atrix P\textbf{P}rocrustes O\textbf{O}rthogonalization for T\textbf{T}ransformers), a training-free compression framework that uses a small calibration dataset to estimate a sparse weight factorization. COMPOT employs orthogonal dictionaries that enable closed-form Procrustes updates for the dictionary and analytical single-step sparse coding for the coefficients, reducing significantly the overall optimization cost. To handle heterogeneous layer sensitivity under a global compression budget, COMPOT further introduces a one-shot dynamic allocation strategy that adaptively redistributes layer-wise compression rates. Extensive experiments across diverse architectures and tasks show that COMPOT consistently delivers a superior quality-compression trade-off over strong low-rank and sparse baselines, while remaining fully compatible with post-training quantization for extreme compression.

Denis Makhov, Dmitriy Shopkhoev, Magauiya Zhussip +3

Fundamental Research Center MWS AI · ITMO

Orthogonal Procrustes AlignmentModel CompressionNeural Network Compression
AskPDF
Feb 16, 2026

Algorithmic Simplification of Neural Networks with Mosaic-of-Motifs

Large-scale deep learning models are well-suited for compression. Across a variety of tasks, methods like pruning, quantization, and knowledge distillation have been used to achieve massive reductions in model parameters with only marginal performance drops. This raises the central question: Why are deep neural networks suited for compression? In this work, we take up the perspective of algorithmic complexity to explain this behavior. We hypothesize that the parameters of trained models have more structure and, hence, exhibit lower algorithmic complexity compared to the weights at (random) initialization. Furthermore, model compression methods harness this reduced algorithmic complexity to compress models. Although an unconstrained parameterization of model weights, w∈Rn\mathbf{w} \in \mathbb{R}^n, can represent arbitrary weight assignments, the solutions found during training exhibit repeatability and structure, making them simpler to implement than a trivial program. To this end, we formalize the Kolmogorov complexity of w\mathbf{w} by K(w)\mathcal{K}(\mathbf{w}). We introduce a constrained parameterization w^\widehat{\mathbf{w}} that partitions parameters into blocks of size ss and restricts each block to be selected from a set of kk reusable motifs, specified by a reuse pattern (or mosaic). The resulting method, Mosaic-of-Motifs\mathit{Mosaic\text{-}of\text{-}Motifs} (MoMos), provides a theoretically justified parameterization that biases optimization toward algorithmically simpler solutions. Empirical evidence from multiple experiments shows that MoMos consistently lowers the algorithmic complexity of neural networks during training while preserving the performance of unconstrained models. These results suggest that parameter compressibility is not only observed after training, but can be induced from the optimization domain.

Pedram Bakhtiarifard, Tong Chen, Jonathan Wenshøj +2

Department of Computer Science, University of Copenhagen, Denmark

Kolmogorov ComplexityNeural Network CompressionNeural Network Optimization
AskPDF
Feb 16, 2026

RNM-TD3: N:M Semi-structured Sparse Reinforcement Learning From Scratch

Sparsity is a well-studied technique for compressing deep neural networks (DNNs) without compromising performance. In deep reinforcement learning (DRL), neural networks with up to 5% of their original weights can still be trained with minimal performance loss compared to their dense counterparts. However, most existing methods rely on unstructured fine-grained sparsity, which limits hardware acceleration opportunities due to irregular computation patterns. Structured coarse-grained sparsity enables hardware acceleration, yet typically degrades performance and increases pruning complexity. In this work, we present, to the best of our knowledge, the first study on N:M structured sparsity in RL, which balances compression, performance, and hardware efficiency. Our framework enforces row-wise N:M sparsity throughout training for all networks in off-policy RL (TD3), maintaining compatibility with accelerators that support N:M sparse matrix operations. Experiments on continuous-control benchmarks show that RNM-TD3, our N:M sparse agent, outperforms its dense counterpart at 50%-75% sparsity (e.g., 2:4 and 1:4), achieving up to a 14% increase in performance at 2:4 sparsity on the Ant environment. RNM-TD3 remains competitive even at 87.5% sparsity (1:8), while enabling potential training speedups.

Isam Vrce, Andreas Kassler, Gökçe Aydos

Deggendorf Institute of Technology, Dieter-Görlitz-Platz 1, Deggendorf, Germany · Department of Engineering Technology, Technical University of Denmark, Ballerup, Denmark

Structured SparsityNeural Network Compression
AskPDF
Oct 31, 2025

Exploiting heterogeneous delays for efficient computation in low-bit neural networks

Neural networks rely on learning synaptic weights. However, this overlooks other learnable neural parameters. One such parameter is delay: neurons communicate asynchronously with heterogeneous transmission delays. We propose that, rather than a cost to minimize, this heterogeneity supports computation when task-relevant information unfolds over time. We test this hypothesis by jointly learning weights and delays at different precisions in feedforward spiking neural networks. Delay heterogeneity supports competitive performance on neuromorphic benchmarks while compressing weights over an order of magnitude more aggressively than typically studied weight-only networks. Performance depends on task-matched delay distributions, with temporally complex tasks selectively relying on longer delays. These benefits persist under explicit delay constraints, while neural time constants only partially substitute for long delays. Our results connect the temporal structure of data with efficient model design, highlighting temporal heterogeneity as a computational resource with implications for embodied intelligence and neuromorphic hardware.

Pengfei Sun, Jascha Achterberg, Zhe Su +2

Department of Electrical and Electronic Engineering, Imperial College London. · Centre for Neural Circuits and Behaviour, University of Oxford. · Institute of Neuroinformatics, University of Zurich and ETH Zurich. +2

Neuromorphic ComputingSpiking Neural NetworksNeural Network Compression
AskPDF
Oct 16, 2025

Real-Time Neural Video Compression with Unified Intra and Inter Coding

Neural video compression (NVC) technologies have advanced rapidly in recent years, yielding state-of-the-art schemes such as DCVC-RT that offer superior compression efficiency to H.266/VVC and real-time encoding/decoding capabilities. Nonetheless, existing NVC schemes have several limitations, including inefficiency in dealing with disocclusion and new content, interframe error propagation and accumulation, among others. To eliminate these limitations, we borrow the idea from classic video coding schemes, which allow intra coding within inter-coded frames. With the intra coding tool enabled, disocclusion and new content are properly handled, and interframe error propagation is naturally intercepted without the need for manual refresh mechanisms. We present an NVC framework with unified intra and inter coding, where every frame is processed by a single model that is trained to perform intra/inter coding adaptively. Moreover, we propose a simultaneous two-frame compression design to exploit interframe redundancy not only forwardly but also backwardly. Experimental results show that our scheme outperforms DCVC-RT by an average of 12.1% BD-rate reduction, delivers more stable bitrate and quality per frame, and retains real-time encoding/decoding performances. Code and models will be released.

Hui Xiang, Yifan Bian, Li Li +3

University of Science and Technology of China · Tencent Shannon Lab

Learned Image CompressionNeural Video CompressionVideo CompressionNeural Network Compression
AskPDF
Sep 29, 2025

BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

Activation-aware low-rank factorization techniques yield strong compression results but are generally confined to linear layers, while existing whitening-based theory typically makes an implicit full-rank assumption on activations. We introduce a layer representation framework that extends activation-aware factorization beyond linear layers, including standard and grouped convolutions. Within this framework, our whitening-based formulation is more general than prior ones, naturally covering rank-deficient activations, and yields an optimal low-rank projection that attains the reconstruction error of the best low-rank approximation to layer activations. The resulting singular spectrum provides a closed-form per-layer distortion proxy, which we use to allocate per-layer ranks under explicit FLOP or parameter-count budgets via a Lagrangian relaxation with negligible overhead. Together, these components form BALF, an end-to-end pipeline for efficient vision model compression. Across CNNs and vision transformers on CIFAR-10 and ImageNet-1K, BALF generally achieves higher accuracy than SVD-based factorization baselines at matched FLOP or parameter count targets and remains competitive with other fine-tuning-free compression techniques.

David González-Martínez

Department of Computer Science and Artificial Intelligence, Universidad de Sevilla · Tübingen AI Center

Low-Rank Matrix DecompositionLow-Rank CompressionLow-Rank ApproximationNeural Network Compression
AskPDF
Jul 23, 2025

On the Interaction of Compressibility and Adversarial Robustness

As demands for resource efficiency and safety in modern neural networks intensify, substantial research effort has gone into model compression and adversarial robustness. Yet despite progress on each in isolation, a systematic understanding of how compressibility shapes robustness remains elusive. In this paper, we develop a principled framework to analyze how different forms of structured compressibility - such as neuron-level and spectral compressibility - affect adversarial robustness. We show that structured compressibility can induce a small number of highly sensitive directions in the representation space, which adversaries can exploit to construct effective perturbations. Our analysis yields a robustness bound that reveals how neuron and spectral compressibility impact ℓ∞\ell_\infty and ℓ2\ell_2 robustness via their effects on the learned representations. Crucially, the vulnerabilities we identify arise irrespective of how compressibility is achieved - whether via regularization, architectural bias, or learning dynamics. Through empirical evaluations across synthetic and realistic tasks, we confirm our theoretical predictions, and further demonstrate that these vulnerabilities persist under adversarial training and transfer learning, and contribute to the emergence of universal adversarial examples. Our findings show a fundamental tension between structured compressibility and robustness and highlight new pathways for designing models that are efficient and safe.

Melih Barsbey, Antônio H. Ribeiro, Umut Şimşekli +1

Department of Computing, Imperial College London, UK · Department of Information Technology, Uppsala University, Sweden · INRIA, CNRS, Département d’Informatique de l’Ecole Normale Supérieure / PSL, France

Neural Network RobustnessAdversarial RobustnessAdversarial PerturbationsNeural Network Compression
AskPDF