Pruning Binarized Neural Networks: A Dedicated Framework and Globally Weighted Algorithms

Roan Rubiales; Jean Pierre David

cs.LGAug 26, 2026

Pruning Binarized Neural Networks: A Dedicated Framework and Globally Weighted Algorithms

Authors: Roan Rubiales, Jean Pierre David

Organizations: Polytechnique Montreal

Abstract

Extreme compression of deep neural networks, up to full binarization, dramatically reduces memory footprint and arithmetic complexity, facilitating deployment on constrained edge hardware with field-programmable gate arrays (FPGAs) and microcontrollers. Although combining binarization with pruning promises additional efficiency gains, existing pruning strategies are ill-suited to binarized representations and rarely translate into meaningful hardware savings. We introduce a PyTorch-based, research-oriented framework that incorporates freezing and pruning mechanisms for designing and optimizing binarized neural networks. The framework enables rapid and reproducible evaluation of state-of-the-art approaches and the fast prototyping of new ones. Leveraging this framework, we propose a novel pruning method that accounts for the relative importance of learned parameters across abstraction levels. Such a global weighting mechanism consistently achieves a superior trade-off between model accuracy and pruning rate, achieving a 70% pruning rate on VGG11 with constant accuracy, while state-of-the-art results reach only 41% in the binarized setting.

Figures & tables

Figure 1: Block diagram describing our BNN framework.

Method	Baseline accuracy (%)	Pruned accuracy (%)	Weights removed (%)
Layer sensitivity	80.6	80.9	27.0
Layer sensitivity*	80.5	80.7	27.4
Layer sensitivity**	89.7	89.6	32.8
Algorithm 1	89.7	89.3	46.5
Algorithm 2	89.7	89.7	45.0
Algorithm 3	89.7	89.7	27.9

Table 1: Comparison of pruning algorithms on the BinaryNet architecture (14M parameters). * Result reproduced in our environment without data augmentation (as in the original paper). ** Result reproduced in our environment with data augmentation.

Method	Baseline accuracy (%)	Pruned accuracy (%)	Weights removed (%)
Ternary pruning	84.4	84.4	41.0
Ternary pruning*	84.8	84.4	41.0
Algorithm 2	84.8	84.4	70.0
Algorithm 3	84.8	84.8	56.3

Table 2: Comparison of pruning algorithms test accuracies and pruning rates on the VGG11 architecture (28M parameters). * Result reproduced in our environment.

Method	Baseline accuracy (%)	Pruned accuracy (%)	Weights removed (%)
Weight-flipping	86.5	86.0	16.6
Ternary pruning	85.3	84.1	39.0
Algorithm 2	84.6	84.6	49.9

Table 3: Comparison of pruning algorithms on the NiN architecture (966k parameters).

Panel of Figure 2 — Figure 2: Landscapes of single pruning thresholds and measured losses for several global weighting transforms, obtained through algorithm 2 .

Target pruning rate (%)	Method	Validation loss after pruning	Baseline test accuracy (%)	Pruned test accuracy (%)
45.0	Magnitude only	$0.539$	$89.65$	$89.38$
	BatchNorm folding	$0.473$		$89.68$
	BatchNorm folding + L $\infty$ layer norm	$0.507$		$89.04$
55.0	Magnitude only	$0.575$	$89.65$	$88.45$
	BatchNorm folding	$0.462$		$89.41$
	BatchNorm folding + L $\infty$ layer norm	$0.553$		$89.05$

Table 4: Performance of each tested global weighting transformation on BinaryNet at different pruning rates

Model	Base size	Pruned size	Base BMACs	Pruned BMACs
BinaryNet	1.76MB	0.97MB	617M	479M
VGG11	3.53MB	1.08MB	172M	122M
NiN	0.18MB	0.12MB	222M	127M

Table 5: Performance metrics of our models on their best result in pruning rate while keeping accuracy constant.

Figure 3: Pruning rates of each layer in VGG11 pruned by single threshold sweep (see Table 2 ).

Explore similar work

Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
May 23, 2026Leonid Berlyand, Theo Bourdais, Houman Owhadi +1Structured Pruning Sparse Neural Networks
Department of Mathematics Pennsylvania State University University Park, PA 16802, USA · Department of Computing and Mathematical Sciences California Institute of Technology Pasadena, CA 91125, USA
Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference
Mar 16, 2026Nitin Priyadarshini Shankar, Soham Lahiri, Sheetal Kalyani +1Binary Neural Networks Federated Learning
Department of Electrical Engineering, Indian Institute of Technology Madras · Department of Computer Science and Engineering, Jadavpur University
Bonsai: A Framework for Convolutional Neural Network Acceleration Using Criterion-Based Pruning
Feb 19, 2026Joseph Bingham, Sam HelmichNeural Network Pruning Efficient Neural Network Inference
Rutgers University, New Brunswick NJ 08901, USA · John Deere Intelligent Solutions Group, Urbandale, IA 50322, USA

May 23, 2026J/K move · Enter open · S save

Pruning Deep Neural Networks via the Marchenko--Pastur Distribution

Leonid Berlyand, Theo Bourdais, Houman Owhadi, Yitzchak Shmalo

Department of Mathematics Pennsylvania State University University Park, PA 16802, USA · Department of Computing and Mathematical Sciences California Institute of Technology Pasadena, CA 91125, USA

Structured Pruning Sparse Neural Networks Image Classification Neural Network Pruning

Open paper AskPDF arXiv

We study a Marchenko--Pastur (MP) random-matrix approach to pruning deep neural networks with very small post-pruning fine-tuning budgets. The main practical contribution is accuracy retention under short calibration and fine-tuning schedules, rather than a long post-pruning reoptimization pipeline. The theory gives deterministic data-path certificates: if the removed component

R

has small propagated logit effect

L_s \| R ψ_1(s) \|_\infty

, pruning decreases an elastic-net objective and preserves samples whose dense margin exceeds twice the perturbation. The zero-budget case gives perfect pruning; a prune--restore extension models weight restoration inside a fixed sparse-execution pattern; and an additive

L_2

-regularized model shows admissible random-like components vanish at the training limit, with persistent spikes stabilizing as the MP bulk collapses. Under iid-Gaussian sufficient conditions, the fitted MP edge

σ_+

gives a high-probability layerwise budget signal. On ImageNet-1k, after only three distillation epochs, ViT-B/16

2{:}4{+}

ToMe reaches

83.41\%

top-1 (

-1.70

pp from dense) at

59.81\%

sparse-execution MAC reduction, with

1.388\times

best-observed A40 native-

2{:}4

backend speedup for the same checkpoint and ToMe graph; a separate no-ToMe A100 endpoint gives

2.705\times

. At structured sparsity, ViT-B/16

6{:}12

reaches

83.74\%

, ViT-L/16

8{:}16

dense+permutation reaches

85.33\%

(

-0.51

pp), and ConvNeXtV2-Base

12{:}16

reaches

86.35\%

(

-0.37

pp). For CNNs, ResNet50

8{:}16

dense+permutation reaches

75.87\%

(

-0.26

pp), and ResNet152d CAST-conv+permutation reaches

81.33\%

(

-1.53

pp) at

{\sim}50\%

MAC accounting with a

1.62\times

A40 im2col

+2{:}4

sparse-GEMM audit.

Abstract

Explore similar work

Pruning Deep Neural Networks via the Marchenko--Pastur Distribution

Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference

Bonsai: A Framework for Convolutional Neural Network Acceleration Using Criterion-Based Pruning