Pruning reduces the number of weights in magnetic resonance imaging (MRI) reconstruction networks. Equal sparsity, however, can retain weights with different computational roles and different compatibility with the trained network. We study these effects across 120 convolutional U-Net and vision transformer models using controlled edits evaluated before retraining. In U-Net, preserving high-resolution computation improves reconstruction at equal deletion counts across all tested sparsity levels. Equal-operation controls reveal additional sensitivity of the first convolution, which operation count alone cannot explain. In transformers, retained pretrained weight energy correlates with quality within sparsity levels, yet the same masks reverse their reconstruction ranking when the surrounding trained weights change. At 90% sparsity, matching intermediate output scales largely removes a high-energy-mask penalty while retaining an advantage for the network's original mask. Thus sparsity alone does not characterize reconstruction quality: architecture-specific descriptors are informative, but mask quality can still depend on the surrounding trained network.
Figures & tables
Model
Descriptor
fastMRI
M4Raw
U-Net
Active count
−.17 to .39
−.34 to .19
Operations
.54 to .72
.14 to .69
ViT
Energy
.81 to .97
.77 to .96
Table 1: Within-sparsity Pearson correlations with PSNR. Entries span five levels, with 12 trained models per level. ViT energy uses Eq. 2 . These are descriptive associations.
Sparsity
F
F+J
M
M+J
50%
3.49-8.72
1.28-4.34
1.28-4.22
.81-3.01
60%
2.30-9.20
.86-5.17
.64-4.44
.35-3.45
70%
1.52-8.07
.36-4.41
.85-4.21
.43-3.01
80%
2.11-7.66
.06-4.22
.95-3.74
.53-2.89
90%
2.35-7.89
.79-4.48
.87-4.33
.48-3.27
Table 2: U-Net PSNR difference H minus L (dB). H deletes at low resolution first, L at high resolution first, with equal counts. Each range spans 12 models after averaging five deletion seeds. F denotes fastMRI, M denotes M4Raw, and J denotes sampling jitter. All 240 means are positive.
Figure 1: M4Raw reconstruction from a SNIP U-Net at 70% sparsity, using the first deletion seed. H preserves high-resolution computation, while L deletes it first. The slice was selected before visual inspection as the example closest to the median H-minus-L PSNR difference. Labels give PSNR in dB. Image intensities and absolute errors relative to the target use shared ranges [0,1] and [0,.2] .
Figure 2: ViT mask exchange on M4Raw, with 144 combinations per sparsity level. Rows select trained context and columns select masks in decreasing retained energy. S denotes SNIP, R random, P PUN-IT, and G RigL. Color shows PSNR relative to each context’s median across 12 masks, clipped at ±6 dB. The line separates the three highest-energy masks. High-energy masks benefit SNIP contexts, while their preference reverses in random contexts at 90% sparsity. All MLP values are reset to the shared pretrained reference.
Model family
First minus
Encoder minus
adjacent
decoder
PUN-IT
−1.43
−.05 to .08
RigL
−1.09
−.04 to .01
SNIP
−2.87 to −1.35
−.18 to .14
Random
−.13 to −.05
−.34 to .09
Table 3: Equal-count, equal-operation U-Net deletions on M4Raw at 70% sparsity. Entries are scale-invariant PSNR differences (dB). First minus adjacent compares the first convolution with its neighbor. Encoder minus decoder spans four matched resolutions. Negative means the first-named location is more damaging. Ranges span masks and, in the last column, resolutions.
Trained context
Difference (dB)
Interval
SNIP
+4.243
[4.034,4.458]
Random
−.177
[−.566,.235]
PUN-IT
+.513
[−.034,1.098]
RigL
−.474
[−.847,−.061]
Table 4: SNIP-mask minus RigL-mask PSNR on M4Raw at 70% ViT sparsity. All masks use shared pretrained MLP values and 94.935 GFLOPs. Brackets give group-bootstrap intervals adjusted within 16 comparisons. Positive favors SNIP.
Figure 3: Controlled retained-energy test on M4Raw. Masks keep equal counts and have zero overlap with the context’s original mask (Eq. 3 ). Each line evaluates low-energy, random and high-energy selections in one of two randomly pruned contexts. Fitted energy-quality slopes are positive at 60 to 80% sparsity and negative at 90%. Each line contains only three masks, and the middle-energy mask is best in one 90% context.
Original-mask advantage
50%
70%
90%
Before retraining
1.73
2.45
4.64
After one epoch
.95
1.35
1.85
Table 5: ViT controls on M4Raw (dB). Top: original-mask advantage over the 11 foreign masks, averaged over 12 contexts per level, before and after one training epoch. Bottom: the 90% scale-control comparison. Original-mask lead uses the same contrast. High-energy effect is the reported high-energy-minus-count-matched-random mask contrast. Scale factors use 32 unlabeled slices.
At very high sparsity, neural network pruning does more than decide which weights remain. It also determines where pruning induced damage is placed across the network, and whether that damage can be recovered by a fixed lightweight repair procedure. We study this problem through the lens of repair conditioned sparsity allocation. We introduce Relative Repairability (RR), a calibration based diagnostic that compares the raw activation distortion caused by layerwise pruning with the residual distortion left after channelwise variance matching repair. RR estimates the fraction of local damage that remains after repair, using only unlabeled calibration data. Across ResNet18, ResNet34, and VGG16 BN on CIFAR10 and CIFAR100, we find that RR is not a universally dominant allocation rule. Instead, it is most useful near an architecture dependent recoverability transition, where standard structural or magnitude based allocation priors begin to lose reliability but post repair recovery has not yet fully collapsed. On CIFAR100 ResNet18, a fine grained sweep shows that RR improves over ERK across the central transition band and surpasses LAMP near the upper part of this band. A projection forced ablation further shows that capped ERK can over protect projection layers, shifting excessive sparsity onto regular convolutions and reducing post repair recovery. These results suggest that high sparsity pruning should allocate not only retained weights, but also repairable damage.
Qishi Zhan, Liang He, Minxuan Hu +1
Marquette University · Tongji University · Cornell University +1
Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance. Although the Lottery Ticket Hypothesis (LTH) shows that sparse subnetworks can match dense networks when trained from suitable initializations, its iterative pruning procedure requires multiple complete training cycles. This work evaluates progressive magnitude-based pruning as a single-cycle alternative. The method gradually increases sparsity during training using a linear schedule and updates pruning masks based on active weight magnitudes. We conduct systematic experiments on CIFAR-10 and MNIST across ResNet, VGG-style, and LeNet architectures, comparing the proposed method with representative iterative and initialization-based pruning baselines, including LTH, SNIP, and GraSP. On CIFAR-10, the method achieves 95.12% accuracy on ResNet-18 at 72.9% sparsity, compared with 90.5% reported for LTH. At extreme sparsity, it achieves 93.13% accuracy on a VGG-like architecture at 97% sparsity, compared with approximately 92.0% for SNIP, and 93.44% accuracy on VGG-19 at 97.97% sparsity, compared with 92.19% for GraSP at 98% sparsity. A sparsity-accuracy analysis on ResNet-18 further shows that accuracy remains within 0.1 percentage points of the dense baseline across 70--85% sparsity. These results indicate that progressive magnitude-based pruning provides an effective single-cycle approach for neural network sparsification under the evaluated settings.
Romana Qureshi, Hafida Benhidour, Said Kerrache +1
Department of Computer Science, King Saud University, Riyadh, Saudi Arabia
Unstructured magnitude pruning at high sparsity can reduce neural network accuracy to near-random performance, while labeled retraining may be unavailable in practical deployment settings. Label-free post-pruning repair methods can partially recover collapsed sparse models, but their effectiveness depends on the sparse model left by the upstream pruning allocation. This paper studies how sparsity allocation shapes post-repair recoverability under a fixed activation-statistic repair backend. We compare ERK and LAMP allocations under the same label-free repair protocol across CIFAR-10, CIFAR-100, and Imagenette with ResNet-18, ResNet-34, and ResNet-50 at sparsities from 90% to 95.5%. The results show that allocation choice can substantially change post-repair accuracy at the same global sparsity, and that the preferred allocation varies with architecture, dataset difficulty, and sparsity level. We identify a repair-sensitive transition regime in which BatchNorm recalibration begins to fail, while activation-statistic repair still recovers nontrivial accuracy. Additional validation on ImageNet-100 and DenseNet-121 shows that the location and width of this recoverable regime depend on data scale and connectivity structure. These findings suggest that pruning allocation and post-pruning repair should be studied jointly, since the allocation determines how much activation signal remains available for label-free recovery.
Qishi Zhan, Minxuan Hu, Liang He
Marquette University · Cornell University · Tongji University