cs.CVSep 16, 2026

Unified Response Geometry for Structured Pruning

Authors: Kaixiang Shu

Organizations: Independent Researcher

Abstract

Structured pruning is commonly formulated as ranking individual channels, although channel responses can be complementary or cancel through downstream mixing. Motivated by these response interactions, we formulate pruning as the selection of a subset with large joint response capacity, followed by a separate functional realization step. Our unified response geometry maps each candidate set to M(D,R)=D1/2RD1/2M(D,R)=D^{1/2}RD^{1/2} and uses its determinant together with Schur-greedy residuals to select non-redundant coordinates. The same construction yields two information-conditioned instances: an unlabeled instance based on activation covariance, and a task-conditioned instance that combines activation and gradient variance for response scale with gradient correlation for complementarity. To convert the selected subset into an executable network, we fold predictable removed responses into successor weights through ridge compensation and recalibrate batch-normalization statistics, without fine-tuning the network. On ImageNet ResNet-50, the unlabeled instance reaches 65.4%65.4\% and 53.9%53.9\% Top-1 accuracy at 30% and 40% deletion, versus 59.8%59.8\% and 43.1%43.1\% for strength-only selection; the task-conditioned instance reaches 67.7%67.7\% and 56.3%56.3\% under the same protocol. A six-family screen shows architecture-dependent behavior, with positive relative contrasts in several convolutional and expansion-layer settings and clear boundary cases in windowed attention. These results support response geometry as a conditional principle for structured pruning, with its benefit determined jointly by the observed response and the architecture in which that response is realized.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. WUSPruner: Second-Moment-Optimal Transforms for Structured Pruning

    Oct 11, 2026Shengkun Tang, Dan Alistarh, Zhiqiang ShenLLM PruningStructured Pruning

  2. Artificial Structure Function Search: Preserving Artificial Functional Connectivity for Structured Pruning

    Sep 21, 2026Mindula Illeperuma, Rafael Pina, Charuka Herath +2Model CompressionFunctional Connectivity

  3. Optimal Pruning for Neural Architectures using Fisher Information Distances

    Sep 14, 2026David S. Berman, Yen-Yu Fu, Edward Hirst +1Neural Network PruningNeural Network Compression