Layer-wise Input Recoverability in Deep Neural Networks

Maxwell Spiridonov; Alireza Rahmansetayesh; Pierre Baldi

cs.LGOct 9, 2026

Layer-wise Input Recoverability in Deep Neural Networks

Authors: Maxwell Spiridonov, Alireza Rahmansetayesh, Pierre Baldi

Organizations: Department of Computer Science, University of California, Irvine

Abstract

The inner workings of deep neural networks are still not well understood. The information bottleneck framework, which characterizes deep networks by the mutual information between their hidden representations and the network's input, has been used to identify compression of input information with respect to an input image and output labels. We instead study how much of a layer's own input is recoverable from the layer's output in convolutional neural networks (CNNs). A convolutional layer applies its filters to one input patch at a time, so we evaluate recoverability on sampled input patches with a reconstruction-based probe and not on the entire feature map. The probe's size thus depends on channels and kernel size and not on feature map size, which lets our method scale to deep CNNs with wide feature maps. For each layer, we compare pretrained, random, and reconstruction-optimized encoders, using linear and linear-plus-residual-MLP decoders. We measure recoverability as $R^2$ between input patches and their reconstructions on a held-out set. Across nine ImageNet CNNs, recoverability generally decreases with depth but is more consistently predicted by the compression of the layer. Learning-attributed recoverability, the difference between pretrained and random encoders, is not uniform across architectures. We introduce single-property random controls, which transfer one pretrained property onto a random encoder, and find that each control changes recoverability with a consistent sign across models: copying pretrained singular vectors tends to raise recoverability, while copying the pretrained singular-value spectrum or activation sparsity lowers it.

Figures & tables

Figure 1: Method overview. A. Patches are sampled from each probed layer’s input feature map (Section 3.1 ). B. Each layer’s patches are scored under three encodings of the same architecture: 1) the network’s pretrained encoder, 2) a random encoder, and 3) an encoder trained jointly with the decoder. All decoders are trained to minimize reconstruction mean squared error (MSE).

Figure 2: Recoverability ( $R^{2}$ ) over depth for convolutional layers with kernel size $K>1$ across 9 CNN architectures, comparing pretrained, randomly-initialized, and end-to-end (e2e) trained encoders. Top row: VGG16, ResNet18, ResNet50. Middle row: GoogLeNet, MobileNetV2, SqueezeNet 1.1. Bottom row: AlexNet, DenseNet121, and EfficientNet-B0. Recoverability tends to decrease over depth for most models, with the exception of MobileNet and EfficientNet.

Figure 3: Pretrained recoverability ( $R^{2}_{pre}$ ) against effective compression $k_{98}/C_{out}$ (log scale) for each of the nine architectures. Each point is one layer, colored by relative depth (0 = first layer, 1.0 = last layer). The black line is a least-squares fit of $R^{2}_{pre}$ on $\log(k_{98}/C_{out})$ and the dotted vertical line marks $k_{98}=C_{out}$ , where a layer stops being a bottleneck. Recoverability is strongly correlated with effective compression in seven of the nine models ( $\rho=-0.61$ to $-0.91$ ) and moderately so in GoogLeNet ( $\rho=-0.55$ ); AlexNet has too few layers to analyze.

Figure 4: Effect of transferring one pretrained property onto a random encoder, averaged over layers. Rows are the nine ImageNet models followed by VGG16 and ResNet18 trained on CIFAR-10 and CIFAR-100. The left column is the net effect of training, mean $R^{2}_{pre}-R^{2}_{rand}$ over layers; the other columns are the effect of matching the pretrained singular vectors ( direcm ), singular-value spectrum ( specm ), activation sparsity ( sparsem ), or BatchNorm scale and shift ( bnm ), mean $R^{2}_{control}-R^{2}_{rand}$ over layers. Blue cells represent a positive mean difference, and red cells represent a negative mean difference. The fraction under each value is the number of layers with the same sign as the mean; bold marks a 95% bootstrap CI over layers that excludes zero. Each column except for pretrained has a mostly consistent direction of change from the random initialization across models, positive for direcm , and negative for specm , sparsem , and bnm .

Figure 5: Recoverability ( $R^{2}$ ) over depth for VGG16 (left) and ResNet18 (right) trained on CIFAR-10 and CIFAR-100, for pretrained (solid) and random (dashed) encoders. Error bars are the standard error over probe seeds (5 for CIFAR-10, 3 for CIFAR-100) and are smaller than the markers. Deep layer recoverability increases more for the CIFAR-10 trained variants than the CIFAR-100 trained variants.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Patches-per-image ablation for ResNet18 trained on CIFAR-10. Left: pretrained recoverability ( $R^{2}$ ). Right: learning-attributed recoverability ( $R^{2}_{pre}-R^{2}_{rand}$ ). One line per layer; circles keep 50,000/10,000 patches in total and reduce the number of images, squares fix 2,500/500 images and sample 64 or 256 distinct patches per image, and the open diamond is the main-text setting (40,000/8,000 images, one patch each). Error bars are standard errors over three patch seeds.

Figure 7: Patches-per-image ablation for ResNet18 trained on ImageNet, with a fixed total of 40,000/8,000 patches (one patch seed).

Figure 8: Decoder saturation and memorization against effective compression $k_{98}/C_{out}$ , one point per (architecture, layer, encoder). Left: saturation gap $|R^{2}_{16\times}-R^{2}_{4\times}|$ against the gate threshold of 0.01. Right: memorization-control $R^{2}$ at $4\times$ against the gate threshold of 0.1. Failures of the saturation gate lie at $k_{98}/C_{out}>1$ .

Figure 9: Held-out $R^{2}$ of the real decoder and of the shuffled-target memorization control against decoder width ( $C_{out}$ multiples), for layers with $k_{98}/C_{out}<1$ (left, $n=21$ ) and $k_{98}/C_{out}\geq 1$ (right, $n=39$ ). Lines are means and shading is $\pm 1$ standard deviation over cells; the dotted line is the memorization gate at 0.1.

Figure 10: Held-out recoverability (solid) and training recoverability (dashed) against decoder size for each (architecture, layer) in the capacity study, for pretrained, random, and e2e encoders.

Figure 11: Recoverability ( $R^{2}$ ) against exact remaining uncertainty $H(X\mid Z)/H(X)$ for $3\times 3$ binary image patches, for a linear decoder (left), a linear-plus-residual-MLP decoder (middle), and the Bayes-optimal decoder (right). Each point is one encoder; color gives the encoder type and marker the activation function.

Figure 12: Gaussian patch models with covariances from four ResNet18 layers. Top: exact mutual information between input patches and noisy encoder outputs. Bottom: recoverability ( $R^{2}$ ) of the optimal linear decoder, against noise level, for the layer’s pretrained filters, random filters, orthogonal filters, and spectrum-matched filters ( specm ).

model	RMS gap	orth	bnm	specm	sparsem	specsp	specbn	polar	gainxfer
VGG16	0.047	$-0.16$	–	$+0.36$	$-0.39$	$-1.48$	–	$-1.99$	$-0.04$
ResNet18	0.055	$-0.08$	$+0.31$	$+0.17$	$+0.15$	$-0.07$	$+0.13$	$-0.59$	–
ResNet50	0.035	$-0.31$	$+0.15$	$-0.49$	$-0.02$	$-1.20$	$-1.00$	$-2.59$	–
GoogLeNet	0.102	$-0.13$	$+0.31$	$+0.27$	$+0.31$	$+0.39$	$+0.42$	$-0.26$	–
MobileNetV2	0.034	$-0.04$	$-0.27$	$-0.10$	$-8.95$	$+0.26$	$+0.24$	$\mathbf{+0.77}$	–
SqueezeNet 1.1	0.141	$-0.04$	–	$+0.10$	$\mathbf{+0.79}$	–	–	$+0.69$	–

Appendix

Table 1: Share of the summed squared gap over layers removed by each control (rows: models). Bold marks the highest-scoring control that reaches 0.5 where the RMS gap is at least 0.02; polar keeps the pretrained singular vectors with a flat spectrum. Dashes mark controls that are undefined for that model.

Figure 13: Pretrained recoverability ( $R^{2}_{pre}$ ) over depth, colored by layer type: convolution where kernel size $K>1$ (black), $1\times 1$ channel reduce (red), $1\times 1$ channel expand (blue), and DenseNet’s transition layers (purple).

Figure 14: Learning-attributed recoverability, $R^{2}_{pre}-R^{2}_{rand}$ , over depth for all layers of each architecture, colored by layer type as in Figure 13 . Positive values mean the pretrained encoder recovers more of the input than a random encoder of the same shape; the horizontal line marks zero. Error bars are standard errors over probe seeds and all panels share one y-axis.

Figure 15: Learning-attributed recoverability and the control that accounts for it in SqueezeNet 1.1, over depth within each layer type. Black: $R^{2}_{pre}-R^{2}_{rand}$ . Color: the gap that remains after transferring one property, $R^{2}_{pre}-R^{2}_{control}$ , for the best-scoring control in each layer type ( direcm for $1\times 1$ reduce layers, sparsem for $K>1$ convolutions and $1\times 1$ expand layers); a control that reproduces the pretrained encoder lies on zero. Error bars are standard errors over probe seeds (5 for the random baseline, 3 for the controls). We see that copying pretrained singular vectors accounts for all of the learning-attributed gap in SqueezeNet $1\times 1$ reduce layers.

Explore similar work

Aug 22, 2026

Width-Independent Compressibility of Deep Neural Networks

It has long been known that well-trained neural networks can be compressed very strongly without affecting their performance, an important phenomenon that remains poorly understood. We prove a uniform compressibility theorem for deep multilayer perceptrons with analytic activations. For a deep, wide fixed teacher network, there exists a narrow (same depth) network that approximately represents the same function as the original. The reachable compressed width is strikingly independent of the original width, but is

O((\log(1/\varepsilon))^{d_{in}})

, where

\varepsilon

is the error budget and

d_{in}

is the effective input dimension. Our construction involves a novel derivative-matching technique which is aware of the low-dimensional input, and a layer-wise reweighting that preserves the input-output mapping.

Hong-Yi Wang, Mingze Wang, Liu Ziyin

Princeton University Princeton, NJ 08544 · Peking University Beijing, China 100871 · Massachusetts Institute of Technology Cambridge, MA 02139

Neural Network Compression Neural Network Approximation Theory Multilayer Perceptrons

Ask PDF

May 7, 2026

Efficient Techniques for Data Reconstruction, with Finite-Width Recovery Guarantees

Data reconstruction attacks on trained neural networks aim to recover the data on which the network has been trained and pose a significant threat to privacy, especially if the training dataset contains sensitive information. Here, we propose a unified optimization formulation of the data reconstruction problem based on initial and trained parameter values, incorporating state-of-the-art proposals. We show that in the random feature model, this formulation provably leads to training data reconstruction with high probability, provided the network width is sufficiently large; this unprecedented finite-width result uses PAC-style bounds. Furthermore, when the data lies in a low-dimensional subspace, we show that the network width requirement for successful reconstruction can be relaxed, with bounds depending on the subspace dimension rather than the ambient dimension. For general neural network models and unknown data orientations, we propose an efficient reconstruction algorithm that approximates the low-dimensional data subspace through the change in the first-layer weights during training and uses only the last-layer weights for reconstruction, thus reducing the search space dimension and the required network width for high-quality reconstructions. Our numerical experiments on synthetic datasets and CIFAR-10 confirm that our subspace-aware reconstruction approach outperforms standard full-space techniques.

Edward Tansley, Roy Makhlouf, Estelle Massart +1

Mathematical Institute, Woodstock Road, University of Oxford, Oxford, UK, OX2 6GG. · ICTEAM Institute, UCLouvain, Euler Building, Avenue Georges Lemaˆıtre, 4 - bte L4.05.01, Louvain-la-Neuve, B - 1348, Belgium.

Topic pending

Ask PDF

Jul 6, 2026

Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity

We develop a unified function space theory of deep fully connected neural networks. Functions in our spaces are defined recursively as

\ell^1

-bounded linear combinations of activated functions from preceding layers, with a dictionary of affine functions at the first layer. Unlike existing theories that are largely specialized to homogeneous activations such as the ReLU, our framework provides a meaningful notion of functional complexity for deep networks with a broad range of homogeneous and non-homogeneous activation functions commonly used in practice. This simple construction unites several seemingly disparate ideas from the literature, including norm-based complexity bounds and variational characterizations of depth, and facilitates novel analyses of what kinds of functions deep norm-constrained networks can represent. To this end, we prove a novel representer theorem for our spaces and establish novel function-space complexity bounds showing that the associated function classes remain qualitatively small at arbitrary depth. In the univariate ReLU case, we prove a "depth saturation" result: depth in this setting yields only a small constant rescaling of the function class, with no added functional diversity. As a consequence, we show that deep norm-controlled ReLU functions in any dimension cannot exhibit high frequencies along any direction. This finding reveals that some commonly cited expressivity benefits of depth disappear once network complexity is controlled by an appropriate function space norm, rather than parameter count or other representational costs that permit compounded rescaling across layers. Overall, our results illustrate how a function space perspective yields new structural insights into the relationship between depth and complexity.

Julia Nakhleh, Robert D. Nowak

Department of Computer Science · University of Wisconsin-Madison · Department of Electrical & Computer Engineering

ReLU Neural Networks Neural Network Approximation Theory Neural Network Expressivity

Ask PDF