Hidden Activations are not Enough I: Knowledge Matrices as Higher Representations

Marco Armenta

cs.LGSep 28, 2026

Hidden Activations are not Enough I: Knowledge Matrices as Higher Representations

Authors: Marco Armenta

Organizations: Institut quantique, Université de Sherbrooke

Abstract

We study the knowledge matrix of a trained feedforward network as a higher representation of its inputs. A network is a pair $(W,f)$ , a thin representation $W$ of its quiver and an activation $f$ ; its function factorizes through the space of quiver representations, each input $x$ inducing a representation, and the knowledge matrix $M(x)\in\mathbb{R}^{C\times(d+1)}$ is the contraction of that representation to one matrix whose rows sum exactly to the logits. At one trained network we ask what determines it, what it is invariant to, what it determines, and what its geometry measures. Under (LCS), a locally constant slope diagonal, as for ReLU, the matrix at a regular input is a function of the realized germ; its stabilizer among encodings regular there is exactly the germ stabilizer at inputs with no vanishing coordinate, neuron permutation a special case; and it recovers the germ, whereas hidden activations, gauge-covariant and germ-incomplete, are not enough. Under (LCS) it equals per-class gradient $\times$ input plus an exact aggregate bias attribution, grounding it in attribution theory and computing it by $C$ vector-Jacobian products instead of probing. The fixed shape gives an alignment-free per-sample distance between ResNet-152, DenseNet-121 and GoogLeNet; the row-sum identity gives an exact visible/invisible displacement decomposition whose unit-free coherence $A=(d_Ψ/d_M)^2$ puts adversarial germ motion at median $A\le 0.23$ , with an attack-family ordering concordant across six architectures (Kendall $W=0.921$ ; $0.97$ on the three networks at full scale). Two honest negatives: on AlexNet/CIFAR-10 penultimate features win 5 of 6 detectors and all 16 attacks, and a matrix-direction counterfactual fails 0/54.

Figures & tables

Panel of Figure 1 — Figure 1: The knowledge matrix encodes the germ of the network function. At almost every input, the network computes an affine map $x^{\prime}\mapsto Jx^{\prime}+c$ on the surrounding linear region — its germ (Definition 3.3 ). The knowledge matrix arranges this germ as one fixed-shape matrix $\mathrm{M}(x)=[\,J(x)\,\mathrm{diag}(x)\mid c(x)\,]$ : its first $d$ columns are per-class gradient $\times$ input contributions, its last column collects every bias term, and each row sums exactly to the corresponding logit. Because column $i$ is scaled by $x_{i}$ , the germ is recoverable from the matrix at coordinates where $x_{i}\neq 0$ (Remark E.6 ); at $x_{i}=0$ that column vanishes. Everything in this paper follows from this picture: invariance (the germ does not change under reparameterizations that preserve the function), completeness (the germ is recoverable from the matrix where the input is nonzero), and the distance geometry (the row-sum constraint splits any matrix change into a function-visible part and a function-invisible remainder).

Symbol	Meaning	Introduced
$Q$	network quiver: vertices are the neurons, arrows carry the weights	Section 3
$(W\!,f)$	the network: a thin representation $W$ of $Q$ (one weight per arrow) and the activation $f$	Section 3
$\Psi(W\!,f):\mathbb{R}^{d}\to\mathbb{R}^{C}$	network function realized by $(W,f)$ ; $d$ input coordinates, $C$ classes	Section 3
$\phi(W\!,f)(x)$	the thin representation of $Q$ induced by the input $x$ ; its contraction is $\mathrm{M}(W\!,f)(x)$	Section 3
$z^{(\ell)}_{q}(x)$ , $h^{(\ell)}_{q}(x)$	pre-activation and activation of hidden unit $q$ in layer $\ell$	Section 3
$D^{(\ell)}(x)$	slope diagonal of layer $\ell$ , $\mathrm{diag}\bigl(f(z_{q})/z_{q}\bigr)_{q}$	Eq. ( 1 )

Table 1: Notation of Sections 3 – 4 , with the place each symbol is introduced.

Measure	Invariance class	ResNet-152	DenseNet-121	GoogLeNet
debiased CKA	orth + iso-scale	$0.908\,[0.906,0.910]$	$0.902\,[0.900,0.904]$	$0.935\,[0.935,0.936]$
angular CKA ↓	orth + iso-scale	$0.432\,[0.428,0.436]$	$0.445\,[0.441,0.450]$	$0.361\,[0.359,0.363]$
Procrustes	orth	$776.6\,[773.8,779.4]$	$1471.4\,[1459.1,1484.2]$	$632.1\,[629.2,634.9]$
Bures	orth + iso-scale	$0.920\,[0.919,0.921]$	$0.925\,[0.924,0.927]$	$0.956\,[0.956,0.957]$
soft-matching	permutation	$21.93\,[21.85,22.01]$	$60.73\,[60.23,61.24]$	$32.22\,[32.09,32.34]$
RSA	rotation + monotone	$0.841\,[0.837,0.845]$	$0.908\,[0.906,0.910]$	$0.958\,[0.957,0.958]$

Table 2: No measure of the panel returns the exact invariance that the transformation guarantees: the distances and the bounded similarities alike register the drift, as their invariance classes predict. Representational-similarity panel between penultimate features

h_{W}(x)

and their neural-teleportation image

h_{\tilde{W}}(x)

, mean

[95\%

]

over

T=50

teleportations per architecture (

N=25{,}000

samples). Distances (Procrustes, soft-matching) grow with the teleportation drift; bounded similarities (CKA, Bures, dCor, RSA) report high agreement; square-root output JSD

\approx 0

confirms the transformation is function-preserving. Every magnitude is conditional on the change-of-basis distribution

\tau\sim U[0,2]

( cob_range

=1

); the analytic prediction is “not exactly

1

”, nothing more. GW omitted: the entropic solver returned a degenerate plan (

0.0

) on every cell (Section 6.2 ). Angular CKA is neither (marked

\downarrow

: smaller is more similar): it is

\arccos(\text{CKA})\in[0,\pi/2]

in radians, so small means similar — the

0.36

–

0.45

entries are the arccosines of the debiased-CKA row above (

\arccos 0.908=0.432

), not a report of low agreement. The knowledge matrix is not a column here: teleportation is an exact function-preserving isomorphism (Appendix C ), so the knowledge matrix does not move; the implementation check that realizes this is Appendix C.5 . What the intervals do and do not cover : the resampled unit is the

T=50

teleportation draws, with the

N=25{,}000

images held fixed (percentile bootstrap of the mean,

B=10{,}000

, seeded; Section 5 ). They are therefore intervals on the variability of the change-of-basis draw alone, which is why several are narrower than the printing precision; they say nothing about how a cell would move on a different image set or a different architecture. Cui random-network and Murphy shuffled-pair controls reported in the text.

Figure 2: On every bounded measure the teleportation pairs sit between the identity line and the controls, and on none of them at the identity: no measure of the panel absorbs a transformation that changes nothing about the function. The similarity panel of Table 2 as a picture: each panel is one measure, each column one architecture. Filled dots are the mean over

T=50

neural teleportations of the value between the penultimate features

h_{W}(x)

and their teleported image

h_{\tilde{W}}(x)

N=25{,}000

images, with the 5th–95th percentile range of the

50

draws as a bar (narrower than the marker on most panels). The dashed line is the value each measure returns for identical representations (

1

for the bounded similarities,

0

for angular CKA, output JSD and the two distances). Hollow squares are the Cui random-network control and hollow triangles the Murphy shuffled-pair control of Table 8 ; they are omitted for Procrustes and soft-matching, whose control values were computed on

n=2{,}048

inputs and are not comparable in magnitude to the panel.

attack	ResNet-152	DenseNet-121	GoogLeNet	ResNet-18	AlexNet	VGG
FGSM	0.0303	0.0606	0.05	0.0932	0.453	0.191
PGD	0.0998	0.216	0.203	0.338	1.72	1.25
CW	0.0239	0.0272	0.037	0.0547	0.168	0.0619
DeepFool	0.0164	0.01	0.0108	0.015	0.099	0.0255
APGD	0.113	0.192	0.228	0.229	1.28	0.837
Square	0.0146	0.0273	0.0183	0.0373	0.228	0.0834

Table 3: On the three deep ImageNet networks every attack family moves the germ at median coherence

A\leq 0.23

— less coherent than a single within-region pixel — with the iterative-PGD family consistently the most coherent. Median coherence

A=(d_{\Psi}/d_{M})^{2}

of adversarial knowledge-matrix motion (

d{+}1=150{,}529

;

\rho_{\mathrm{vis}}=A/(d{+}1)

);

A=1

is the single-pixel coherence line and

A\leq d

the within-region cap. On the two default-budget raters the PGD family crosses the one-pixel line (AlexNet: PGD

1.72

, APGD

1.28

; VGG: PGD

1.25

). Caveat (median interpolation): the per-pair median is exact only for odd valid-pair counts; only

9/36

cells are odd-count/exact, so even-count cells (the DenseNet-121 column in particular) are interpolation-approximate. Caveat (cell sizes): each cell holds

200

nominal pairs, of which

110

–

200

are valid; FGSM and PGD keep all

200

, while the CW, DeepFool, APGD and Square cells keep

149

–

161

(ResNet-152),

150

–

152

(DenseNet-121),

120

–

123

(GoogLeNet),

140

–

141

(ResNet-18),

110

(AlexNet) and

198

–

200

(VGG), so read those cells as low-

n

. Each entry is a median over the valid pairs of that one cell ; nothing here is pooled across attacks or across architectures. Accordingly the “

A\leq 0.23

” headline in Section 7 is the largest of the

18

cell medians of the three networks (GoogLeNet/APGD,

0.228

), not the median of a pooled population.

architecture	ordering by median $d_{M}/d_{\Psi}$ (desc.)	LOO Spearman
ResNet-152	Square $>$ DeepFool $>$ CW $>$ FGSM $>$ PGD $>$ APGD	0.771
DenseNet-121	DeepFool $>$ CW $>$ Square $>$ FGSM $>$ APGD $>$ PGD	0.943
GoogLeNet	DeepFool $>$ Square $>$ CW $>$ FGSM $>$ PGD $>$ APGD	0.928
ResNet-18	DeepFool $>$ Square $>$ CW $>$ FGSM $>$ APGD $>$ PGD	0.986
AlexNet	DeepFool $>$ CW $>$ Square $>$ FGSM $>$ APGD $>$ PGD	0.943
VGG	DeepFool $>$ CW $>$ Square $>$ FGSM $>$ APGD $>$ PGD	0.943

Table 4: Six architectures rank the attack families in the same order: Kendall

W=0.921

(6 raters

\times

6 attacks; permutation

p<10^{-6}

10^{6}

Monte-Carlo draws). The last column is the leave-one-architecture-out Spearman of each architecture’s ranking against the mean consensus rank of the other five (midrank ties). The six raters are the paper’s three networks (ResNet-152, DenseNet-121, GoogLeNet) plus three ordering-only cross-check raters (ResNet-18, AlexNet, VGG) that enter no coherence-magnitude statistic and no cross-architecture distance. The stable structure is family-level:

\{

DeepFool, CW, Square

\}>

FGSM

\gg\{

PGD, APGD

\}

d_{M}/d_{\Psi}

. The two gaps are of different sizes: the top family leads FGSM by

1.13

–

1.51\times

across the six raters, while FGSM leads

\max\{

PGD, APGD

\}

1.57

–

2.09\times

. What separates the families is the denominator. Ordering the same

36

cells by median logit displacement

d_{\Psi}

alone is more concordant than the ratio ordering (

W=0.975

against

0.921

), and on four of the six raters (DenseNet-121, ResNet-18, AlexNet, VGG) the ratio ordering is the exact reversal of the

d_{\Psi}

ordering (rank Spearman

-1.00

;

-0.94

on GoogLeNet,

-0.77

on ResNet-152). The families are separated by how far past the decision boundary an attack pushes — minimal-perturbation attacks stop at it, large-step attacks drive well beyond it — and not by any per-unit-displacement property of the germ motion. Attack budgets are those of Section 5 : the overrides listed there apply to ResNet-152, DenseNet-121, GoogLeNet and ResNet-18, whereas AlexNet and VGG ran the torchattacks 3.5.1 defaults throughout. Six raters do not settle what that asymmetry costs: the two default-budget raters (AlexNet, VGG) produce rank rows identical to DenseNet-121’s, which ran the overrides, while the most deviant rater is ResNet-152 (LOO

0.771

) — the one rater with a DeepFool budget of its own ( steps

=100

, against

200

on the other three override architectures). Budget-insensitivity and a budget-driven deviation are both consistent with

m=6

; we state the fact and do not adjudicate it. Restricting the concordance to the four raters that ran the override budgets (ResNet-152, DenseNet-121, GoogLeNet, ResNet-18) gives

W=0.92

(

p=3.8\times 10^{-6}

, exact permutation test over the

(6!)^{3}

null), the six-rater value to two decimals, so dropping the two default-budget raters leaves the concordance where it is. The restriction of this concordance to the three networks of Section 5 is

W=0.91

(

p=4.9\times 10^{-4}

, exact permutation test). The appendix panel (Table 10 ) recomputes the three networks’ ordering on the full-scale reduce, on a population-matched pair set and a different image set, and reproduces the family-level grouping on all three architectures (

W=0.97

Figure 3: Every cell sits below the single-pixel line

A=1

, and the iterative-PGD family is the most coherent on all three architectures. Coherence

A=(d_{\Psi}/d_{M})^{2}

of adversarial knowledge-matrix motion, per attack family and architecture, on the full-scale pair set (boxes: median and interquartile range; whiskers: 5th–95th percentiles;

n=449

–

1000

pairs per cell after the attack-success filter

d_{\Psi}\geq 1

d_{M}>0

). Colors group the attack families of the ordering: minimal-norm attacks (DeepFool, CW, Square), FGSM, and the iterative-PGD family (PGD, APGD). The dashed line is the single-pixel reference

A=1

of Theorem 4.3 . Hollow diamonds are the

200

-pair headline medians of Table 3 , computed on different images, with the library-default attack budgets and the division-guard convention; the two sets agree on the family ordering and differ most on GoogLeNet’s PGD family, whose

200

-pair cells are the smallest (

120

–

123

valid pairs) and whose full-scale generator uses different PGD and APGD budgets (Section 5 ).

Pair	KM Frob RMS	deb. CKA	ang. CKA	Bures	soft-match	RSA	out-JSD	dCor
RN–DN	$0.0169$	$0.442$	$1.113$	$0.488$	$123.98$	$0.419$	$0.342$	$0.661$
RN–GN	$0.0107$	$0.358$	$1.205$	$0.478$	$63.63$	$0.334$	$0.388$	$0.594$
DN–GN	$0.0180$	$0.539$	$1.002$	$0.595$	$109.11$	$0.524$	$0.354$	$0.751$

Table 5: The knowledge-matrix distance is available for all three architecture pairs with no alignment step, while the panel measures return population-level scores and two of them are unavailable. Alignment-free cross-architecture comparison (Study 3): KM Frobenius distance (RMS-per-coordinate, mean over

N=25{,}000

ImageNet-val samples) is computed directly between architectures and is a distance per sample , whose logit-visible component is exactly the logit displacement divided by

\sqrt{d{+}1}

(Theorem 4.1 ); the panel measures beside it return population-level similarity scores in their own metric spaces. GW omitted (the entropic solver returned a degenerate plan on all pairs; Section 6.2 ); cross-architecture Procrustes omitted (a pipeline limitation, see text). Cui/Murphy controls pending a controls-enabled reduce. Per sample, the knowledge-matrix distance has median

0.0146

(interquartile range

0.0113

–

0.0200

5

th–

95

th percentiles

0.0083

–

0.0328

) on RN–DN;

0.0096

(interquartile range

0.0074

–

0.0126

5

th–

95

th percentiles

0.0054

–

0.0195

) on RN–GN;

0.0162

(interquartile range

0.0126

–

0.0214

5

th–

95

th percentiles

0.0092

–

0.0327

) on DN–GN (

N=25{,}000

samples each; the mean exceeds the median on every pair). Read the two unnormalized columns with care. KM Frobenius RMS and soft-matching carry the units of the representations they compare, and the three architectures are not on a common scale: DenseNet-121’s centered penultimate Gram carries

3.5

–

3.8\times

the squared Frobenius energy of the other two (

2.91\times 10^{6}

1.10\times 10^{7}

3.18\times 10^{6}

for ResNet-152, DenseNet-121, GoogLeNet), so on both of those columns the two DenseNet-containing pairs are the two largest and the table partly orders the pairs by whether DenseNet takes part. Dividing soft-matching by

\sqrt{\|\tilde{X}\|_{F}^{2}+\|\tilde{Y}\|_{F}^{2}}

leaves the order intact (

0.0332

0.0289

0.0258

for RN–DN, DN–GN, RN–GN) but shrinks the spread: RN–GN rises from

51\%

78\%

of RN–DN. We give no confidence intervals: three pairs generated by three architectures leave no residual degrees of freedom, and an interval would decorate the confound rather than address it. Read the row as a demonstration that the comparison is computable without alignment, not as a ranking of architectural similarity.

Limitation	Consequence, and forward pointer
(L1) Both invariance arms are quiver isomorphisms; the cross-architecture claim has no transform-based evidence.	The permutation and teleportation checks confirm an identity the theory already guarantees (Lemma 3.5 ), whereas the non-automatic cross-architecture invariance of Theorem 3.6 is exercised by no same-architecture transform, and Study 3 compares networks computing different functions. The missing experiment — two genuinely different architectures realizing the same function on a neighborhood, by dead-unit insertion, neuron splitting or a width-changing re-encoding — is the highest-priority follow-up; the permutation check runs on ResNet-152 alone because a post-pool channel permutation does not compose through DenseNet’s concatenations or GoogLeNet’s Inception branches, a software limit that neural teleportation ( Armenta et al., 2024 ) , acting on a more general change-of-basis structure, does not share.
(L2) The cross-architecture study omits the Cui/Murphy controls.	Study 3 (Section 8 ) is reported without the random-network control of Cui et al. (2022) or the shuffled-pair control of Murphy et al. (2024) that accompany the same measures in Section 6.2 ; those controls were not run in the cross-architecture reduce. Its numbers establish availability — a per-sample alignment-free distance whose logit-visible component is the logit displacement divided by $\sqrt{d{+}1}$ (Theorem 4.1 ) — and not a control-calibrated quality ranking; running the two controls is mechanical follow-up.
(L3) Single-region LP-counterfactual fails on ImageNet.	Section 9 reports region_ok $=0/54$ on the box-constrained LP, which closes off the claim that knowledge matrices admit a clean adversarial-style counterfactual; per-nonlinearity constraints keeping $\delta$ in the source region, or a homotopy/continuation method across regions, are the natural extensions and are not pursued here. The per-LP records behind the count are not preserved in the supplementary material, so the count and the $\\|\delta\\|$ magnitudes cannot be recomputed from stored data; the surviving record is Figure 4 , with its embedded $\\|\delta\\|_{1}=128.0$ and $\\|\delta\\|_{\infty}=4.10$ .
(L4) Procrustes, Bures and soft-matching benchmarks deferred.	Section 6.2 runs the nine-measure panel with the random-network ( Cui et al., 2022 ) and shuffled-pair ( Murphy et al., 2024 ) controls, and Section 8 extends it across architectures. Deeper benchmarking of the shape-space metrics ( Williams et al., 2021 ; Harvey et al., 2024 ; Khosla & Williams, 2024 ) and of Gromov–Wasserstein ( Mémoli, 2011 ) — multi-architecture sweeps including transformer families, the full ReSi protocol ( Klabunde et al., 2025 ) , direct comparison to model stitching ( Bansal et al., 2021 ) — is deferred to future work.
(L5) CNN-only.	We cover the residual, dense and inception families (ResNet-152, DenseNet-121, GoogLeNet); the quiver-representation framework ( Armenta & Jodoin, 2021 ; Armenta et al., 2022 ) applies to attention-based architectures, but the knowledgematrix library does not yet support attention layers. Extension to transformers is library engineering rather than theory, and is left as future work.
(L6) The cross-architecture penultimate metric is confounded by feature scale.	Wherever penultimate distances are shown across widths ( $D=2048$ for ResNet-152, $D=1024$ for DenseNet-121 and GoogLeNet) we use the RMS-per-dimension form $\\|h(x)-h(x^{\prime})\\|_{2}/\sqrt{D}$ (Section 5 ), which is not normalized by $\\|h(x)\\|_{2}$ , so an absolute cross-architecture ranking built on it reflects feature scale and we read none. Within-architecture comparisons and the two implementation checks of Appendix C.3 — the permutation arm in absolute $\ell_{2}$ , the teleportation arm through the panel (Procrustes $632$ – $1471$ , soft-matching $22$ – $61$ ) — are unaffected.

Table 6: Eight limitations that the experiments of this paper do not resolve, in decreasing order of importance, each with its consequence and where it is taken up.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

(a) Permutation-noise floors (KM Frobenius; penultimate L2)
architecture	KM mean	KM median (pooled)	KM max	penult. mean
ResNet-152	0.116	0.005084	5.78	15.05
ResNet-18	0.0704	0.01061	0.865	29.32
AlexNet	1.54e-05	1.454e-05	3e-05	105.7
VGG	2.13e-05	2.050e-05	3.84e-05	57.7
(b) Adversarial signal-to-noise (mean signal / mean noise): KM $\mid$ penultimate

Appendix

Table 7: The knowledge matrix registers an attack

7\times 10^{2}

–

1.7\times 10^{7}

times more strongly than a neuron permutation, whereas the penultimate features register the permutation at least as strongly as the attack in

16

24

cells. Permutation invariance, signal-relative: penultimate adversarial signal is at most

3.05\times

its own permutation noise (below

1\times

in 16/24 cells); the KM signal sits

7\times 10^{2}

–

1.7\times 10^{7}

above its noise floor. Worst-case tail on ResNet-152 (min signal / max noise): 1.14

\times

(DeepFool); the ResNet-152 floor is the wide-face permutation subgroup, and its residual is the resolution of the pipeline (Appendix G ), not a motion of the matrix. In panel (a), “KM median (pooled)” is the median of the

250

pooled per-sample distances (

5

draws

\times

50

samples), while “KM mean” and “KM max” average and maximize the per-draw statistics. DenseNet-121/GoogLeNet have no permutation run by design (covered by teleportation); penultimate per-pair quantiles are unrecoverable from stored summaries. In panel (b) the bar in “

x\mid y

” is a separator, not a division:

x

is the knowledge matrix’s ratio and

y

the penultimate features’ own, each formed in its own units. Both cells are mean adversarial signal

\div

mean permutation noise , the two scales defined in Section 5 , so a value above

1

says the representation registers the attack more strongly than it registers a function-preserving relabeling of neurons. Worked example (ResNet-152/FGSM):

83.98/0.1157=726

and

7.600/15.05=0.51

	Murphy (shuffled pairs)			Cui (random network)
Measure	RN-152	DN-121	GN	RN-152	DN-121	GN
debiased CKA †	$2.4\!\times\!10^{-5}$	$2.6\!\times\!10^{-4}$	$5.4\!\times\!10^{-4}$	$0.010$	$0.058$	$0.052$
RSA †	$-0.0062$	$-0.0033$	$-0.0063$	$0.015$	$0.181$	$0.130$
dCor †	$0.000$	$0.019$	$0.024$	$0.127$	$0.296$	$0.255$
angular CKA (rad)	$1.5708$	$1.5705$	$1.5703$	$1.5607$	$1.5124$	$1.5184$
Bures	$0.278$	$0.264$	$0.294$	$0.129$	$0.286$	$0.383$

Appendix

Table 8: Three measures return to zero under the shuffled-pair control and five do not, Bures least of all. The controls in full, all eight measures, so that the three the automated gate checks (distance correlation, debiased CKA, RSA — marked † ) are not read as the whole control. Murphy permutes the sample alignment between

X

and

Y

; Cui replaces

Y

by an identically-initialized but untrained network. Both are computed on

n=2{,}048

inputs, not on the panel’s

N=25{,}000

, so the two unnormalized distances (Procrustes, soft-matching) are roughly

3.5\times

smaller here than a panel-sized draw would make them and are not comparable to Table 2 in magnitude. Both blocks are bare point estimates: no confidence intervals, and no permutation-null

p

-value is computed anywhere in this paper. Angular CKA is

\arccos(\mathrm{CKA})

, so its null value is

\pi/2=1.5708

, and

1.5703

–

1.5708

is the correct reading for “no agreement”. Bures does not go to zero under either control, which caps how much the cross-architecture Bures values of Table 5 (

0.478

–

0.595

) can be read to mean.

attack	ResNet-152	DenseNet-121	GoogLeNet	ResNet-18	AlexNet	VGG
FGSM	0.19	0.34	0.2	0.389	1.7	1.09
PGD	0.662	0.68	0.88	0.789	3.78	3.06
CW	0.379	0.319	0.151	0.307	0.957	0.729
DeepFool	0.297	0.237	0.132	0.197	1.66	—
APGD	0.597	0.774	1.19	0.791	4.52	—
Square	0.458	0.238	0.0858	0.187	0.737	—

Appendix

Table 9: Even the most coherent pair of each cell of the three networks stays near or below the single-pixel line, and three VGG cells are unbounded because some pair has

d_{M}=0

exactly. Maximum per-pair coherence per cell,

A_{\max}=\max_{i}\,(d_{\Psi}^{(i)}/d_{M}^{(i)})^{2}

, computed as the inverse square of the smallest stored per-pair ratio

d_{M}/d_{\Psi}

of the cell (’—’: cells in which some pair has

d_{M}=0

exactly — clean and adversarial input in the same linear region — so the smallest ratio is

0

and

A_{\max}

is unbounded; the entry is a display artifact of that division, not a cell removed by any filter: those three cells keep

198

of their

200

pairs, two fewer than VGG’s FGSM, PGD and CW cells, which keep all

200

). The associated floor-saturation —

\sqrt{d{+}1}

times the smallest per-pair ratio

d_{M}/d_{\Psi}

of the cell — ranges across cells from a low of

182.6

to roughly

1325

. Cell sizes as in Table 3 (

200

nominal pairs,

110

–

200

valid; the CW, DeepFool, APGD and Square cells are low-

n

Architecture	Ordering by median $d_{M}/d_{\Psi}$ (desc.)	$N$ /cell
ResNet-152	DeepFool $>$ Square $>$ CW $>$ FGSM $>$ PGD $>$ APGD	449–512
DenseNet-121	DeepFool $>$ CW $>$ Square $>$ FGSM $>$ PGD $>$ APGD	871–1000
GoogLeNet	DeepFool $>$ CW $>$ Square $>$ FGSM $>$ PGD $>$ APGD	770–1000
Kendall’s $W=0.97$ (3 architectures $\times$ 6 attacks)

Appendix

Table 10: Attack-family ordering cross-check on the three networks at full scale (appendix). Each architecture’s six attack families ranked by median

d_{M}/d_{\Psi}

over the attack-success-filtered pairs (

d_{\Psi}\geq 1

d_{M}>0

), descending (equivalently ascending coherence

A=(d_{\Psi}/d_{M})^{2}

), computed on the full-scale pair set (its own attack budgets, every departure from the torchattacks 3.5.1 defaults: PGD steps

=7

; CW steps

=100

; DeepFool steps

=100

; APGD steps

=50

, the DLR loss; Square n_queries

=20{,}000

; FGSM at the library defaults; Table 4 uses the

200

-pair set of Section 5 , which is drawn from different images). Cells are matched on a common image population — the intersection of the six cells’ stored pair indices,

n=512

on ResNet-152 and

n=1000

on the other two — because the ResNet-152 DeepFool run stopped at

512

1000

pairs, so an unmatched panel would compare medians across different images (matching moves ResNet-152 CW by

+12.9

%, Square by

+8.5

%, while the cells the filter leaves intact — FGSM and PGD — move by under

1

%). The ranking is unchanged under all three filtering conventions. Kendall

W=0.97

across the three architectures (

p=3.1\times 10^{-5}

, exact permutation test; the

m=3

null is supported on

77

lattice points spaced

0.0127

apart, so

W

is quoted to two decimals) cross-checks the six-architecture headline concordance (

W=0.921

, Table 4 ), and the point-estimate ordering reproduces its family-level grouping

\{

DeepFool, CW, Square

\}>

FGSM

>\{

PGD, APGD

\}

on all three architectures (gaps

1.09

–

1.43\times

and

1.28

–

1.87\times

respectively). Bootstrap resolution (BCa and percentile

95\%

CIs,

B=20{,}000

, seeded, over pairs): ResNet-152 reads DeepFool

\approx

Square

\approx

\approx

FGSM

>

PGD

>

APGD, while on DenseNet-121 and GoogLeNet every adjacent gap is resolved. Those intervals are printed rather than only asserted: all

18

cell intervals in Table 11 and all

15

adjacent-gap intervals, in both constructions, in Table 12 , which also states the uncorrected multiplicity of the gap tests. Computed on the metric-invariant ranks, so no raw/RMS unit choice enters.

Architecture	Attack	$n$	median $d_{M}/d_{\Psi}$	BCa $95\%$ CI
ResNet-152	DeepFool	449	$7.875$	$[6.744,\,8.835]$
	Square	456	$7.610$	$[6.980,\,8.309]$
	CW	462	$7.290$	$[6.523,\,8.295]$
	FGSM	512	$6.707$	$[6.108,\,7.114]$
	PGD	512	$3.581$	$[3.413,\,3.833]$
	APGD	466	$3.011$	$[2.833,\,3.255]$

Appendix

Table 11: Per-cell medians behind Table 10 , with seeded bias-corrected and accelerated (BCa)

95\%

bootstrap intervals (

B=20{,}000

; the resampled unit is the adversarial pair inside that cell, so the interval covers sampling variability over pairs and nothing else).

n

is the number of pairs the attack-success filter keeps inside the matched population. Ratios are raw-unit

d_{M}/d_{\Psi}

; the RMS rescaling is a per-architecture constant and cancels from every rank and every ratio of two cells in the same architecture.

Architecture	Adjacent pair	$\Delta$ median	independent $95\%$ CI	paired $95\%$ CI
ResNet-152	DeepFool $>$ Square †	$0.265$	$[-1.017,\,1.441]$	$[-0.583,\,1.000]$
	Square $>$ CW †	$0.320$	$[-0.841,\,1.317]$	$[-0.427,\,0.941]$
	CW $>$ FGSM †	$0.583$	$[-0.254,\,1.755]$	$[-0.080,\,1.542]$
	FGSM $>$ PGD	$3.127$	$[2.481,\,3.606]$	$[2.626,\,3.488]$
	PGD $>$ APGD	$0.570$	$[0.281,\,0.883]$	$[0.396,\,0.758]$
DenseNet-121	DeepFool $>$ CW	$1.588$	$[1.116,\,2.282]$	$[1.348,\,2.030]$

Appendix

Table 12: Adjacent-rank gaps of Table 10 : the difference of the two cell medians, with

95\%

percentile bootstrap intervals computed twice (

B=20{,}000

, seeded) — resampling the two cells independently , and resampling the matched pair indices jointly . A gap counts as resolved only when both intervals exclude zero, which is the conservative verdict. Multiplicity : the

15

adjacent-gap verdicts (

3

architectures

\times

5 gaps) carry no multiple-comparison correction. Under a global null of no ordering, at a nominal

5\%

per test the chance that at least one gap is resolved by accident is about

54\%

; requiring both the independent and the paired interval to exclude zero makes each individual verdict conservative but does not control the family-wise rate. The claim this panel supports is the family-level grouping, which is one statement, not the resolution of any single adjacent gap.

Limitation	Consequence, and forward pointer
(L9) The perturbation budget $\varepsilon$ is never varied.	FGSM, PGD, APGD and Square all run at the torchattacks default $\varepsilon=8/255$ in $\ell_{\infty}$ on every rater and in both pair sets, CW and DeepFool are not $\varepsilon$ -budgeted, and the only per-architecture overrides are to step counts, the APGD loss and the Square query budget (Appendix G ). Nothing here establishes that the coherence magnitudes of Section 7 , or the attack-family ordering Kendall’s $W$ summarizes, survive a change of $\varepsilon$ — a larger budget crosses more region walls, which is exactly what $A$ measures — so an $\varepsilon$ -sweep is the most informative robustness check the design omits.
(L10) The cross-architecture panel is computed on raw unequal dimensions, and no measure in it is dimension-neutral.	The panel of Section 8 is computed on the $2048$ - and $1024$ -dimensional features with no projection and no PCA; the measures are well defined (normalized Bures reads the $n\times n$ kernel, so zero-column padding leaves it unchanged, invariance ratio $1.000000$ ), but on a matched-signal probe the unequal-dimension pair scores higher than the equal-dimension one by $+0.02$ to $+0.04$ across debiased CKA, Bures, RSA and distance correlation at the widths used here ( scripts/verify_dimension_bias_probe.py ; the magnitude is probe-dependent, the sign is not). The bias inflates the two pairs that finish behind (ResNet-152/DenseNet-121, ResNet-152/GoogLeNet), so the reported ordering is conservative with respect to it, but it is uncorrected and no cross-dimensional similarity should be compared to an equal-dimensional one at the third decimal place.
(L11) The shuffled-pair control is reported only where its null is $\approx 0$ .	The Murphy control ( Murphy et al., 2024 ) was computed for every measure of the within-architecture panel, but the three with a null at zero — debiased CKA, RSA, distance correlation — are the ones the text foregrounds and the only ones the pipeline’s automated gate checks. Bures’s shuffled null is $0.264$ – $0.294$ while the cross-architecture Bures values of Section 8 are $0.478$ – $0.595$ — a real margin over the null but much smaller than the raw value suggests; the null is a fidelity between two positive semidefinite kernels and depends on $n$ as well as on the spectrum, and it was computed at $n=2{,}048$ against panel values at $N=25{,}000$ , so the $1.6$ – $2.3\times$ ratio is indicative only, and the same caution applies to the other bounded measures whose nulls we do not quote.
(L12) The coherence medians carry no intervals, and the appendix gap tests are uncorrected.	The per-cell medians of Table 3 (the “median $A\leq 0.23$ ” headline) are point estimates, the reduce behind them storing aggregates only, and only the appendix ordering panel, whose per-pair ratios are stored, carries bootstrap intervals; that panel’s $15$ adjacent-gap tests ( $3$ architectures $\times$ $5$ gaps, Table 12 ) carry no family-wise correction, so under a global null the family-wise error rate at $\alpha=0.05$ approaches $1-0.95^{15}\approx 54\%$ . We disclose both at the point of use and adjust neither, since requiring both the independent and the paired interval to exclude zero already makes each verdict conservative in an unquantified direction, and stacking a correction on that would give a number we could not interpret.

Appendix

Table 13: Further limitations (L9–L12): caveats of measurement convention that apply across several studies, with the consequence of each and where it is taken up.

					median $A$ by $H$ -tertile
attack	$n$	$\rho_{S}(H,d_{\Psi})$	$\rho_{S}(H,A)$	$\rho_{S}(H,A\mid d_{\Psi})$	low	mid	high
APGD	141	-0.30	-0.45	-0.34	1.46	1.03	0.835
DeepFool	134	+0.72	+0.44	-0.30	0.0222	0.0851	0.101
Square	141	+0.68	+0.13	-0.34	0.0943	0.116	0.118

Appendix

Table 14: On all three attacks the size-controlled partial rank correlation of

H

with

A

is negative, the sign the crossing conjecture predicts. This is the first direct test of the smooth+crossing mechanism (per-pair VGG debug data). The size confound is real on two of the three attacks and absent on the third, so we report the diagnostic column

\rho_{S}(H,d_{\Psi})

rather than assert it. On DeepFool and Square, pairs that cross more walls are also pairs whose logits move further (

\rho_{S}(H,d_{\Psi})=+0.72

and

+0.68

), and the raw

\rho_{S}(H,A)

inherits that sign (

+0.44

+0.13

) — the opposite of what the mechanism predicts; controlling for perturbation size via

d_{\Psi}

turns both negative. On APGD there is no such confound:

H

and

d_{\Psi}

are negatively associated (

-0.30

), the raw correlation already carries the predicted sign (

-0.45

), and the control moves it toward zero (

-0.45\to-0.34

). What the three attacks share is the size-controlled column: the partial rank correlation is negative in every case (

-0.30

-0.34

) — crossings reduce coherence, the sign Conjecture H.1 predicts (crossing dyads carry knowledge-matrix mass with zero logit displacement, the latter by Theorem 4.4 ).

H

lower-bounds the segment crossing count. Pilot caveats: single architecture;

d_{\Psi}

is a proxy for

\|\delta\|

(the exact control lives in pairs.pth ); three attacks on one network is not a corrected multiple test and no

p

-value is claimed here; the three-network,

\|\delta\|

-controlled replication is registered as follow-up work, and this pilot fixes its analysis plan.

Detector	Penultimate	All-layer	Knowledge matrix
Mahalanobis	0.917	0.208	0.638
$k$ -NN	0.904	0.165	0.798
KDE	0.909	0.289	0.788
GMM	0.847	0.176	0.374
one-class SVM	0.512	0.356	0.550
Isolation Forest	0.853	0.175	0.518

Appendix

Table 15: Penultimate features win five of the six detectors on AlexNet/CIFAR-10, and the external baseline beats all three representations. The detector bake-off in full, with its true scope: mean AUROC over

16

torchattacks attack families, AlexNet / CIFAR-10 — one architecture and one dataset, not the three ImageNet networks the rest of this paper uses. The three representations are the penultimate features, the concatenation of all hidden-layer activations, and the knowledge matrix; the third arm is the all-layer concatenation, not the logits . Bold marks the best representation per detector. Penultimate features win

5

6

; the exception, one-class SVM, puts the knowledge matrix at

0.550

against

0.512

, both essentially at the

0.5

chance line on the one detector that works for nothing, so we do not describe it as a knowledge-matrix win. The Lee et al. row is an external baseline, not one of the three arms: a multi-layer Mahalanobis detector with per-attack logistic regression, which beats all three.

Figure 4: The LP perturbation leaves the linear region it was solved in, and the source class still wins at $x+\delta$ . One worked instance of the single-region LP-counterfactual structural failure, source class $207$ (golden retriever) $\to$ target class $282$ (tiger cat). Left: original image $x$ . Center: the would-be counterfactual $x+\delta$ in ImageNet-normalized space, at $\|\delta\|_{1}=128.0$ , $\|\delta\|_{\infty}=4.10$ , $\|\delta\|_{2}=18.7$ . Right: at the actual (un-clamped) $x+\delta$ the source class still wins by a large margin, $14.77$ against $2.27$ ; region_ok = False , i.e. the region the LP solved against does not contain $x+\delta$ . Every one of the $54$ source/target pairs across the three architectures fails identically ( region_ok $=0/54$ , Limitation L3); this panel’s $\|\delta\|_{\infty}$ sits at the per-channel saturation cap $1/\sigma_{c}\approx 4.4$ . Provenance: the figure is the surviving record of this LP — the per-LP $\delta$ and region_ok dumps behind the $0/54$ count are not preserved in the supplementary material, and the run’s architecture is not recorded in the figure, so we do not attribute the panel to a specific network.

Explore similar work

Jun 8, 2026cs.LG

Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics

We study feed-forward ReLU networks with fixed readout and quadratic loss, and rewrite gradient descent as a collective dynamics of activation fields and conjugate fields on the training set. Working to first order in the learning rate inside a fixed activation chamber, we derive explicitly the one-, two- and three-hidden-layer cases, and then give the arbitrary-depth recursion. For one hidden layer the activation dynamics closes directly and the residual update is governed by the product of an input Gram matrix and a co-activation/backpropagation Gram matrix. For two hidden layers a conjugate field is required, but no nontrivial pullback Gram metric has yet appeared. For three hidden layers the first weight-induced pullback Gram metric enters the conjugate-field dynamics. At arbitrary depth, activation variations propagate forward through a recursive response operator

\cU_\ell^{αβ}

, while conjugate-field variations propagate backward through an effective transport operator

\cM_\ell^{αβ}

. Their contractions reconstruct a layerwise residual kernel

K_{αβ}^{(L)}=\sum_{\ell=1}^{L}Q_{αβ}^{(\ell-1)}S_{αβ}^{(\ell)}.

The resulting description exposes a duality between push-forward and pullback transport across every layer cut, and identifies the first Gram metrics as the lowest nontrivial terms in a broader hierarchy of activation-conditioned transport operators. We deliberately stop at the level of collective fields, conjugate fields, residual kernels and cut-wise transport metrics, leaving the later tensorial geometric formulation outside the scope of this paper.

Claudio Nordio

Rectified Linear Unit Networks Learning Dynamics Layer-Wise Scalar Field Riemannian Manifolds Hierarchical

Ask PDF

May 5, 2026cs.LG

Most ReLU Networks Admit Identifiable Parameters

We study the realization map of deep ReLU networks, focusing on when a function determines its parameters up to scaling and permutation. To analyze hidden redundancies beyond these standard symmetries, we introduce a framework based on weighted polyhedral complexes. Our main result shows that for every architecture whose input and hidden layers have width at least two, there exists an open set of identifiable parameters. This implies that the functional dimension of every such architecture is exactly the number of parameters minus the number of hidden neurons. We further show that minimal functional representations can still have non-trivial parameter redundancies. Finally, we establish a generic depth hierarchy, whereby for an open set of parameters the realized function cannot be represented generically by any shallower network.

Moritz Grillo, Guido Montúfar

Max Planck Institute for Mathematics in the Sciences, Leipzig · Departments of Mathematics and Statistics & Data Science, UCLA

Rectified Linear Unit Networks

Ask PDF

Sep 22, 2026math.RT

Graded Representation Theory of Equivariant Neural Networks

Nonlinear activations can create equivariant interactions between irreducible representations that linear maps cannot. We use the Gaussian degree decomposition to extend ordinary polynomial degree to such nonlinear maps, and prove that for a fixed coordinatewise equivariant layer each degree factors into a polynomial determined by the linear maps and a scalar determined by the activation. This separates three distinct obstructions, coming from symmetry, coordinates, and activation.

Mani Shayestehfar

Equivariant Neural Networks Equivariance Activation Functions Representation Space Neural Network Theory

Ask PDF