Do ResNets Route? Sparse Interaction Experts in Residual Networks

Liang Yan; Siying Chen; Kaijie Chen; Bo Li; Jinghao Zhang; Mu Miao

Symbol	Meaning
Residual network, masks, and readouts
$x,\ N,\ C$	Input image; number of images; number of classes.
$L,\ [L];\ l,i,j\in[L]$	Number and index set of residual blocks; block indices.
$F_{l},\ s_{l},\ \phi_{l}$	Residual branch, shortcut, and activation of block $l$ .
$x_{l}(m)$	Hidden state at block $l$ under mask $m$ .
$m\in\{0,1\}^{L}$	Residual-branch mask; $m_{l}=0$ disables branch $l$ only.

Appendix

Table 2: Notation. Superscript

r\in\{v,h\}

marks the scalar (predicted-class) or vector (full-logit) readout; per-input statistics are averaged over images. Masks are analytical interventions, not learned routing decisions. Symbols used within a single appendix are defined there.

Model	Reconstruction	Norm ratio $(\rightarrow 1)$	Rel. error $(\downarrow)$	Scaled error $(\downarrow)$	Cosine $(\uparrow)$	Top-1 agree $(\uparrow)$
ResNet-18	Raw sum	$79.366$	$79.150$	$0.9606$	$0.2217$	$0.0064$
	Mean output	$0.3100$	$0.9771$	$0.9606$	$0.2217$	$0.0064$
	Centered sum	$77.412$	$77.190$	$0.9595$	$0.2288$	$0.0062$
	Möbius	$1.0000$	$8.506{\times}10^{-7}$	$8.550{\times}10^{-7}$	$1.0000$	$1.0000$
ResNet-34	Raw sum	$18{,}852.3$	$18{,}851.9$	$0.8890$	$0.4400$	$0.0250$
	Mean output	$0.2877$	$0.9104$	$0.8890$	$0.4400$	$0.0250$

Appendix

Table 3: Naive aggregation versus exact Möbius reconstruction. Entries are per-image means over fixed ImageNet-1K test subsets. ResNet-18 uses

N=10{,}000

images and

256

masks per image; ResNet-34 uses

N=1{,}000

images and

65{,}536

masks per image. Top-1 agreement is measured against the corresponding full-model prediction.

Figure 6: Naive masked-response aggregation fails, whereas Möbius inversion reconstructs the full logits. The top and bottom rows show ResNet-18 and ResNet-34, respectively. Columns report norm ratio, relative error, optimally scaled error, cosine similarity, and top-1 agreement with the full model. Bars and error bars denote per-image means and standard errors over the fixed ImageNet-1K test subsets.

Figure 7: Residual interaction-order spectra at the natural operating point. Results are shown for ResNet-18 (top;

N=10{,}000

,

L=8

) and ResNet-34 (bottom;

N=1{,}000

,

L=16

). From left to right, the columns show the normalized squared coefficient mass

\widetilde{E}_{k}

, cumulative low-order mass

C_{\leq k}

, effective interaction order

\kappa

, and mean per-interaction mass

M_{k}

. Scalar readouts use the class predicted by the full model, while vector readouts use all logits. Both models peak above

L/2

, and the large high-order

M_{k}

values show that this pattern is not explained solely by the number of subsets at each order.

Model	Readout	Peak $k$ ( $\widetilde{E}_{k}$ )	$C_{\leq 3}$	$C_{\leq L/2}$	$\kappa$	$M_{L}/M_{1}$
ResNet-18	Scalar	$5\;(0.2721)$	$0.1417$	$0.3545$	$5.001$	$32.21$
	Vector	$5\;(0.2951)$	$0.1356$	$0.3675$	$4.929$	$22.31$
ResNet-34	Scalar	$10\;(0.1951)$	$0.001153$	$0.2421$	$9.887$	$341.4$
	Vector	$10\;(0.1956)$	$0.001116$	$0.2419$	$9.884$	$296.4$

Appendix

Table 4: Summary of the interaction-order spectra.

C_{\leq L/2}

includes the midpoint order, while

M_{L}/M_{1}

compares the dataset-mean per-interaction masses at the highest and first orders. Values are rounded to approximately four significant digits.

$k$	Scalar	Vector	Scalar	Vector
	ResNet-18		ResNet-34
1	$0.008619$	$0.007611$	$0.000038$	$0.000036$
2	$0.032546$	$0.027096$	$0.000212$	$0.000196$
3	$0.100549$	$0.100901$	$0.000903$	$0.000884$
4	$0.212804$	$0.231893$	$0.003625$	$0.003595$
5	$\mathbf{0.272092}$	$\mathbf{0.295098}$	$0.012134$	$0.012054$

Appendix

Table 5: Complete normalized interaction-order spectra. Entries report the dataset-mean normalized squared coefficient mass

\widetilde{E}_{k}

. Six decimal places are retained so that the small low-order values of ResNet-34 are not rounded to zero.

Figure 8: Inference-time residual-scaling sweep. Results are shown for pretrained ResNet-18 (

N=10{,}000

) and ResNet-34 (

N=1{,}000

), without retraining. Solid and dashed lines denote ResNet-18 and ResNet-34, respectively. Pink and blue denote scalar and vector readouts. (a) Effective interaction order

\kappa

. (b) Cumulative low-order squared coefficient mass

C_{\leq 3}

. (c) Top-1 agreement between the scaled full-mask prediction and the corresponding prediction at

\lambda=1

. Decreasing

\lambda

shifts both models toward lower-order interactions, but large shifts coincide with substantial changes in prediction.

$\lambda$	$\kappa^{v}$	$\kappa^{h}$	$C_{\leq 3}^{v}$	$C_{\leq 3}^{h}$	Top-1 agreement
1.00	5.0009	4.9295	0.1417	0.1356	1.0000
0.95	4.9756	4.9002	0.1491	0.1447	0.9375
0.90	4.9448	4.8706	0.1573	0.1541	0.8759
0.85	4.9086	4.8389	0.1671	0.1640	0.8093
0.80	4.8635	4.8022	0.1786	0.1751	0.7314
0.75	4.8042	4.7572	0.1933	0.1882	0.6365

Appendix

Table 6: Residual-scaling sweep for ResNet-18. Entries are dataset means over

N=10{,}000

images and are rounded to four decimal places. Agreement compares the full-mask prediction at each

\lambda

with that at

\lambda=1

.

$\lambda$	$\kappa^{v}$	$\kappa^{h}$	$C_{\leq 3}^{v}$	$C_{\leq 3}^{h}$	Top-1 agreement
1.00	9.8872	9.8835	0.0012	0.0011	1.0000
0.95	9.8462	9.8390	0.0014	0.0013	0.9200
0.90	9.8027	9.7938	0.0017	0.0016	0.8780
0.85	9.7616	9.7474	0.0020	0.0020	0.8140
0.80	9.7013	9.6957	0.0025	0.0024	0.7430
0.75	9.6391	9.6373	0.0032	0.0030	0.6630

Appendix

Table 7: Residual-scaling sweep for ResNet-34. Entries are dataset means over

N=1{,}000

images and are rounded to four decimal places. Agreement compares the full-mask prediction at each

\lambda

with that at

\lambda=1

.

Figure 9: Per-input magnitude Top-

K

reconstruction for ResNet-18. (a) Retained squared coefficient mass for scalar and vector rankings. (b–d) Vector relative error, cosine similarity, and agreement with the all-open model under magnitude, random, order-matched, and low-order selection. (e) Predicted-class margin ratio. (f) Order composition of the magnitude Top-

K

sets; color denotes the selected fraction within each order and text denotes enrichment over combinatorial prevalence. (g,h) Mean per-image first-hit budgets and retained fractions for the stated error and mass thresholds.

Figure 10: Per-input magnitude Top-

K

interaction reconstruction for ResNet-34. Results are averaged over

N=1{,}000

unlabeled ImageNet-1K test images. The interaction dictionary contains

M=2^{16}-1=65{,}535

non-empty terms; the shortcut-only baseline is always included and is not counted in

K

. (a) Squared coefficient mass captured by independently ranked scalar predicted-class and vector-logit Top-

K

sets. (b–d) Relative vector-logit error, cosine similarity, and agreement with the all-open model prediction under magnitude Top-

K

, random-

K

, order-matched random, and low-order selection. Agreement is measured against the full-model prediction rather than the ground-truth label. (e) Ratio of the reconstructed predicted-class margin to the full-model margin. (f) Order composition of the vector magnitude Top-

K

sets across retained fractions; color denotes the selected fraction within each interaction order, and cell text gives its enrichment over combinatorial prevalence. (g,h) Effective interaction counts and their fractions of the full dictionary for vector-error and coefficient-mass thresholds; markers summarize the mean and median distributions across inputs. Coefficient mass is concentrated, but faithful reconstruction is much less sparse: retaining

50\%

of the terms captures

97.90\%

of the vector mass, yet yields relative error

13.87

and only

0.60\%

prediction agreement.

ResNet-18 ( $N=10{,}000$ , $M=255$ )
Retained	$K$	$\Gamma_{K}^{v}$	$\mathrm{Err}_{K}^{v}$	$\Gamma_{K}^{h}$	$\mathrm{Err}_{K}^{h}$	$\mathrm{Cos}_{K}$	Agree
1%	3	0.2445	0.9934	0.1450	4.0016	0.0019	0.0019
5%	13	0.6140	0.8625	0.4345	4.0782	0.1214	0.0063
10%	26	0.8161	0.6754	0.6548	3.6881	0.1671	0.0339
25%	64	0.9751	0.2644	0.9364	1.7686	0.4896	0.3048
50%	128	0.9988	0.0575	0.9971	0.2850	0.9554	0.8766

Appendix

Table 8: Magnitude Top-

K

concentration and reconstruction. Entries are dataset means; uncertainty terms are omitted for readability. Scalar and vector supports are ranked separately. Top-1 agreement is measured against the corresponding all-open model prediction.

Model	Retained	$K$	Relative error			Top-1 agreement
			Magnitude	Random	Order-matched	Magnitude	Random	Order-matched
ResNet-18	25%	64	1.7686	7.9048	7.3574	0.3048	0.0028	0.0044
	50%	128	0.2850	9.0794	7.7646	0.8766	0.0042	0.0068
ResNet-34	25%	16,384	35.5789	123.0253	117.9390	0.0020	0.0018	0.0006
	50%	32,768	13.8741	141.9837	111.8046	0.0060	0.0016	0.0024

Appendix

Table 9: Representative vector-readout baseline comparisons. Magnitude selection substantially improves numerical reconstruction over random and order-matched selection, but this does not necessarily translate into prediction agreement. Values are dataset means.

Model	Criterion	Mean $K$	$K/M$
ResNet-18	$\mathrm{Err}_{K}^{h}\leq 0.10$	150.60	59.059%
	$\mathrm{Err}_{K}^{h}\leq 0.05$	186.64	73.191%
	$\Gamma_{K}^{h}\geq 0.90$	55.77	21.870%
	$\Gamma_{K}^{h}\geq 0.95$	68.59	26.898%
	$\Gamma_{K}^{h}\geq 0.99$	99.85	39.155%
ResNet-34	$\mathrm{Err}_{K}^{h}\leq 0.10$	65,203.74	99.495%

Appendix

Table 10: Vector effective budgets. Mean per-image first-hit

K

and its fraction of the

M

non-empty interactions. Error thresholds use the first crossing of a potentially non-monotone curve; mass thresholds are monotone.

Figure 11: Input dependence and class structure of ResNet-18 interaction sets. Results use

N=10{,}000

ImageNet-1K validation images and

M=255

non-empty interactions. (a) Vector-set Jaccard similarity for same-class and different-class pairs, a uniform random-

K

baseline, and per-image overlap with the global vector set. (b) Same-class minus different-class gaps for vector binary, vector weighted, and decision-conditioned scalar signatures, together with the vector label-shuffle control. (c) Same-class/different-class overlap lift for vector and scalar signatures. (d) Leave-one-out 1-nearest-neighbor class accuracy at

K=32

, compared with the empirical

0.10\%

chance level. (e) Similarity of random pairs drawn within the high- and low-confidence terciles. (f) Distribution of per-image overlap with the global vector set at

K=32

(

12.55\%

retention). (g) Same-class and different-class pairwise Jaccard distributions at

K=32

. The vector sets share a strong global core but retain a small class-dependent component, while the decision-conditioned scalar sets show substantially stronger class alignment.

Figure 12: Input dependence and class structure of ResNet-34 interaction sets. Results use the same

N=10{,}000

class-balanced ImageNet-1K validation images and

M=65{,}535

non-empty interactions. Panels (a), (c), and (e) include retained fractions from

0.05\%

to

25\%

; the fraction-matched comparison in panel (b) begins at

1\%

. Panel (f) reports per-image overlap with the global vector set at

K=8{,}520

(

13.0\%

retention), matching the retained fraction used for ResNet-18 rather than its absolute

K

. Panel (g) shows the same-class and different-class Jaccard distributions at the fixed

K=32

used by the nearest-neighbor diagnostic. The positive class gaps replicate the ResNet-18 trend, but high global overlap again indicates class-structured variation around a shared interaction core rather than disjoint routes.

ResNet-18 ( $M=255$ )
Retained	$K$	$J_{\mathrm{same}}^{h}$	$J_{\mathrm{diff}}^{h}$	$\Delta_{\mathrm{class}}^{h}$	Lift h	$J_{\mathrm{rand}}$	$J_{\mathrm{global}}$
1.18%	3	0.6334	0.5972	0.0362	1.061	0.0069	0.6900
3.14%	8	0.6623	0.6318	0.0305	1.048	0.0167	0.7312
5.10%	13	0.6885	0.6560	0.0325	1.050	0.0268	0.7443
10.20%	26	0.7778	0.7557	0.0222	1.029	0.0537	0.8072
12.55%	32	0.8329	0.8150	0.0179	1.022	0.0683	0.8705

Appendix

Table 11: Vector interaction-set overlap. Entries are dataset means.

J_{\mathrm{same}}

and

J_{\mathrm{diff}}

use ground-truth ImageNet classes;

J_{\mathrm{rand}}

is the uniform random-

K

baseline; and

J_{\mathrm{global}}

is the mean per-image overlap with

\mathcal{G}_{K}

. Uncertainty columns are omitted for readability.

ResNet-18
Retained	$K$	$\Delta_{h}$	$\Delta_{w}$	$J_{\mathrm{same}}^{v}$	$J_{\mathrm{diff}}^{v}$	$\Delta_{v}$	$\Delta_{\mathrm{shuf}}$
1.18%	3	0.0362	0.0361	0.1911	0.0710	0.1200	0.0014
3.14%	8	0.0305	0.0327	0.2535	0.1289	0.1246	0.0019
5.10%	13	0.0325	0.0347	0.2994	0.1765	0.1229	0.0005
10.20%	26	0.0222	0.0279	0.3844	0.2804	0.1040	-0.0009
12.55%	32	0.0179	0.0256	0.4195	0.3248	0.0947	0.0003

Appendix

Table 12: Class-alignment gaps across signatures and controls.

\Delta_{h}

is binary vector Jaccard,

\Delta_{w}

is weighted vector Jaccard,

\Delta_{v}

is binary predicted-class scalar Jaccard, and

\Delta_{\mathrm{shuf}}

is the vector gap after label permutation. Scalar results are decision-conditioned because the selected output coordinate can differ across images. Values are means.

Model	Vector 1-NN	Scalar 1-NN	Chance	Max conf. gap
ResNet-18	0.0039	0.2423	0.0010	0.0045
ResNet-34	0.0033	0.0991	0.0010	0.0031

Appendix

Table 13: Signature diagnostics. Leave-one-out 1-NN uses binary Jaccard at the fixed absolute budget

K=32

; this is not a fraction-matched comparison across architectures, and deterministic tie handling makes the values diagnostic. The final column is the largest absolute confidence-tercile gap over the evaluated

K

grid.

Figure 13: Sample difficulty and residual interaction complexity for ResNet-18. Results use

N=10{,}000

ImageNet-1K validation images and

M=255

non-empty interactions. (a) Tie-correct Spearman correlations between label-aware difficulty measures and vector-readout complexity. (b) The same analysis for negative confidence and negative predicted-class margin, both oriented so that larger values denote harder inputs. (c,d) Corresponding results for the decision-conditioned scalar readout. (e) Relative hard–easy change in the vector metrics,

100(\mu_{\mathrm{hard}}-\mu_{\mathrm{easy}})/|\mu_{\mathrm{easy}}|

, between the top and bottom NLL quartiles. (f) Relative change in the vector metrics between misclassified and correctly classified images. Panels (e,f) display

T_{>3}^{h}

, whereas the tables report its equivalent complement

C_{\leq 3}^{h}=1-T_{>3}^{h}

to avoid redundant columns. (g,h) NLL versus the vector effective interaction count

N_{\mathrm{eff}}^{h}

and low-order mass

C_{\leq 3}^{h}

(shown as a percentage), respectively. Associations are uniformly weak; the clearest change is a small broadening of the interaction-magnitude distribution, not a shift toward higher interaction orders.

Figure 14: Sample difficulty and residual interaction complexity for ResNet-34. Results are computed over a class-balanced subset of

N=10{,}000

ImageNet-1K validation images and all

M=65{,}535

non-empty residual interactions. All difficulty scores are oriented so that larger values indicate harder inputs. (a) Tie-correct Spearman correlations between label-aware difficulty measures, including NLL, prediction error, and negative true-class margin, and vector-readout measures of interaction breadth, order, and Top-

K

compressibility. (b) Corresponding correlations for the label-free measures of negative confidence and negative predicted-class margin. (c,d) Scalar-readout counterparts of (a,b), where the scalar coordinate is the class predicted by the all-open model. (e) Relative changes in vector-readout complexity between the highest- and lowest-NLL quartiles. (f) Relative changes between misclassified and correctly classified images. These comparisons cover effective interaction counts, interaction order, high-order coefficient mass, and reconstruction budgets. (g,h) Binned distributions of NLL against the vector effective interaction count

N_{\mathrm{eff}}^{h}

and low-order coefficient mass

C_{\leq 3}^{h}

, respectively; color intensity denotes the number of images in each bin. Across all analyses, the correlations and group differences remain small: harder inputs neither recruit substantially broader interaction supports nor shift coefficient mass toward higher orders, despite the large interaction dictionary.

Difficulty	$N_{\rm eff}^{h}$	$N_{\rm ent}^{h}$	$\bar{k}^{h}$	$\kappa^{h}$	$C_{\leq 3}^{h}$	$K_{.10}^{\rm err}$	$K_{.05}^{\rm err}$	$K_{.90}^{\rm mass}$	$K_{.95}^{\rm mass}$	$K_{.99}^{\rm mass}$
ResNet-18
NLL	0.0313	0.0352	0.0157	0.0182	-0.0080	0.0711	0.0489	0.0267	0.0232	0.0238
Wrong	0.0186	0.0237	0.0063	0.0083	0.0002	0.0264	-0.0009	0.0170	0.0116	0.0103
$-$ True margin	0.0268	0.0306	0.0137	0.0162	-0.0073	0.0575	0.0351	0.0237	0.0187	0.0187
$-$ Confidence	0.0383	0.0407	0.0154	0.0191	-0.0080	0.0883	0.0796	0.0334	0.0298	0.0335
$-$ Predicted margin	0.0339	0.0355	0.0102	0.0145	-0.0052	0.0665	0.0598	0.0315	0.0254	0.0290

Appendix

Table 14: Tie-correct Spearman correlations for vector-readout complexity. Every difficulty score is oriented so that larger values mean harder inputs.

C_{\leq 3}^{h}

is shown instead of the redundant

T_{>3}^{h}=1-C_{\leq 3}^{h}

.

Difficulty	$N_{\rm eff}^{v}$	$N_{\rm ent}^{v}$	$\bar{k}^{v}$	$\kappa^{v}$	$C_{\leq 3}^{v}$
ResNet-18
NLL	0.0533	0.0637	-0.0355	-0.0456	0.0275
Wrong	0.0234	0.0333	-0.0186	-0.0320	0.0211
$-$ True margin	0.0478	0.0571	-0.0369	-0.0435	0.0290
$-$ Confidence	0.0654	0.0757	-0.0326	-0.0447	0.0250
$-$ Predicted margin	0.0550	0.0639	-0.0365	-0.0443	0.0286

Appendix

Table 15: Tie-correct Spearman correlations for scalar-readout complexity. The scalar coordinate is the class predicted by the all-open model and is fixed across masks for each image.

ResNet-18
Quantity	Easy	Hard	Hard $-$ Easy	Relative (%)
$N_{\rm eff}^{h}$	89.1949	89.4823	0.2874	0.322
$N_{\rm ent}^{h}$	62.7012	63.0648	0.3635	0.580
$\kappa^{h}$	4.9283	4.9319	0.0036	0.073
$C_{\leq 3}^{h}$	0.135677	0.135351	-0.000326	-0.241
$K_{.10}^{\rm err}$	148.7320	151.5508	2.8188	1.895

Appendix

Table 16: Loss-quartile comparison. Easy and hard denote the bottom and top NLL quartiles. Relative change is

100(\mathrm{Hard}-\mathrm{Easy})/|\mathrm{Easy}|

. Values are dataset means; no uncertainty columns are included. For

C_{\leq 3}

, the relative percentage can appear comparatively large because the ResNet-34 baseline is close to zero; the absolute gap should be used to assess its practical size.

ResNet-18
Quantity	Correct	Wrong	Wrong $-$ Correct	Relative (%)
$N_{\rm eff}^{h}$	89.2743	89.4619	0.1876	0.210
$N_{\rm ent}^{h}$	62.7944	63.0482	0.2538	0.404
$\kappa^{h}$	4.9300	4.9320	0.0020	0.041
$C_{\leq 3}^{h}$	0.135411	0.135413	$2.22\!\times\!10^{-6}$	0.002
$K_{.10}^{\rm err}$	150.5648	151.3084	0.7436	0.494

Appendix

Table 17: Correctness comparison. The ResNet-18 subset contains

6{,}994

correct and

3{,}006

incorrect predictions; the ResNet-34 subset contains

7{,}355

correct and

2{,}645

incorrect predictions. Relative change is

100(\mathrm{Wrong}-\mathrm{Correct})/|\mathrm{Correct}|

. Absolute gaps are the more informative scale for the small

C_{\leq 3}

baseline.

Model	$N_{\rm eff}^{h}$	$\kappa^{h}$	$C_{\leq 3}^{h}$	$K_{.10}^{\rm err}$	$K_{.95}^{\rm mass}$	$N_{\rm eff}^{v}$	$\kappa^{v}$
ResNet-18	-0.0306	-0.001003	$9.652\!\times\!10^{-5}$	0.1311	-0.0086	0.0913	-0.007939
ResNet-34	-5.7765	-0.000242	$2.762\!\times\!10^{-7}$	0.8280	-5.1910	0.9821	0.000174

Appendix

Table 18: Class-controlled NLL slopes. Each entry is the raw

\beta_{1}

from a within-class fixed-effect regression. Because the quantities have different units and scales, their signed magnitudes should be interpreted within each column rather than compared directly across metrics or architectures.

	Method	Norm ratio	Rel. error ( $\downarrow$ )	Scaled error ( $\downarrow$ )	Cosine ( $\uparrow$ )	Top-1 agree ( $\uparrow$ )
18	$A_{x}^{\mathrm{raw}}$	$79.400$	$79.200$	$0.961$	$0.222$	$0.006$
	$A_{x}^{\mathrm{mean}}$	$0.310$	$0.977$	$0.961$	$0.222$	$0.006$
	$A_{x}^{\mathrm{ctr}}$	$77.400$	$77.200$	$0.959$	$0.229$	$0.006$
	Möbius 1 1 1 ResNet-18, before rounding: relative error $8.51\times 10^{-7}$ , optimally scaled error $8.55\times 10^{-7}$ .	1.000	0.000	0.000	1.000	1.000
34	$A_{x}^{\mathrm{raw}}$	$18{,}852.327$	$18{,}851.887$	$0.889$	$0.440$	$0.025$
	$A_{x}^{\mathrm{mean}}$	$0.288$	$0.910$	$0.889$	$0.440$	$0.025$

Do ResNets Route? Sparse Interaction Experts in Residual Networks

Abstract

Figures & tables

Appendix

Residual Learning

When Does Routing Become Interpretable? Causal Probes on Block Attention Residuals

Rank Collapse, Fixed Points, and the Renormalization Group Structure of MLP Residual Networks

Early learning in large residual networks

Abstract

Residual Learning

Explore similar work

When Does Routing Become Interpretable? Causal Probes on Block Attention Residuals

Rank Collapse, Fixed Points, and the Renormalization Group Structure of MLP Residual Networks

Early learning in large residual networks