Probabilistic State Space Enumeration in Neural Network Computation
Authors: Victor O. K. Li, Yang Han, Jacqueline C. K. Lam
Organizations: ElohaX Ltd. · The Hong Kong University of Science and Technology (Guangzhou), China · The Chinese University of Hong Kong, Hong Kong SAR, China · CASBS, Stanford University, United States
Many methods for pruning neural networks rank weights or groups of weights by importance and retain a single subset. We study how channel groups contribute jointly to the output of multilayer perceptrons (MLPs) in Transformer language models. Inspired by network reliability analysis, probabilistic state space enumeration (PSSE) represents the dense MLP output as the expected importance-corrected masked output over 2G joint states of G channel groups. Summing the most probable states' probability-weighted outputs gives an approximation whose absolute error is bounded by the omitted probability mass times the maximum masked-output norm. NORDER-II adapts ORDER-II to enumerate states in decreasing probability without sorting the full space. Its first B states maximize the probability mass captured at that budget. A weight-entropy scorer fixes each layer's ordering from pretrained weights. We evaluate all 168 MLP layers across six models from the Llama, Gemma, and Qwen families on two datasets with 8, 16, and 32 channel groups. In all 1,008 comparisons, the 95%-mass prefix yields lower mean relative error than uniform sampling at the same state count, with or without bias correction. At all three group counts, it also yields lower median and 99th-percentile relative error than a single top-k mask. For downstream inference, we use the number of distinct channels in each prefix to set pruned MLP widths. With one validation-selected group count per model, PSSE achieves lower perplexity than both top-k pruning baselines in all 36 matched-FLOP settings. Our pruned decoder MLPs achieve mean speedups of 1.11×, 1.23×, and 1.36× over dense MLPs at 10%, 20%, and 30% target MLP projection FLOP savings, respectively. PSSE offers a principled path toward both probability-mass-controlled approximation and more efficient inference in pretrained language models.
Figures & tables
Figure 1: Captured probability mass and relative MLP output error (logarithmic vertical axes). (a) PSSE stops when its m most probable states reach 95% mass. A uniform sample of m states captures m/2G in expectation. For the single mask, the plotted mass is its joint probability (Eq. ( 5 )). Points are medians of 12 model–corpus means. (b) Markers show median error and caps show the 99th percentile across 86,016 prompt–layer comparisons per G . The single mask sums contributions from the k groups with the highest inclusion probabilities, with k set to the layer’s rounded mean active count per selected state. Uniform and Uniform + HT use the same draws (Appendix Table 4 , panel B).
PSSE / Best top- k (channel or grouped)
Model
G
Dataset
10%
20%
30%
Llama-3.2-1B
12
WikiText-2
15.12 / 16.09
24.23 / 30.93
47.55 / 97.16
PubMed
12.31 / 13.25
19.73 / 29.85
50.85 / 98.29
Llama-3.2-3B
28
WikiText-2
11.93 / 12.08
17.87 / 22.77
32.69 / 68.01
PubMed
9.39 / 9.59
13.79 / 17.28
28.34 / 44.66
Gemma-3-1B
16
WikiText-2
22.18 / 27.49
33.23 / 38.28
50.89 / 60.92
Table 1: Held-out perplexity at three target reductions in MLP projection FLOPs. Best top- k is the lower perplexity of channel top- k and grouped top- k in each setting. Bold marks the lower value in each pair. All three methods use the indicated validation-selected G and the same projection FLOPs.
Figure 2: Held-out perplexity as MLP projection FLOP savings increase. Each point is the median across six models on the indicated dataset. Each model uses one validation-selected G across methods and budgets. Lower perplexity is better.
Target FLOP savings
Dense
PSSE
Channel top- k
Grouped top- k
10%
12.45
11.21 ( 1.110× )
11.31 ( 1.108× )
11.31 ( 1.108× )
20%
12.50
10.06 ( 1.233× )
10.07 ( 1.230× )
10.08 ( 1.230× )
30%
12.50
9.13 ( 1.363× )
9.31 ( 1.350× )
9.32 ( 1.350× )
Table 2: Decoder MLP latency (ms) and speedup over dense MLPs. We time all methods on the same GPU model using one 512-token sequence per pass. For each model and dataset, we sum the MLP times across layers and take the median over 40 passes. At each savings target, latency is the mean of these 12 medians (six models, two datasets). Speedup in parentheses is the mean of the 12 within-pair ratios of dense to pruned median latency.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
G
2G
Selected
Frac.
Match
4
16
10
0.625
Yes
8
256
113
0.441
Yes
12
4096
1774
0.433
Yes
16
65536
12684
0.194
Yes
Appendix
Table 3: Controlled exhaustive-enumeration check at τ=0.95 . NORDER-II selects the same ordered states as exhaustive sorting in all rows.
Figure 3: Mean KL divergence from the dense classifier’s output distribution to its approximation, by number of states. Probability-ordered prefixes yield lower divergence than uniform samples of the same size.
A. Mean error by model and corpus
WikiText-2
PubMed
Model
G
m
PSSE
Uniform
HT
Top- k mask
PSSE
Uniform
HT
Top- k mask
Llama-3.2-1B
8
4.7
0.598
0.985
2.382
0.695
0.605
0.985
2.379
0.704
16
27.1
0.549
1.000
2.847
0.712
0.553
1.000
2.797
0.715
32
2,731.9
0.474
1.000
2.276
0.704
0.481
1.000
2.362
0.705
Llama-3.2-3B
8
3.7
0.600
0.989
2.422
0.680
0.597
0.989
2.431
0.679
Appendix
Table 4: Relative error of local MLP output approximations (lower is better). Uniform and HT use the same random draw at PSSE’s state count, without and with Horvitz–Thompson correction, respectively. The single top- k mask retains each layer’s rounded mean number of active groups per selected state. Panel A averages layer means within each model. The column m gives the mean prefix length at 95% mass. Panel B pools all 86,016 prompt–layer errors per method and G (256 prompts per layer, 168 layers, two corpora) before computing the median and 99th percentile (P99).
WikiText-2
PubMed
m
Error
m
Error
Model
G
Weight
Norm
Weight
Norm
Weight
Norm
Weight
Norm
Llama-3.2-1B
16
27.1
70.9
0.549
0.445
27.1
59.8
0.553
0.456
32
2,731.9
1.56×105
0.474
0.306
2,731.9
3.52×104
0.481
0.313
Llama-3.2-3B
16
19.6
64.8
0.564
0.444
19.6
66.5
0.564
0.447
32
1,667.5
8.33×104
0.474
0.303
1,667.5
7.36×104
0.477
0.307
Appendix
Table 5: Local scorer comparison at 95% mass. m is the mean prefix length in states. Error is the mean relative ℓ2 error against the dense MLP output. Both quantities average over prompts and layers within a model. Weight denotes weight entropy and Norm denotes contribution norm.
WikiText-2 error
PubMed error
Channels omitted (%)
Model
G
Contiguous
Random
Contiguous
Random
Contiguous
Random
Llama-3.2-1B
16
0.5495
0.5512 ± 0.0158
0.5526
0.5543 ± 0.0159
30.9
31.0
32
0.4741
0.4664 ± 0.0118
0.4815
0.4722 ± 0.0122
20.5
18.8
Llama-3.2-3B
16
0.5641
0.5475 ± 0.0148
0.5645
0.5508 ± 0.0129
32.6
30.5
32
0.4736
0.4753 ± 0.0083
0.4772
0.4786 ± 0.0071
19.3
19.6
Gemma-3-1B
16
0.3868
0.3896 ± 0.0120
0.4031
0.4100 ± 0.0131
31.5
31.4
Appendix
Table 6: Local error for contiguous groups and ten random partitions (mean ± sample SD across trials). The last two columns give the mean percentage of channels absent from every state in the prefix. For a given partition, the weight-based prefix is the same on both corpora.
Slope α
Floor πmin
Mass τ
States/layer
Channels omitted (%)
Relative error
2
0.00
0.95
1.08×107
0.0
0.094
4
0.00
0.95
6.19×104
2.8
0.222
8
0.00
0.95
536.9
24.9
0.469
2
0.05
0.95
5.19×107
0.0
0.075
8
0.05
0.95
1.68×105
0.0
0.100
8
0.00
0.90
256.2
30.7
0.527
Appendix
Table 7: Probability settings for contiguous groups. Each reported outcome is the median of 24 means, one per model, corpus, and group count. A channel is omitted if it appears in none of the selected states. The pruning experiments set a separate budget for MLP projection FLOPs.
Llama-3.2-1B
Llama-3.2-3B
G
α
Floor
τ
m
% omitted
WT
PM
m
% omitted
WT
PM
16
2
0.00
0.95
4,372.1
0.0
0.118
0.120
4,208.2
0.0
0.124
0.125
4
0.00
0.95
309.8
5.1
0.301
0.305
258.1
6.2
0.316
0.319
8
0.00
0.95
27.1
30.9
0.549
0.553
19.6
32.6
0.564
0.564
2
0.05
0.95
9,835.9
0.0
0.089
0.090
9,587.9
0.0
0.091
0.092
8
0.05
0.95
618.8
0.0
0.130
0.130
535.4
0.0
0.129
0.129
Appendix
Table 8: Sensitivity to the probability settings for Llama-3.2-1B and Llama-3.2-3B. We report mean prefix length ( m ), the percentage of channels absent from every selected state, and mean local relative error on WikiText-2 (WT) and PubMed (PM).
Gemma-3-1B
Gemma-3-4B
G
α
Floor
τ
m
% omitted
WT
PM
m
% omitted
WT
PM
16
2
0.00
0.95
3,976.1
0.0
0.099
0.101
4,211.7
0.0
0.093
0.095
4
0.00
0.95
231.5
7.0
0.239
0.248
292.1
7.7
0.226
0.235
8
0.00
0.95
19.2
31.5
0.387
0.403
30.6
31.6
0.376
0.387
2
0.05
0.95
9,174.8
0.0
0.076
0.077
9,661.8
0.0
0.073
0.074
8
0.05
0.95
554.7
0.0
0.101
0.103
722.4
0.0
0.100
0.102
Appendix
Table 9: Sensitivity to the probability settings for Gemma-3-1B and Gemma-3-4B. We report mean prefix length ( m ), the percentage of channels absent from every selected state, and mean local relative error on WikiText-2 (WT) and PubMed (PM).
Qwen3-1.7B
Qwen3-8B
G
α
Floor
τ
m
% omitted
WT
PM
m
% omitted
WT
PM
16
2
0.00
0.95
5,062.1
0.0
0.122
0.126
4,933.8
0.0
0.118
0.119
4
0.00
0.95
473.6
5.8
0.289
0.294
463.7
6.4
0.312
0.316
8
0.00
0.95
41.6
28.6
0.513
0.520
54.1
29.0
0.510
0.514
2
0.05
0.95
1.09×104
0.0
0.087
0.089
1.08×104
0.0
0.089
0.089
8
0.05
0.95
721.1
0.0
0.124
0.125
819.4
0.0
0.124
0.125
Appendix
Table 10: Sensitivity to the probability settings for Qwen3-1.7B and Qwen3-8B. We report mean prefix length ( m ), the percentage of channels absent from every selected state, and mean local relative error on WikiText-2 (WT) and PubMed (PM).
Figure 4: Held-out perplexity reduction for each model, dataset, and budget. Entries give 100(1−PPLPSSE/PPLbest) , where PPLbest is the lower of the two top- k perplexities. Positive values favor PSSE. Each row uses the same selected G for all methods.
WikiText-2
PubMed
Model
G
Save %
Dense
PSSE
Ch.
Grp.
Dense
PSSE
Ch.
Grp.
Llama-3.2-1B
12
10
11.10
15.12
85.77
16.09
9.01
12.31
33.62
13.25
20
11.10
24.23
150.49
30.93
9.01
19.73
59.61
29.85
30
11.10
47.55
2907.95
97.16
9.01
50.85
231.09
98.29
Llama-3.2-3B
28
10
9.02
11.93
19.10
12.08
7.60
9.39
13.54
9.59
20
9.02
17.87
30.66
22.77
7.60
13.79
21.06
17.28
Appendix
Table 11: Held-out perplexity of models with pruned MLPs at matched projection FLOPs using one validation-selected G per model. Dense gives the original model’s PPL. Save % is the target reduction in MLP projection FLOPs. Ch. and Grp. denote channel top- k and grouped top- k . Bold marks the lowest PPL among the three pruning methods on each corpus.
WikiText-2
PubMed
Model
G
Save %
Dense
PSSE
Ch.
Grp.
Dense
PSSE
Ch.
Grp.
Llama-3.2-1B
12
10
5.19
4.79
4.75
4.74
5.21
4.81
4.77
4.76
20
5.16
4.48
4.60
4.60
5.21
4.49
4.61
4.61
30
5.17
4.10
4.32
4.32
5.23
4.11
4.33
4.32
Llama-3.2-3B
28
10
12.67
10.73
10.40
10.40
12.68
10.74
10.44
10.44
20
12.68
10.12
10.12
10.12
12.69
10.13
10.13
10.13
Appendix
Table 12: Median decoder MLP latency (ms) by model, validation-selected group count G , MLP projection savings target, and corpus. Each entry is the median of 40 passes on an RTX 5090, summing MLP times across layers for one 512-token sequence per pass. Ch. and Grp. denote channel top- k and grouped top- k .