Function-Preserving Audits of Visual Weight Readers
Organizations: LG Uplus, Seoul, South Korea
Abstract
A neural field can encode the same image with different weights. A reader that recognizes one such representation may therefore rely on parameter coordinates rather than the visual function. We study this distinction through function-preserving transformations, measuring recognition, prediction disagreement, and numerical fidelity together. Across shape, digit, and natural-image fields, high accuracy on the original weights coexists with substantial sensitivity to equivalent representations. On 10,000 official CIFAR10 test origins, one-period hidden-bias shifts change 7.3% of a released HiddenProbe configuration's predictions while barely changing accuracy; three-period shifts change 22.0% and reduce accuracy from 57.8% to 54.4%. Output-only ProbeGen remains unchanged. These effects are smaller than in a separate adapted-reader study, where matched phase and sign controls remove the corresponding sensitivities. Controlled readout studies show why invariance is insufficient: complete canonical hidden features reach 38.7% on MNIST, whereas explicitly composing those activations, outgoing weights, and output bias into field outputs reaches 81.5%. The information is present, but the learned readout does not recover it as effectively. A separate 720-response study of language models reading numerical weights shows why stochastic audits require repeats on both inputs: observed accuracy losses need not yield rejection of response-law equality. Together, these results distinguish invariance, useful visual information, and the evidence required to evaluate stochastic readers.
Figures & tables
| Shapes | MNIST | |||||
|---|---|---|---|---|---|---|
| Reader | Raw | Joint | Disagree | Raw | Joint | Disagree |
| Flat ridge | 32.5 | 20.0 | 75.0 | 13.5 | 7.0 | 89.5 |
| Stats ridge | 96.7 | 24.2 | 75.0 | 33.0 | 11.5 | 89.0 |
| Norm+Stats | 97.5 | 97.5 | 0.0 | 33.5 | 33.5 | 0.0 |
| Canon+Flat | 71.7 | 71.7 | 0.0 | 33.5 | 33.5 | 0.0 |
| Probe ridge | 100.0 | 100.0 | 0.0 | 74.0 | 74.0 | 0.0 |
| Reader | Raw | Phase 1 | Phase 3 | Sign | |
|---|---|---|---|---|---|
| HiddenProbe | 57.8 | 57.6 | 54.4 | 57.9 | 7.3 |
| ProbeGen | 57.3 | 57.3 | 57.3 | 57.3 | 0.0 |
| Population | Reader features | Raw (%) | Averaged (%) | Change | |
|---|---|---|---|---|---|
| CIFAR10 | Field outputs | 24.2 | 24.2 | +0.0 | |
| CIFAR10 | Hidden activations | 16.8 | 18.4 | +1.6 | 0.644 |
| CIFAR10 | Hidden preactivations | 21.8 | 11.4 | -10.4 | 0.774 |
| CIFAR10 | Weight statistics | 17.8 | 10.6 | -7.2 | 0.916 |
| MNIST | Field outputs | 81.1 | 81.1 | +0.0 | |
| MNIST | Hidden activations | 14.3 | 12.5 | -1.8 | 0.893 |
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
| Reader | Raw | Perm. | Scale | Joint | Dis. | Both |
|---|---|---|---|---|---|---|
| Flat ridge | 32.5 | 38.3 | 15.8 | 20.0 | 75.0 | 10.8 |
| Stats ridge | 96.7 | 96.7 | 21.7 | 24.2 | 75.0 | 24.2 |
| Norm+Stats | 97.5 | 97.5 | 97.5 | 97.5 | 0.0 | 97.5 |
| Canon+Flat | 71.7 | 71.7 | 71.7 | 71.7 | 0.0 | 71.7 |
| Probe ridge (81 queries) | 100.0 | 100.0 | 100.0 | 100.0 | 0.0 | 100.0 |
| Set | 96.7 | 96.7 | 26.7 | 20.0 | 80.8 | 19.2 |
| Reader | Raw | Joint | Dis. |
|---|---|---|---|
| Flat ridge | 13.5 | 7.0 | 89.5 |
| Stats ridge | 33.0 | 11.5 | 89.0 |
| Norm+Stats | 33.5 | 33.5 | 0.0 |
| Canon+Flat | 33.5 | 33.5 | 0.0 |
| Probe ridge | 74.0 | 74.0 | 0.0 |
| Set | 68.5 | 20.0 | 83.5 |
| Reader | Raw [95% CI] | Joint [95% CI] | Change [95% CI] | C W | W C |
|---|---|---|---|---|---|
| Flat ridge | 32.5 [24.8, 41.3] | 20.0 [13.8, 28.0] | -12.5 [-22.5, -3.3] | 26 | 11 |
| Stats ridge | 96.7 [91.7, 98.7] | 24.2 [17.4, 32.6] | -72.5 [-78.3, -66.7] | 87 | 0 |
| Norm+Stats | 97.5 [92.9, 99.1] | 97.5 [92.9, 99.1] | 0.0 [0.0, 0.0] | 0 | 0 |
| Canon+Flat | 71.7 [63.0, 79.0] | 71.7 [63.0, 79.0] | 0.0 [0.0, 0.0] | 0 | 0 |
| Probe ridge (81 queries) | 100.0 [96.9, 100.0] | 100.0 [96.9, 100.0] | 0.0 [0.0, 0.0] | 0 | 0 |
| Set | 96.7 [91.7, 98.7] | 20.0 [13.8, 28.0] | -76.7 [-80.8, -71.7] | 93 | 1 |
| Reader | Raw [95% CI] | Joint [95% CI] | Change [95% CI] | C W | W C |
|---|---|---|---|---|---|
| Flat ridge | 13.5 [9.4, 18.9] | 7.0 [4.2, 11.4] | -6.5 [-12.0, -1.0] | 25 | 12 |
| Stats ridge | 33.0 [26.9, 39.8] | 11.5 [7.8, 16.7] | -21.5 [-27.5, -16.0] | 56 | 13 |
| Norm+Stats | 33.5 [27.3, 40.3] | 33.5 [27.3, 40.3] | 0.0 [0.0, 0.0] | 0 | 0 |
| Canon+Flat | 33.5 [27.3, 40.3] | 33.5 [27.3, 40.3] | 0.0 [0.0, 0.0] | 0 | 0 |
| Probe ridge (81 queries) | 74.0 [67.5, 79.6] | 74.0 [67.5, 79.6] | 0.0 [0.0, 0.0] | 0 | 0 |
| Set | 68.5 [61.8, 74.5] | 20.0 [15.0, 26.1] | -48.5 [-56.0, -40.5] | 112 | 15 |
| Dataset | Quartile | Raw | Joint |
|---|---|---|---|
| Shapes | Q1 | 93.3 | 23.3 |
| Shapes | Q2 | 96.7 | 16.7 |
| Shapes | Q3 | 96.7 | 20.0 |
| Shapes | Q4 | 100.0 | 20.0 |
| MNIST | Q1 | 66.0 | 20.0 |
| MNIST | Q2 | 72.0 | 18.0 |
| Reader | Raw accuracy (%) | Phase-one change (pp) | Phase-one | Phase-three acc. | Sign acc. | Sign |
|---|---|---|---|---|---|---|
| HiddenProbe | 30.0 [27.1, 33.0] | -8.1 [-10.4, -6.0] | 61.6 | 14.2 | 29.4 | 27.0 |
| HiddenPhase | 29.3 [26.4, 32.2] | 0.0 [0.0, 0.0] | 0.0 | 29.3 | 28.6 | 19.1 |
| HiddenAugment | 29.4 [26.5, 32.3] | 0.8 [-0.1, 1.8] | 18.0 | 27.8 | 29.3 | 13.2 |
| HiddenGram | 28.7 [25.9, 31.6] | 0.0 [0.0, 0.0] | 0.0 | 28.7 | 28.7 | 0.0 |
| PhasorW12 | 24.5 [22.5, 26.6] | 0.0 [0.0, 0.0] | 0.0 | 24.5 | 24.5 | 0.0 |
| ProbeGen | 29.2 [26.3, 32.1] | 0.0 [0.0, 0.0] | 0.0 | 29.2 | 29.2 | 0.0 |
| Reader | Raw accuracy (%) | Phase-one change (pp) | Phase-one | Phase-three acc. | Sign acc. | Sign |
|---|---|---|---|---|---|---|
| HiddenProbe | 30.1 [27.2, 33.1] | -9.1 [-11.1, -7.0] | 60.2 | 13.9 | 30.0 | 24.7 |
| HiddenPhase | 29.4 [26.6, 32.4] | 0.0 [0.0, 0.0] | 0.0 | 29.4 | 29.3 | 18.9 |
| HiddenAugment | 30.0 [27.1, 32.9] | 0.4 [-0.6, 1.4] | 17.5 | 28.2 | 28.8 | 12.5 |
| HiddenGram | 29.7 [26.8, 32.6] | 0.0 [0.0, 0.0] | 0.0 | 29.7 | 29.7 | 0.0 |
| PhasorW12 | 20.3 [18.7, 22.0] | 0.0 [0.0, 0.0] | 0.0 | 20.3 | 20.3 | 0.0 |
| ProbeGen | 29.2 [26.4, 32.1] | 0.0 [0.0, 0.0] | 0.0 | 29.2 | 29.2 | 0.0 |
| Bundle 0 | Bundle 1 | Bundle 2 | ||||
|---|---|---|---|---|---|---|
| Reader | Raw | Phase one | Raw | Phase one | Raw | Phase one |
| HiddenProbe | 26.8 | 17.1 | 31.4 | 25.2 | 31.8 | 23.3 |
| HiddenPhase | 28.0 | 28.0 | 27.8 | 27.8 | 32.0 | 32.0 |
| HiddenAugment | 27.8 | 28.8 | 30.1 | 31.4 | 30.2 | 30.4 |
| HiddenGram | 28.8 | 28.8 | 29.3 | 29.3 | 27.9 | 27.9 |
| PhasorW12 | 22.9 | 22.9 | 24.1 | 24.1 | 26.5 | 26.5 |
| Reader | Parameters | Selected epochs |
|---|---|---|
| HiddenProbe | 447,908 | 30–200 |
| HiddenPhase | 447,908 | 40–220 |
| HiddenAugment | 447,908 | 20–220 |
| HiddenGram | 447,908 | 20–240 |
| PhasorW12 | 1,875,270 | 20–180 |
| ProbeGen | 579,362 | 10–150 |
| Features | Train/test inputs | Accuracy (%) | Averaging change (pp) | |
|---|---|---|---|---|
| Field outputs | Original | 81.25 | +0.00 | |
| Hidden activations | Original | 15.17 | -0.67 | 0.87217 |
| Hidden + outgoing | Original | 14.17 | -0.50 | 0.87735 |
| Hidden activations | Canonical | 38.50 | +0.00 | 0.00050 |
| Hidden + outgoing | Canonical | 40.08 | +0.00 | 0.00054 |
| Preactivation + outgoing | Canonical | 29.75 | +0.00 | 0.00119 |
| Reader features | Test (%) | Averaged (%) | |
|---|---|---|---|
| Field outputs | 33.3 | 33.3 | |
| Hidden activations | 0.778 | 8.4 | 8.2 |
| Hidden preactivations | 0.858 | 11.6 | 10.1 |
| Weight statistics | 0.835 | 18.1 | 11.5 |
| Features | Input size | Parameters | Accuracy (%) | |
|---|---|---|---|---|
| Direct outputs (original) | 169 | 111,882 | 81.33 | |
| Canonical hidden | 10816 | 2,837,514 | 37.75 | 0.00035 |
| Hidden + | 10880 | 2,853,898 | 39.92 | 0.00060 |
| Hidden + | 10881 | 2,854,154 | 38.67 | 0.00076 |
| Fixed output composition | 169 | 111,882 | 81.50 |
| Reader | Training / evaluation inputs | Accuracy (%) | Disagree (%) |
|---|---|---|---|
| Stats ridge | Original / original | 96.67 0.00 | 74.38 |
| Stats ridge | Canonical / canonical | 97.50 0.00 | 0.00 |
| Norm+Stats | Original / original | 97.50 0.00 | 0.00 |
| Norm+Stats | Canonical / canonical | 97.50 0.00 | 0.00 |
| Set | Original / original | 96.25 0.42 | 80.42 |
| Set | Canonical / canonical | 95.62 0.91 | 0.00 |
| Reader | range | Accuracy change (pp) |
|---|---|---|
| HiddenProbe | 0.925–0.984 | -1.50 to +2.25 |
| MPGMN | 0.848–0.893 | -4.00 to +2.25 |
| WeightStats | 0.926–0.985 | -7.25 to -1.75 |
| ProbeGen | +0.00 to +0.00 | |
| Grid64 | +0.00 to +0.00 | |
| Grid256 | +0.00 to +0.00 |
| Band | Fit (dB) | Pair (dB) | Acc. (%) |
|---|---|---|---|
| 1.00 | 32.09 | 35.00 | 27.0 |
| 0.50 | 26.06 | 39.87 | 25.7 |
| 0.35 | 22.98 | 46.29 | 26.7 |
| 0.25 | 20.95 | 50.80 | 24.3 |
| Reader | Training | S0 | S1 | S2 | I0 |
|---|---|---|---|---|---|
| HiddenProbe | S0 | 25.0 (31/23/21) | 25.3 (29/27/20) | 23.3 (30/20/20) | 24.3 (31/23/19) |
| HiddenProbe | S0/S1 | 22.3 (21/24/22) | 24.0 (23/27/22) | 21.7 (19/24/22) | 23.3 (24/23/23) |
| ProbeGen | S0 | 25.7 (27/27/23) | 26.0 (27/28/23) | 26.3 (27/29/23) | 26.0 (26/29/23) |
| ProbeGen | S0/S1 | 22.7 (24/20/24) | 23.3 (26/20/24) | 23.0 (25/20/24) | 22.7 (24/20/24) |
| MPGMN | S0 | 24.7 (34/23/17) | 12.7 (11/17/10) | 16.3 (19/14/16) | 18.0 (17/21/16) |
| MPGMN | S0/S1 | 20.7 (24/18/20) | 23.0 (22/23/24) | 20.0 (24/18/18) | 16.7 (18/17/15) |
| Reader | S2 S0 | I0 S0 | Balanced gain on S2 |
|---|---|---|---|
| HiddenProbe | -1.7 [-3.7,+0.3] | -0.7 [-3.7,+2.3] | -1.7 [-6.3,+3.0] |
| ProbeGen | +0.7 [-0.3,+2.0] | +0.3 [-1.7,+2.3] | -3.3 [-7.3,+0.7] |
| MPGMN | -8.3 [-15.0,-1.7] | -6.7 [-14.3,+0.7] | +3.7 [-2.3,+9.7] |
| Grid64 | +1.0 [-0.3,+2.3] | +0.7 [-0.7,+2.0] | -0.7 [-2.7,+1.0] |
| Grid256 | -0.7 [-1.7,+0.0] | -0.3 [-1.0,+0.0] | +1.3 [-1.7,+4.3] |
| WeightStats | -3.0 [-8.7,+2.7] | -3.7 [-10.0,+2.7] | +1.7 [-2.3,+5.7] |
| Source | Median | 10th pct. | Minimum | Below 30 |
|---|---|---|---|---|
| S0 | 31.92 | 29.79 | 27.25 | 69 |
| S1 | 31.37 | 29.26 | 26.79 | 118 |
| S2 | 31.39 | 29.22 | 26.15 | 125 |
| I0 | 31.70 | 29.65 | 26.89 | 93 |
| Model | Rule | Raw [95% CI] | Joint [95% CI] | Canon. [95% CI] | Dis. | Both |
|---|---|---|---|---|---|---|
| Qwen2.5 0.5B | original | 21.7 [13.1, 33.6] | 21.7 [13.1, 33.6] | 21.7 [13.1, 33.6] | 0.0 | 21.7 |
| Qwen2.5 0.5B | calibrated | 21.7 [13.1, 33.6] | 21.7 [13.1, 33.6] | 21.7 [13.1, 33.6] | 0.0 | 21.7 |
| Qwen2.5 1.5B | original | 18.3 [10.6, 29.9] | 16.7 [9.3, 28.0] | 16.7 [9.3, 28.0] | 6.7 | 16.7 |
| Qwen2.5 1.5B | calibrated | 21.7 [13.1, 33.6] | 23.3 [14.4, 35.4] | 23.3 [14.4, 35.4] | 3.3 | 21.7 |
| SmolLM2 0.36B | original | 21.7 [13.1, 33.6] | 21.7 [13.1, 33.6] | 21.7 [13.1, 33.6] | 1.7 | 21.7 |
| SmolLM2 0.36B | calibrated | 23.3 [14.4, 35.4] | 23.3 [14.4, 35.4] | 23.3 [14.4, 35.4] | 5.0 | 23.3 |
| Representation | Acc. | Changed /60 |
|---|---|---|
| Original | 21.7 | 0 |
| Permutation | 21.7 | 0 |
| Scaling | 21.7 | 0 |
| Joint | 21.7 | 0 |
| Joint then normalize | 21.7 | 0 |
| Joint then sort | 21.7 | 0 |
| Requested model | Condition | Correct /60 | Accuracy [95% CI] | Invalid /60 |
|---|---|---|---|---|
| Astra 6 | raw | 52/60 | 86.7 [75.8, 93.1] | 0 |
| Astra 6 | raw_repeat | 54/60 | 90.0 [79.9, 95.3] | 0 |
| Astra 6 | perm | 51/60 | 85.0 [73.9, 91.9] | 0 |
| Astra 6 | joint | 47/60 | 78.3 [66.4, 86.9] | 0 |
| Astra 6 | canonical | 51/60 | 85.0 [73.9, 91.9] | 0 |
| Astra 6 | canonical_repeat | 55/60 | 91.7 [81.9, 96.4] | 0 |
| Requested model | First/second | Dis. | Harm | Help | Wrong flip | acc. [95% CI] |
|---|---|---|---|---|---|---|
| Astra 6 | Raw/Perm. | 9 | 4 | 3 | 2 | -1.7 [-10.0, +6.7] |
| Astra 6 | Raw/Joint | 12 | 8 | 3 | 1 | -8.3 [-18.3, +1.7] |
| Astra 6 | Raw/Repeat | 5 | 1 | 3 | 1 | +3.3 [-1.7, +10.0] |
| Astra 6 | Canon./Repeat | 7 | 1 | 5 | 1 | +6.7 [+0.0, +13.3] |
| Astra 6 | Raw/Canon. | 9 | 5 | 4 | 0 | -1.7 [-11.7, +8.3] |
| Astra 6 | Prior/Raw | 47 | 0 | 42 | 5 | +70.0 [+61.7, +78.3] |
| Requested model | 95% CI | |
|---|---|---|
| Astra 6 | +11.7 | [+1.7, +20.0] |
| Sol 6.1 | +3.3 | [-5.0, +11.7] |
| Terra 6.1 | +8.3 | [-5.0, +23.3] |
| Luna 6 | +3.3 | [-10.0, +18.3] |
| Configuration | Raw/Joint | Raw repeat/Joint | Raw/Canonical | Joint excess |
|---|---|---|---|---|
| Astra 6 | 0.2266 | 0.0654 | 1.0000 | 0.0654 |
| Sol 6.1 | 1.0000 | 0.5078 | 0.3877 | 0.7266 |
| Terra 6.1 | 0.0044 | 0.1892 | 1.0000 | 0.4049 |
| Luna 6 | 0.8145 | 1.0000 | 1.0000 | 0.8318 |
| Input | Accuracy (%) | Calibrated (%) | Mass (%) |
|---|---|---|---|
| raw | 11.7 | 21.7 | 99.93 |
| joint | 15.0 | 18.3 | 99.93 |
| canonical | 15.0 | 26.7 | 99.93 |
| redacted | 15.0 | – | 99.93 |
| textcue | 100.0 | – | 99.99 |
| Repeats per side | Null rejection | Alternative rejection |
|---|---|---|
| 2 | 3.8 [2.8, 5.2] | 7.0 [5.6, 8.8] |
| 3 | 3.7 [2.7, 5.1] | 11.5 [9.7, 13.6] |
| Reader | Raw (%) | Joint (%) | Change (pp), 95% interval | ||
|---|---|---|---|---|---|
| Astra | 88.3 | 81.1 | 0.0185 | 0.162 | |
| Sol | 86.7 | 79.4 | 0.0259 | 0.162 |
| Config. | H4 | H16 | |||
|---|---|---|---|---|---|
| Astra | 0.0208 | 5 | 0.2263 | 0.6024 | 1.0000 |
| Sol | 0.0750 | 7 | 0.0087 | 0.0348 | 0.1303 |
| Terra | -0.0417 | 12 | 0.9615 | 0.9615 | 1.0000 |
| Luna | 0.0375 | 16 | 0.2008 | 0.6024 | 1.0000 |
| Config. | 4 | 3+1 | 2+2 | 2+1+1 | 1+1+1+1 | |
|---|---|---|---|---|---|---|
| Astra | 49 | 6 | 2 | 3 | 0 | |
| Sol | 41 | 12 | 5 | 2 | 0 | |
| Terra | 37 | 11 | 4 | 8 | 0 | |
| Luna | 25 | 17 | 8 | 8 | 2 |
| Reader | Seed | Native | Raw | Phase 1 | Phase 3 | Sign | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| HiddenProbe | 0 | 57.73 | 57.83 | 57.42 | 53.82 | 57.82 | 7.03 | 22.98 | 10.14 | 2.19 |
| HiddenProbe | 1 | 58.25 | 58.16 | 58.01 | 54.76 | 58.38 | 7.99 | 22.90 | 11.16 | 2.37 |
| HiddenProbe | 2 | 57.47 | 57.52 | 57.38 | 54.69 | 57.60 | 6.81 | 20.22 | 10.25 | 2.24 |
| ProbeGen | 0 | 57.36 | 57.36 | 57.36 | 57.36 | 57.36 | 0.00 | 0.00 | 0.00 | 0.00 |
| ProbeGen | 1 | 56.62 | 56.62 | 56.62 | 56.62 | 56.62 | 0.00 | 0.00 | 0.00 | 0.00 |
| ProbeGen | 2 | 57.86 | 57.86 | 57.86 | 57.86 | 57.86 | 0.00 | 0.00 | 0.00 | 0.00 |
| Reader | Action | Change (pp) | 95% interval (pp) | Disagreement (%) |
|---|---|---|---|---|
| HiddenProbe | Perm. | +0.00 | 0.00 | |
| HiddenProbe | Phase 1 | -0.23 | 7.28 | |
| HiddenProbe | Phase 3 | -3.41 | 22.03 | |
| HiddenProbe | Sign | +0.10 | 10.52 | |
| ProbeGen | Perm. | +0.00 | 0.00 | |
| ProbeGen | Phase 1 | +0.00 | 0.00 |
Neural Network Interpretability
Neural network interpretability studies how models represent information and make predictions, and how to explain, test, or control their internal computations.
See all 367 papers