We present this as a negative result with an explanatory mechanism, not as a formal upper bound. Predictive coding networks (PCNs) admit a K-way energy probe in which each candidate class is fixed as a target, inference is run to settling, and the per-hypothesis settled energies are compared. The probe appears to read a richer signal source than softmax, since the per-hypothesis energy depends on the entire generative chain. We argue this appearance is misleading under the standard Pinchetti-style discriminative PC formulation. We present an approximate reduction showing that with target-clamped CE-energy training and effectively-feedforward latent dynamics, the K-way energy margin decomposes into a monotone function of the log-softmax margin plus a residual that is not trained to correlate with correctness. The decomposition predicts that the structural probe should track softmax from below. We test this across six conditions on CIFAR-10: extended deterministic training, direct measurement of latent movement during inference, a post-hoc decoder fairness control on a backpropagation network, a matched-budget PC vs BP comparison, a five-point Langevin temperature sweep, and trajectory-integrated MCPC training. In every condition the probe sat below softmax. The gap was stable across training procedures within the discriminative PC family. Final-state and trajectory-integrated training produced probes whose AUROC_2 values differed by less than 10^-3 at deterministic evaluation. The empirical regime is small: single seed, 2.1M-parameter network, 1280 test images. We frame the result as a preprint inviting replication. We discuss conditions under which the decomposition does not apply (bidirectional PC, prospective configuration, generative PC, non-CE energy formulations) and directions for productive structural probing the analysis does not foreclose.
Figures & tables
Epoch
Structural Acc
Softmax Acc
Structural AUROC 2
Softmax AUROC 2
5
37.42%
69.77%
0.6470
0.8020
10
60.78%
73.91%
0.7487
0.8150
15
65.47%
77.03%
0.7543
0.8319
20
67.27%
77.73%
0.7770
0.8513
25
69.92%
78.20%
0.7661
0.8712
Table 1
Figure 1: Structural probe AUROC 2 versus softmax AUROC 2 over 25 epochs of standard discriminative PC training (TinyConvPCN, CIFAR-10, single seed). The shaded region indicates the probe-vs-softmax gap. The gap narrows between epochs 5 and 10 as both probes improve from under-trained baselines, then re-widens through epoch 25 as softmax continues to improve while the structural probe plateaus.
Quantity
Value
Per-layer mean | Δ h_l| over T=13 steps, range across l=1,2,3
1.6×10−4 to 2.1×10−3
Gradient magnitude at latents, range across l=1,2,3
1.3×10−4 to 1.7 ×10−3
Energy decrease over inference loop
25.09 → 25.07 (0.08%)
Final MSE between settled latent and feedforward init, range across l=1,2,3
4 ×10−8 to 9 ×10−6
Table 3
Probe
Accuracy
AUROC 2
BP softmax (encoder readout)
71.17%
0.7765
BP+decoder structural probe
71.09%
0.7674
Table 4
Epoch
PC Acc
PC Softmax AUROC 2
BP Acc
BP Softmax AUROC 2
Δ AUROC 2
5
69.77%
0.8020
71.17%
0.7765
+0.0255
10
73.91%
0.8150
72.19%
0.8146
+0.0004
15
77.03%
0.8319
76.64%
0.8442
-0.0123
20
77.73%
0.8513
77.81%
0.8433
+0.0080
25
78.20%
0.8712
77.73%
0.8534
+0.0178
Table 5
Eval σ
Max per-layer mean | Δ h|
Structural Acc
Structural AUROC 2
Δ AUROC 2 vs softmax
0
3.4×10−3
58.05%
0.7531
− 0.0759
10−3
6.8×10−3
58.05%
0.7515
− 0.0775
10−2
5.6×10−2
56.72%
0.7360
− 0.0930
10−1
5.6×10−1
28.59%
0.5726
− 0.2564
1.0
5.6×100
10.16%
0.5093
− 0.3197
Table 6
Figure 2: K-way energy probe AUROC 2 on a Langevin-trained network as a function of eval-time noise σ (log scale, with σ =0 shown at the left edge). The softmax baseline (blue dashed) is invariant to inference noise. The structural probe (red) sits below the softmax baseline at σ≤10−2 and collapses to near-chance accuracy at σ≥10−1 .
Eval σ
Phase A AUROC 2 (final-state)
Phase B AUROC 2 (MCPC)
Δ (B − A)
0
0.7531
0.7537
+0.0006
10−2
0.7360
0.7467
+0.0107
Table 8
Condition
Probe AUROC 2
Softmax AUROC 2
Δ
C1: Det. PC, 25 epochs
0.7661
0.8712
− 0.105
C3: BP + post-hoc decoder
0.7674
0.7765
− 0.009
C5 ( σ =0): Langevin-trained, det. eval
0.7531
0.8290
− 0.076
C5 ( σ = 10−2 ): Langevin-trained, σ = 10−2 eval
0.7360
0.8290
− 0.093
C6 ( σ =0): MCPC-trained, det. eval
0.7537
0.8310
− 0.077
C6 ( σ = 10−2 ): MCPC-trained, σ = 10−2 eval
0.7467
0.8310
− 0.084
Table 9
Figure 3: Probe-vs-softmax gap (probe AUROC 2 minus softmax AUROC 2 ) across the six tested conditions, colour-coded by training family. The BP+decoder condition (gap − 0.009) is the smallest, consistent with Prediction 4: under post-hoc reconstruction training of the decoder, the residual term is forced to be approximately k -invariant, and the K-way margin tracks the log-softmax margin from the BP encoder. The four discriminative-PC-family conditions cluster between − 0.076 and − 0.093, consistent with Prediction 5’s approximate invariance under training procedure within the discriminative PC family.