Evidence or Marginal? What Moment-Vector Propagation Computes in Polynomial Density Neurons
Organizations: AGH University of Krakow
Abstract
When a neuron's input is lost, or known through a belief , what should it output? For a neuron storing a joint polynomial density with input density , two formulas compete. costs one matrix-vector product; Hierarchical Correlation Reconstruction introduces it as a constant-denominator approximation to . For a density model both are exact answers to different questions: is Bayesian updating with the belief as virtual evidence; installs the belief as a new marginal (Jeffrey's rule). With a missing input returns the model's conditional; generally does not. This post hoc reading is proved, and illustrated outside the pair family up to inputs; it has no pre-registered test, only one exploratory real-data pilot. Post hoc, closed forms fall within a factor 2 of all 112 measured gaps, and negative density begins near input correlation . In a pre-registered synthetic study (up to 32 inputs), 5 of 15 hypotheses are supported, 2 inconclusive and 8 not supported, including all three on transfer beyond equicorrelated Gaussians. Post hoc, where is defined, truncation usually outweighs the gap for Beta beliefs in pair neurons; triple neurons under flat and data beliefs are exceptions.
Figures & tables
| pair basis | triple basis | ||||
|---|---|---|---|---|---|
| first | convex | measured | corner | measured | |
| 2 | 0.152 | 0.289 | 0.437 | — | — |
| 4 | 0.085 | 0.201 | 0.204 | 0.093 | 0.094 |
| 8 | 0.046 | 0.127 | 0.129 | 0.044 | 0.045 |
| 16 | 0.024 | 0.073 | 0.074 | 0.022 | 0.023 |
| 32 | 0.012 | 0.039 | 0.039 | 0.012 | 0.011 |
| claim | verdict | evidence | |
|---|---|---|---|
| H1a | population onset falls with | supported | 4/4 cells strictly decreasing |
| H1b | same at | supported | 4/4; SUM/pair 0.435 ( ) vs 0.007 ( ) |
| H1c | one power law of | inconclusive | within factor 2: SUM/pair 0.55, CONTRAST/pair 0.52, SUM/triple 1.00, CONTRAST/triple 0.48 |
| H2a | higher , earlier onset | supported | 20/20 |
| H2b | triple earlier than pair (population) | supported | 8/8 |
| H2b | triple earlier than pair ( ) | supported | 8/8 |
| basis | belief | evaluations | ratio | EU closer | EC beats EU | |||
|---|---|---|---|---|---|---|---|---|
| pair | S2 | 2,902 | 0.0024 | 0.0119 | 0.0144 | 5.3 | 50 % | 16 % |
| S3a | 5,037 | 0.0004 | 0.0471 | 0.0472 | 120 | 52 % | 24 % | |
| S3b | 6,200 | 0.0041 | 0.1063 | 0.1096 | 28 | 69 % | 38 % | |
| S3c | 3,249 | 0.0019 | 0.0690 | 0.0675 | 30 | 24 % | 8 % | |
| S3d | 7,022 | 0.0002 | 0.0824 | 0.0830 | 420 | 67 % | 39 % | |
| S4 | 2,902 | 0.0002 | 0.0004 | 0.0003 | 2.0 | 41 % | 0 % |
| additive features | pair features | |||
|---|---|---|---|---|
| 2 | 0.019 | 0.023 | 0.017 | 0.009 |
| 4 | 0.046 | 0.056 | 0.041 | 0.008 |
| 8 | 0.076 | 0.073 | 0.074 | 0.013 |
| 16 | 0.089 | 0.086 | 0.091 | 0.023 |
Appendix figures & tables38 assets
Supplementary material from the paper’s appendix.
Appendix
| symbol | meaning | symbol | meaning |
|---|---|---|---|
| joint model, input density, conditional | and | ||
| row polynomial and its coefficients | and | ||
| belief, tilted belief, flat belief | distance of from flat; | ||
| moments of ; moment vector of | |||
| message, marginal-preserving form, clipped form, ground truth | latent factor, , , | ||
| inputs, degree, sample size | pair and triple coefficient arrays |
| claim | status |
|---|---|
| Semantics, tilt, conversion (Thm. 1 , Props. 2 , 3 ) | Proved; classical mathematics, applied here. Post hoc. No pre-registered test. Illustrated in full and triple models up to ; the pair family loses the contrast. |
| Kernel sum rule as the Jeffrey form (Sec. 7 ) | Population-level correspondence, not a theorem. Post hoc. |
| Gap identity and bounds (Prop. 4 ) | Proved; identity classical; bound checked on population cases. |
| Order of the gap, closed forms (Prop. 5 , Cor. 6 ) | Proved under stated assumptions; the scaling is a leading-order reading; compared post hoc with H1c points. |
| Data belief and pair structure (Prop. 7 ) | Proved; checked post hoc on population evaluations. |
| Onset bounds, corners (Props. 8 , 9 ) | Proved for the equicorrelated Gaussian copula; convex value computed on a grid; closeness to the measured onset is numerical. The value of at large is a post hoc computation, with no limit theorem. |
| evidence | posterior | marginal | |||||
|---|---|---|---|---|---|---|---|
| CD | EU | CD | EU | CD | EU | ||
| 0.1 | 0.5 | 0.013 | 0.013 | 0.014 | 0.013 | 0.014 | 0.013 |
| 1.0 | 0.008 | 0.010 | 0.011 | 0.008 | 0.010 | 0.009 | |
| 0.2 | 0.5 | 0.011 | 0.013 | 0.014 | 0.011 | 0.014 | 0.012 |
| 1.0 | 0.008 | 0.015 | 0.017 | 0.008 | 0.015 | 0.009 | |
| 0.3 | 0.5 | 0.011 | 0.016 | 0.015 | 0.011 | 0.014 | 0.011 |
| evidence | posterior | marginal | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CD | EU | closer | CD | EU | closer | CD | EU | closer | |||
| 4 | 0.10 | 0.5 | 0.131 | 0.133 | 93 % | 0.131 | 0.132 | 88 % | 0.133 | 0.133 | 88 % |
| 1.0 | 0.078 | 0.092 | 93 % | 0.075 | 0.081 | 73 % | 0.080 | 0.088 | 75 % | ||
| 0.15 | 0.5 | 0.132 | 0.136 | 90 % | 0.129 | 0.132 | 85 % | 0.133 | 0.132 | 83 % | |
| 1.0 | 0.072 | 0.078 | 92 % | 0.074 | 0.073 | 53 % | 0.088 | 0.089 | 63 % | ||
| 8 | 0.05 | 0.5 | 0.139 | 0.141 | 73 % | 0.139 | 0.139 | 63 % | 0.142 | 0.143 | 68 % |
| evidence | posterior | marginal | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| model | CD | EU | closer | CD | EU | closer | CD | EU | closer | |||
| full | 3 | 0.15 | 0.5 | 0.024 | 0.030 | 85 % | 0.029 | 0.025 | 25 % | 0.028 | 0.025 | 28 % |
| 1.0 | 0.010 | 0.027 | 92 % | 0.028 | 0.011 | 3 % | 0.023 | 0.011 | 3 % | |||
| 4 | 0.15 | 0.5 | 0.026 | 0.034 | 87 % | 0.031 | 0.026 | 18 % | 0.028 | 0.023 | 17 % | |
| 1.0 | 0.013 | 0.042 | 97 % | 0.053 | 0.013 | 0 % | 0.038 | 0.016 | 2 % | |||
| triple | 3 | 0.15 | 0.5 | 0.033 | 0.043 | 92 % | 0.037 | 0.033 | 45 % | 0.036 | 0.035 | 52 % |
| MSE of the clipped mean | mean clipped log density | |||||||
|---|---|---|---|---|---|---|---|---|
| scenario | CD | EU | REG | boosting | marginal | CD | EU | REG |
| all inputs seen | 0.046–0.059 | 0.042–0.056 | 0.019–0.021 | 0.012 | 0.083 | 0.18–0.32 | 0.21–0.36 | 0.55–0.68 |
| half missing | 0.057–0.064 | 0.071–0.075 | 0.057–0.058 | 0.038 | 0.083 | 0.14–0.21 | 0.02–0.08 | 0.07–0.12 |
| noisy | 0.058–0.064 | 0.065–0.070 | 0.050–0.057 | 0.046 | 0.083 | 0.13–0.17 | 0.04–0.13 | 0.01–0.17 |
| scenario | below the floor | rows with | CD ahead of EU | largest raw MSE of CD |
|---|---|---|---|---|
| all inputs seen | 0.6–1.2 % | 0.6–1.2 % | 0/40 | |
| half missing | 17.3–24.3 % | 0.5–0.9 % | 40/40 | |
| noisy | 12.9–22.9 % | 1.1–1.5 % | 37/40 |
| pair | triple | |||||||
|---|---|---|---|---|---|---|---|---|
| first | convex | search | measured | corner | measured | |||
| 2 | 2 | 0.247 | 0.406 | 0.545 | 0.545 | 1.09 | — | — |
| 4 | 0.140 | 0.261 | 0.261 | 0.261 | 1.04 | — | 0.199 | |
| 8 | 0.075 | 0.158 | 0.158 | 0.158 | 1.26 | — | 0.087 | |
| 16 | 0.039 | 0.090 | 0.090 | 0.090 | 1.44 | — | 0.041 | |
| 32 | 0.020 | 0.049 | 0.049 | 0.049 | 1.57 | — | 0.021 | |
| basis | measured | corners | cube | on the boundary | ||
|---|---|---|---|---|---|---|
| pair | 2 | 1.0 | 1.2 | 1.2 | 100 % | 1.7 |
| 4 | 1.8 | 1.5 | 1.7 | 75 % | 2.4 | |
| 8 | 2.5 | 2.0 | 2.4 | 75 % | 3.3 | |
| 16 | 3.5 | 2.9 | 3.8 | 75 % | 4.7 | |
| 32 | 4.4 | 4.4 | 5.5 | 77 % | 6.7 | |
| triple | 4 | 1.8 | 1.5 | 1.8 | 100 % | 2.4 |
| basis | ratio | ||||||
|---|---|---|---|---|---|---|---|
| pair | 2 | 24 | 0.008 | 0.008 | 1.0 | ||
| 4 | 112 | 0.019 | 0.035 | 1.8 | |||
| 8 | 480 | 0.040 | 0.099 | 2.5 | |||
| 16 | 1,984 | 0.083 | 0.292 | 3.5 | |||
| 32 | 8,064 | 0.169 | 0.746 | 4.4 | |||
| triple | 4 | 368 | 0.077 | 0.134 | 1.8 |
| basis | onset, | onset, pop. | undefined from | |||
|---|---|---|---|---|---|---|
| pair | 2 | 0.192 | 0.359 | 0.435 | 0.437 | 0.450 |
| 4 | 0.102 | — | 0.201 | 0.204 | 0.200 | |
| 8 | 0.067 | — | 0.123 | 0.129 | 0.150 | |
| 16 | 0.043 | — | 0.059 | 0.074 | 0.075 | |
| 32 | — | — | 0.007 | 0.039 | 0.010 | |
| triple | 2 | 0.085 | — | 0.435 | 0.437 | 0.450 |
| type | basis | onset, | onset, pop. | undefined from | |||
|---|---|---|---|---|---|---|---|
| SUM | pair | 2 | 0.192 (0.192–0.193) | 0.359 (0.359–0.360) | 0.435 (0.435–0.436) | 0.437 | 0.450 |
| 4 | 0.102 (0.101–0.102) | — | 0.201 (0.201–0.201) | 0.204 | 0.200 | ||
| 8 | 0.067 (0.067–0.067) | — | 0.123 (0.123–0.123) | 0.129 | 0.150 | ||
| 16 | 0.043 (0.043–0.043) | — | 0.059 (0.059–0.060) | 0.074 | 0.075 | ||
| 32 | — | — | 0.007 (0.007–0.008) | 0.039 | 0.010 | ||
| triple | 2 | 0.085 (0.085–0.086) | — | 0.435 (0.435–0.436) | 0.437 | 0.450 |
| type | basis | at | |||
|---|---|---|---|---|---|
| SUM | pair | 2 | 0.283 (0.282–0.283) | — | 0.992 |
| 4 | 0.147 (0.146–0.147) | — | 0.965 | ||
| 8 | 0.095 (0.095–0.095) | — | 0.901 | ||
| 16 | — | — | 0.708 | ||
| 32 | — | — | 0.254 | ||
| triple | 2 | 0.246 (0.245–0.246) | — | 0.992 |
| type | S2 | S3a | S3b | S3c | S3d | S4 | |
|---|---|---|---|---|---|---|---|
| SUM | 2 | 0.192 / 0.45 | — / 0.45 | — / 0.45 | 0.217 / 0.45 | — / 0.60 | — / 0.45 |
| 4 | 0.102 / 0.20 | — / 0.25 | 0.039 / 0.25 | 0.125 / 0.20 | — / 0.35 | — / 0.20 | |
| 8 | 0.067 / 0.15 | — / 0.15 | 0.014 / 0.20 | — / 0.15 | — / 0.35 | — / 0.15 | |
| 16 | 0.043 / 0.075 | — / 0.15 | 0.009 / 0.90 | — / 0.075 | — / — | — / 0.075 | |
| 32 | — / 0.010 | 0.054 / — | 0.027 / — | — / 0.050 | — / — | — / 0.010 | |
| CONTRAST | 2 | 0.192 / 0.45 | — / 0.45 | — / 0.45 | 0.102 / 0.45 | — / 0.60 | — / 0.45 |
| basis | belief | evaluations | share undefined | CD closer than EC | |||
|---|---|---|---|---|---|---|---|
| pair | S2 | 6,098 | 68 % | 0.098 | 0.086 | 0.146 | 64 % |
| S3a | 3,963 | 44 % | 0.054 | 0.008 | 0.050 | 30 % | |
| S3b | 2,800 | 31 % | 0.067 | 0.011 | 0.057 | 25 % | |
| S3c | 5,751 | 64 % | 0.067 | 0.052 | 0.086 | 64 % | |
| S3d | 1,978 | 22 % | 0.046 | 0.001 | 0.041 | 25 % | |
| S4 | 6,098 | 68 % | 0.000 | 0.017 | 0.017 | 97 % |
| belief | type | ||||||
|---|---|---|---|---|---|---|---|
| S2 | SUM | 0.1 | 0.0025 | 0.0111 | 0.0131 | 0.0241 | 0.0106 |
| 0.2 | 0.0108 | 0.0076 | 0.0287 | 0.0358 | 0.0180 | ||
| 0.4 | 0.0736 | 0.0376 | 0.1002 | 0.0627 | 0.0265 | ||
| CONTRAST | 0.1 | 0.0025 | 0.0123 | 0.0129 | 0.0026 | 0.0154 | |
| 0.2 | 0.0108 | 0.0095 | 0.0267 | 0.0195 | 0.0374 | ||
| 0.4 | 0.0547 | 0.0238 | 0.0397 | 0.0617 | 0.0906 |
| type | basis | points | within | cell verdict | ||
|---|---|---|---|---|---|---|
| CONTRAST | pair | 33 | 1.59 | 0.040 | 0.52 | inconclusive |
| triple | 23 | 1.00 | 0.083 | 0.48 | not supported | |
| SUM | pair | 33 | 1.76 | 0.070 | 0.55 | inconclusive |
| triple | 23 | 0.88 | 0.107 | 1.00 | supported |
| type, basis | points | (H1c) | power law | closed form | ratio range | median ratio |
|---|---|---|---|---|---|---|
| SUM, pair | 33 | 1.76 | 55 % | 100 % | 0.61–1.65 | 0.88 |
| CONTRAST, pair | 33 | 1.59 | 52 % | 100 % | 0.61–1.65 | 0.87 |
| SUM, triple | 23 | 0.88 | 100 % | 100 % | 0.57–0.99 | 0.79 |
| CONTRAST, triple | 23 | 1.00 | 48 % | 100 % | 1.00–1.76 | 1.07 |
| pair | triple | |||||
|---|---|---|---|---|---|---|
| 2 | 0.545 | 0.437 | 0.398 | 0.545 | 0.437 | 0.398 |
| 4 | 0.261 | 0.204 | 0.168 | 0.199 | 0.094 | 0.070 |
| 8 | 0.158 | 0.129 | 0.116 | 0.087 | 0.045 | 0.035 |
| 16 | 0.090 | 0.074 | 0.068 | 0.041 | 0.023 | 0.018 |
| 32 | 0.049 | 0.039 | 0.038 | 0.021 | 0.011 | 0.009 |
| family | cells | min diff. | max diff. | CI inside | CI outside | median ratio |
|---|---|---|---|---|---|---|
| Clayton | 6 | 0.179 | +0.006 | 2 | 4 | 5.34 |
| Gumbel | 6 | 0.128 | 0.013 | 1 | 5 | 16.60 |
| 18 | 0.102 | +0.036 | 15 | 3 | 2.08 | |
| 18 | 0.201 | 0.016 | 3 | 15 | 7.22 |
| defined | drawn | within , pair | within , triple | ||
|---|---|---|---|---|---|
| 2 | 1 | 108 | 200 | 38 % | 49 % |
| 2 | 4 | 165 | 200 | 39 % | 24 % |
| variant | paired values | violations | median change | max change | worst ratio |
|---|---|---|---|---|---|
| 291,249 | 0 | ||||
| 291,258 | 5,460 | ||||
| -nodes | 291,276 | 0 | |||
| -nodes | 291,276 | 0 |
| at layer | H5a | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| belief | basis | ratio | above/below/all | ||||||
| S2 | pair | 0.0648 | 0.0211 | 0.0071 | 0.0024 | 0.0713 | 0.0829 | 0.05 | 1/9/12 |
| triple | 0.0653 | 0.0213 | 0.0071 | 0.0025 | 0.0713 | 0.0829 | 0.05 | 1/9/12 | |
| S3c | pair | 0.0516 | 0.0111 | 0.0043 | 0.0018 | 0.0642 | 0.0826 | 0.04 | 0/12/12 |
| triple | 0.0126 | 0.0071 | 0.0030 | 0.0015 | 0.0642 | 0.0826 | 0.10 | 0/12/12 | |
| S4 | pair | 0.0005 | 0.0010 | 0.0010 | 0.0011 | 0.0833 | 0.0833 | 1.91 | 8/0/12 |
| loss | optimiser | kept initial | coef. dist. | test loss | test loss down | ||
|---|---|---|---|---|---|---|---|
| value | Adam | 23 % | 0.132 | 0.3 % | 52 % | 0.005 | 0.47 |
| L-BFGS | 4 % | 0.131 | 1.1 % | 82 % | 0.000 | 0.34 | |
| NLL | Adam | 0 % | 0.649 | 10.9 % | 100 % | 0.055 | 7.35 |
| L-BFGS | 0 % | 0.660 | 11.1 % | 100 % | 0.043 | 8.06 | |
| end-to-end, | Adam | 3 % | 0.432 | 97.0 % | 96 % | 0.109 | 13.67 |
| L-BFGS | 5 % | 0.216 | 97.7 % | 94 % | 0.060 | 1.21 |
| test | what it checks | result |
|---|---|---|
| M1 | orthonormality of the basis, explicit | pass |
| M2 | index families, coefficient counts, block vs dense estimator | pass |
| M3 | estimator vs population moments, SE scaling, Gauss–Hermite convergence | pass |
| M4 | reproduces the TV values of the study | pass |
| M5 | flat- control: | pass |
| M6 | covariance identity in sample | pass |
| float32 | float64 | ||||
|---|---|---|---|---|---|
| layer | parameters | ms | J | ms | J |
| HCR-CD | 39,232 | ||||
| HCR-CAL | 39,232 | ||||
| MLP | 39,265 | ||||
| KAN | 39,528 | ||||
| batch | layer | params | ms | J | J net | W | MHz | counter | |
|---|---|---|---|---|---|---|---|---|---|
| 2 | 256 | HCR-CAL | 3,904 | 275 | 1,410 | 0.997 | |||
| HCR-CD | 3,904 | 61 | 1,095 | 1.017 | |||||
| KAN | 4,158 | 62 | 1,095 | 0.995 | |||||
| MLP | 3,883 | 62 | 1,095 | 0.997 | |||||
| 16,384 | HCR-CAL | 3,904 | 290 | 1,410 | 1.003 | ||||
| HCR-CD | 3,904 | 256 | 1,410 | 0.980 |
| batch | layer | params | ms | J | J net | W | MHz | counter | |
|---|---|---|---|---|---|---|---|---|---|
| 2 | 256 | HCR-CAL | 3,904 | 284 | 1,410 | 1.000 | |||
| HCR-CD | 3,904 | 63 | 1,095 | 0.980 | |||||
| KAN | 4,158 | 61 | 1,095 | 0.989 | |||||
| MLP | 3,883 | 63 | 1,095 | 0.989 | |||||
| 16,384 | HCR-CAL | 3,904 | 285 | 1,410 | 0.989 | ||||
| HCR-CD | 3,904 | 253 | 1,410 | 0.983 |
| experiment | runs | GPU-h | batch 1 | batch 2 | batch 3 | batch 4 | batch 5 |
|---|---|---|---|---|---|---|---|
| Gaussian sweep | 1,000 | 66.0 | 6 | 137 | 857 | — | — |
| population sweep | 100 | 11.6 | — | 1 | 99 | — | — |
| convergence study | 240 | 23.2 | — | — | 240 | — | — |
| copula families | 1,080 | 42.5 | — | — | 1 | 42 | 1,037 |
| LKJ matrices | 240 | 2.8 | — | 2 | 24 | 1 | 213 |
| chains | 240 | 30.6 | — | — | — | — | 240 |
Polynomial Neural Networks
Polynomial neural networks use polynomial activations or learned polynomial bases to model nonlinear functions for approximation, scientific computing, and efficient inference.
See all 25 papers