cs.LGNov 9, 2026

Evidence or Marginal? What Moment-Vector Propagation Computes in Polynomial Density Neurons

Authors: Przemysław Spyra

Organizations: AGH University of Krakow

Abstract

When a neuron's input is lost, or known through a belief qq, what should it output? For a neuron storing a joint polynomial density NN with input density DD, two formulas compete. pCD=Eq[N]/Eq[D]p_\mathrm{CD}=E_q[N]/E_q[D] costs one matrix-vector product; Hierarchical Correlation Reconstruction introduces it as a constant-denominator approximation to pEU=Eq[N/D]p_\mathrm{EU}=E_q[N/D]. For a density model both are exact answers to different questions: pCDp_\mathrm{CD} is Bayesian updating with the belief as virtual evidence; pEUp_\mathrm{EU} installs the belief as a new marginal (Jeffrey's rule). With a missing input pCDp_\mathrm{CD} returns the model's conditional; pEUp_\mathrm{EU} generally does not. This post hoc reading is proved, and illustrated outside the pair family up to k=8k=8 inputs; it has no pre-registered test, only one exploratory real-data pilot. Post hoc, closed forms fall within a factor 2 of all 112 measured gaps, and negative density begins near input correlation 1/k1/k. In a pre-registered synthetic study (up to 32 inputs), 5 of 15 hypotheses are supported, 2 inconclusive and 8 not supported, including all three on transfer beyond equicorrelated Gaussians. Post hoc, where pEUp_\mathrm{EU} is defined, truncation usually outweighs the gap for Beta beliefs in pair neurons; triple neurons under flat and data beliefs are exceptions.

Figures & tables

Appendix figures & tables38 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 18, 2026

Neural Networks with Differentiable Belief-Function Layers

Yager's 1987 rule of combination ⊞\boxplus was proposed on epistemic grounds: under high conflict, mass should be redistributed to ignorance rather than renormalised onto the class simplex. We observe that backpropagation needs the same rule: Dempster's normaliser (1−κ)−1(1{-}κ)^{-1}, where κκ is the conflict mass, puts a (1−κ)−2(1{-}κ)^{-2} term in the gradient that diverges exactly when evidence sources disagree, while Yager's ⊞\boxplus is bilinear and has no such term. Building on this, we construct a deep evidential pipeline in which every mass in the chain from heads to output is a valid Dempster--Shafer mass function, combined by Yager's rule. The output mass on the ignorance set, m(Θ)m(Θ), serves as the network's ``I do not know'' signal. The network is fit to hard labels and never sees an annotator distribution, yet m(Θ)m(Θ) orders human disagreement in two modalities, visual and linguistic. Dempster's rises too and ranks contested inputs about as well, but its normaliser divides the conflict out: Yager's vacuity is 9×9\times larger on the most contested CIFAR-10H images, so only Yager's is readable as an amount of ignorance without recalibration. For out-of-distribution detection it is competitive with strong single-pass baselines, while the clamped Dempster variant that scores higher becomes gradient-unstable once the clamp is removed.
Nov 5, 2026

Same Answer, Different Future: Dissociating Belief Content and Evidence Weight in Language Models

Memory summaries, prompts and steering vectors set a language model's beliefs and are judged by the prediction they produce. A prediction does not say how firmly it is held. We separate a belief's content, what it predicts now, from its weight, the number of new observations it is worth: for an ideal observer, the amount of evidence behind it. In Bernoulli inference we set beliefs in nine open-weight models of 8 to 397B parameters and follow each belief with the same new observations. We argue that these models hold an answer without its evidence and recover its weight from the text they re-read. Weight therefore follows the form of the evidence and the question asked more than its amount: matched on their current estimate, a record of observations and a summary of its counts usually move apart under the same new observations, in either direction. Scale does not close the gap. Across the nine models 64 observations are worth 20 to 53 as a record and 9 to 151 as a summary, no closer to 64 at 397B than at 8B. Where no text stands behind a belief there is little to re-read: a steering vector can install a record's answer, and sixteen observations undo it where a neutral passage leaves it in place. Where reasoning first writes the counts into the context the weight returns, and the same 64 observations are worth 58 to 67 in either form. A matched answer is not a matched belief, and methods that set beliefs should be judged by the futures they produce.