cs.LGNov 5, 2026

Same Answer, Different Future: Dissociating Belief Content and Evidence Weight in Language Models

Authors: Weihan Li, Junhao Wu, Xinlei Chen, Tianshi Zheng

Organizations: The University of Tokyo · RWTH Aachen University · Harbin Institute of Technology, Shenzhen · The Hong Kong University of Science and Technology

Abstract

Memory summaries, prompts and steering vectors set a language model's beliefs and are judged by the prediction they produce. A prediction does not say how firmly it is held. We separate a belief's content, what it predicts now, from its weight, the number of new observations it is worth: for an ideal observer, the amount of evidence behind it. In Bernoulli inference we set beliefs in nine open-weight models of 8 to 397B parameters and follow each belief with the same new observations. We argue that these models hold an answer without its evidence and recover its weight from the text they re-read. Weight therefore follows the form of the evidence and the question asked more than its amount: matched on their current estimate, a record of observations and a summary of its counts usually move apart under the same new observations, in either direction. Scale does not close the gap. Across the nine models 64 observations are worth 20 to 53 as a record and 9 to 151 as a summary, no closer to 64 at 397B than at 8B. Where no text stands behind a belief there is little to re-read: a steering vector can install a record's answer, and sixteen observations undo it where a neutral passage leaves it in place. Where reasoning first writes the counts into the context the weight returns, and the same 64 observations are worth 58 to 67 in either form. A matched answer is not a matched belief, and methods that set beliefs should be judged by the futures they produce.

Figures & tables

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

    Jun 29, 2026Ankur Samanta, Akshayaa Magesh, Tal Lancewicki +7LLM EvaluationMulti-Turn Language Model Reasoning

  2. LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs

    May 7, 2026Chacha Chen, Matthew Jörke, Adam Goliński +4Belief RevisionLLM Auditing