When Should Models Change Their Minds? Contextual Belief Management in Large Language Models

Haoming Xu; Weihong Xu; Zongrui Li; Mengru Wang; Yunzhi Yao; Chiyu Wu; Jin Shang; Yu Gong

cs.AIMay 28, 2026

When Should Models Change Their Minds? Contextual Belief Management in Large Language Models

Authors: Haoming Xu, Weihong Xu, Zongrui Li, Mengru Wang, Yunzhi Yao, Chiyu Wu, Jin Shang, Yu Gong, +1 more

Organizations: ♠Zhejiang University · ♣HomologyAI

Abstract

Long-horizon interactions require language models to manage accumulating information: when to update their state, when to preserve their state, and what to ignore. We study this challenge as Contextual Belief Management (CBM): maintaining a predicted belief state aligned with formal evidence while isolating task-irrelevant noise. To make CBM measurable, we introduce BeliefTrack, a closed-world benchmark spanning Rule Discovery and Circuit Diagnosis, where a finite belief space and symbolic verifiers enable exact turn-level evaluation. BeliefTrack diagnoses three failures: Failed Stay, Failed Update, and Failed Isolation. Across multiple LLMs, vanilla models exhibit severe CBM failures, while explicit belief-tracking prompts provide limited gains. In contrast, reinforcement learning with belief-state rewards reduces failure rates by 70.9% on average. Further probing reveals latent belief-state dynamics behind these failures, and representation-level steering reduces failure rates by 46.1% across two tasks (Code is available at https://github.com/zjunlp/CBM).

Explore similar work

BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation
Jun 29, 2026Ankur Samanta, Akshayaa Magesh, Tal Lancewicki +7Initial Human Belief Distributions Large Language Model Reasoning
Large Language Models Develop Belief State Geometry In-Context
Sep 15, 2026Daniel Balcells, Andrew Jun Lee, Chirag Rastogi +3In-Context Learning In-Context
Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
May 12, 2026Eric Bigelow, Raphaël Sarfati, Daniel Wurgaft +5In-Context Learning In-Context

Jun 29, 2026 · cs.AIJ/K move · Enter open · S save

BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

Ankur Samanta, Akshayaa Magesh, Tal Lancewicki, Ayush Jain+6

Large language models (LLMs) are typically deployed in multi-turn conversations, where each turn provides new evidence that should reduce epistemic uncertainty about their environment. Acting rationally then requires inferring the unobserved quantities that govern it and updating beliefs about them as evidence accumulates. Yet most evaluations only score the model's final-turn answer in a single-turn format, leaving this process unexamined. We ask how closely LLMs' belief updates match those of a rational Bayesian reasoner in multi-turn settings, and introduce BayesBench, a suite of simulation environments that probe this across three progressively complex tasks: (i) Bayesian estimation, where the model infers an unknown parameter from sequential evidence; (ii) Bayesian prediction, where the model turns inferred beliefs about a latent variable into outcome forecasts; and (iii) latent-framed Bayesian prediction, where observations are filtered through a user-persona framing, requiring joint inference over the latent state and the persona. Across seven LLMs (3B--70B), scaling improves latent inference and evidence accumulation, with updates occasionally matching the Bayesian posterior. However, these gains do not reliably carry over to downstream prediction, exposing a gap between inferring latent structure and using it to rationally update beliefs about the target outcome.

Initial Human Belief Distributions Large Language Model Reasoning Bayesian Belief Multi-Turn Conversations

Open paper Ask about it PDF

Abstract

Explore similar work

BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

Large Language Models Develop Belief State Geometry In-Context

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space