When LLMs Forget What They Believe: Characterizing State Integrity Collapse in Long-Horizon Dialogues

Yancheng Zhu; Yifan Yuan

cs.CLOct 22, 2026

When LLMs Forget What They Believe: Characterizing State Integrity Collapse in Long-Horizon Dialogues

Authors: Yancheng Zhu, Yifan Yuan

Organizations: School of Artificial Intelligence, Shenzhen University, Shenzhen, China

Abstract

Large language models can repeat an instruction while failing to keep obeying it. We study this recall-compliance dissociation in long-horizon dialogue and define State Integrity Collapse: a model preserves a stated constraint in memory but stops applying it to later turns. We introduce StateTrack-Bench, a controlled suite of conversations covering numerical, format, role/persona, factual-memory, and logical-rule constraints. Across seven provider-pinned model snapshots and five dialogue-depth probes, strict joint retention ranges from 62.4% to 100.0%, with degradation concentrated in symbolic constraints and in one especially brittle model. A forked recall audit shows that 88.6% of observed failures are behavioral-drift failures rather than ordinary forgetting; excluding the brittle outlier preserves this diagnosis (84.4% of failures). Extended evaluations at depths up to 50 turns confirm that the dissociation persists beyond the core benchmark, while Structured State Anchoring recovers up to 24.4 points on immutable constraints. The results show that reliability cannot be inferred from single-turn instruction following alone; evaluation must track whether remembered constraints remain operational as dialogue state grows. Code and artifacts are available at: https://github.com/xzzvsxd/StateTrack.

Figures & tables

Figure 1: Overview of the StateTrack-Bench evaluation protocol. Each dialogue passes through constraint injection, distractor-phase expansion, and a forked evaluation branch that independently measures recall fidelity and behavioral compliance across five constraint categories and five turn depths.

Reported model	Logged endpoint/snapshot id
GPT-5.4	gpt-5.4-0429
GPT-5.4-mini	gpt-5.4-mini-0429
Claude-Sonnet-4-6	claude-sonnet-4-6-20250401
Gemini-3.1-Pro	gemini-3.1-pro-preview-0425
DeepSeek-v4-Pro	deepseek-v4-pro-20250415
DeepSeek-v4-Flash	deepseek-v4-flash-20250415

Table 1: Logged model snapshot identifiers used for the primary benchmark runs. Closed-provider names may drift in future releases; these strings are the reproducibility handles stored with our model logs.

Turn	3	5	7	10	12
Strict joint retention	90.9	91.9	90.7	91.4	90.0

Table 2: Average strict joint retention by turn depth across all constraint categories.

Figure 2: Average state-retention rate by model and constraint type. Darker cells indicate lower retention.

Model	Overall	Turn 3	Turn 12	Drop
DeepSeek-v4-Flash	100.0	100.0	100.0	0.0
GPT-5.4-mini	98.4	96.0	100.0	-4.0
DeepSeek-v4-Pro	96.0	92.0	96.0	-4.0
Qwen3.6-Plus	95.2	92.0	96.0	-4.0
GPT-5.4	94.4	100.0	96.0	4.0
Claude-Sonnet-4-6	90.4	92.0	88.0	4.0

Table 3: Strict joint retention by model. Overall is the mean over five constraint categories and five turn depths. Early and late columns average all categories at turns 3 and 12 respectively.

Model	Num.	Format	Logic	Factual	Persona
GPT-5.4	92.0	90.4	88.8	100.0	100.0
GPT-5.4-mini	88.0	84.0	82.4	96.0	94.4
DeepSeek-v4-Flash	100.0	100.0	100.0	100.0	100.0
DeepSeek-v4-Pro	94.4	92.8	91.2	100.0	100.0
Gemini-3.1-Pro	56.0	52.8	48.0	76.8	78.4
Claude-Sonnet-4-6	90.4	88.0	86.4	96.8	96.0

Table 4: Model-by-category strict joint retention (%) averaged across the five dialogue-depth probes. This table is the primary category result; Figure 2 is only a visual summary.

Figure 3: Retention by constraint type across models. Symbolic constraints create the most visible spread even when factual memory remains comparatively stable.

Model	Turn 3	Turn 5	Turn 7	Turn 10	Turn 12
DeepSeek-v4-Flash	100.0	100.0	100.0	100.0	100.0
GPT-5.4-mini	96.0	100.0	100.0	96.0	100.0
DeepSeek-v4-Pro	92.0	100.0	96.0	96.0	96.0
Qwen3.6-Plus	92.0	96.0	96.0	96.0	96.0
GPT-5.4	100.0	92.0	92.0	92.0	96.0
Claude-Sonnet-4-6	92.0	92.0	88.0	92.0	88.0

Table 5: Turn-level retention rates by model and dialogue depth. Scores are percentages averaged over all constraint categories.

Figure 4: Turn-level retention curves across dialogue depth. The main signal is not monotonic collapse, but model-dependent stability versus late-turn drift.

Model	Failures	IF	MF	IF Rate	$\operatorname{SIC}^{+}$
Gemini-3.1-Pro	47	43	4	91.5	34.4
Claude-Sonnet-4-6	12	10	2	83.3	8.0
GPT-5.4	7	6	1	85.7	4.8
GPT-5.4-mini	2	2	0	100.0	1.6
DeepSeek-v4-Flash	0	0	0	—	0.0
DeepSeek-v4-Pro	5	4	1	80.0	3.2

Table 6: Failure-side SIC decomposition across the primary test set. IF = integrity failure (recalled but violated), MF = memory failure (unable to recall).

\operatorname{SIC}^{+}

is normalized by all 125 probes per model; IF rate is conditioned on observed failures.

Model	Collapsed Scen.	Full Recovery (%)	Partial Recovery (%)	No Recovery (%)
Gemini-3.1-Pro	47	53.2	80.9	19.1
Claude-Sonnet-4-6	12	75.0	91.7	8.3
GPT-5.4	7	71.4	85.7	14.3
GPT-5.4-Mini	2	100.0	100.0	0.0
DeepSeek-v4-Pro	5	80.0	100.0	0.0
Qwen3.6-Plus	6	66.7	83.3	16.7

Table 7: Reinstatement intervention outcomes. Full

\subset

Partial (cumulative); No Recovery = 100%

-

Partial. Rates are computed over the set of scenarios in which collapse was detected.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Scenarios	Max Turn	Avg. Tok.	Distr. Jaccard
Numerical Constraints	25	12	28.4	0.042
Role Personas	25	12	74.1	0.038
Format Rules	25	12	51.7	0.035
Factual Memory	25	12	63.2	0.040
Logical Rules	25	12	59.8	0.044
Total / Average	125	12	55.4	0.040

Appendix

Table 8: StateTrack-Bench dataset statistics.

Model	Retention	95% CI
DeepSeek-v4-Flash	100.0	[100.0, 100.0]
GPT-5.4-mini	98.4	[96.0, 100.0]
DeepSeek-v4-Pro	96.0	[92.8, 98.4]
Qwen3.6-Plus	95.2	[92.0, 97.6]
GPT-5.4	94.4	[91.2, 96.8]
Claude-Sonnet-4-6	90.4	[86.4, 93.6]

Appendix

Table 9: 95% bootstrap confidence intervals for overall retention.

Model	Num.	Format	Logic	Factual	Persona
GPT-5.4	92.0	90.4	88.8	100.0	100.0
GPT-5.4-mini	88.0	84.0	82.4	96.0	94.4
DeepSeek-v4-Flash	100.0	100.0	100.0	100.0	100.0
DeepSeek-v4-Pro	94.4	92.8	91.2	100.0	100.0
Gemini-3.1-Pro	56.0	52.8	48.0	76.8	78.4
Claude-Sonnet-4-6	90.4	88.0	86.4	96.8	96.0

Appendix

Table 10: Per-category retention rates (%) averaged across all turn depths.

Model	$\tau=0.0$	$\tau=0.3$	$\tau=0.7$
GPT-5.4	94.4	93.8	92.6
Gemini-3.1-Pro	62.4	61.2	60.8
DeepSeek-v4-Pro	96.0	95.4	94.2

Appendix

Table 11: Retention rates (%) under varying decoding temperatures on a stratified 50-scenario subset.

Model	Turn 12	Turn 20	Turn 30	Turn 50
DeepSeek-v4-Flash	100.0	100.0	97.5	95.0
GPT-5.4	96.0	92.5	87.5	82.5
Gemini-3.1-Pro	54.0	47.5	40.0	32.5

Appendix

Table 12: Retention (%) at extended dialogue depths.

Model	English	Chinese	Japanese
GPT-5.4	93.3	90.0	86.7
Claude-Sonnet-4-6	90.0	86.7	83.3
Gemini-3.1-Pro	63.3	56.7	53.3

Appendix

Table 13: Cross-lingual retention (%) on a 30-scenario subset.

Model	Baseline	+Reinstate	+SSA	$\Delta$ vs. Base
GPT-5.4	96.0	97.6	99.2	+3.2
Claude-Sonnet-4-6	88.0	92.0	96.0	+8.0
Gemini-3.1-Pro	54.0	65.6	78.4	+24.4
Qwen3.6-Plus	96.0	97.6	98.4	+2.4

Appendix

Table 14: Retention (%) with SSA vs. baseline at depth 12.

Model	$n$	Task	Constraint	Strict joint	Recall	SIC +	SIC -
GPT-5.4	10	80.0	100.0	80.0	100.0	0.0	0.0
GPT-5.4-mini	10	80.0	90.0	70.0	100.0	10.0	0.0
Claude-Sonnet-4-6	10	80.0	60.0	40.0	10.0	10.0	60.0
Gemini-3.1-Pro	10	40.0	30.0	20.0	0.0	0.0	30.0
DeepSeek-v4-Pro	10	60.0	90.0	50.0	90.0	10.0	10.0
DeepSeek-v4-Flash	10	50.0	100.0	50.0	100.0	0.0	0.0

Appendix

Table 15: Compact strict-joint versus constraint-only audit on controlled multi-turn histories (10 probes/model). Immediate-task success is judged while ignoring the persistent constraint; constraint-only uses the scenario checker; strict joint requires both.

Category	$n$	Constraint	Recall	SIC +	SIC -
Numerical	14	71.4	71.4	7.1	7.1
Format	14	78.6	78.6	7.1	7.1
Persona	14	71.4	71.4	7.1	7.1
Factual	14	92.9	64.3	0.0	28.6
Logical	14	92.9	71.4	0.0	21.4

Appendix

Table 16: SIC components by constraint category in the compact forked audit. Values are percentages over 14 probes per category (2 scenarios

\times

7 models).

Model	C@12	C@30	C@50	$\Delta$	SIC + @50
GPT-5.4	91.7	100.0	91.7	0.0	8.3
GPT-5.4-mini	91.7	83.3	91.7	0.0	8.3
Claude-Sonnet-4-6	91.7	100.0	91.7	0.0	8.3
DeepSeek-v4-Pro	100.0	91.7	83.3	-16.7	16.7
DeepSeek-v4-Flash	91.7	100.0	83.3	-8.4	16.7
Qwen3.6-Plus	100.0	100.0	100.0	0.0	0.0

Appendix

Table 17: Gemini-excluded symbolic-depth diagnostic audit. C@d is deterministic constraint compliance at depth

d

;

\Delta

is C@50–C@12.

Condition	$n$	Stale enforced	Stale %	Updated/clean %
Base after change	35	1	2.9	94.3
Stale SSA ledger	35	26	74.3	20.0
Updated-ledger SSA	35	0	0.0	97.1

Appendix

Table 18: Dynamic revoke/update stress test for SSA. Stale enforced counts outputs that incorrectly follow the old constraint after it was revoked or updated. Base uses only the dialogue history; stale SSA intentionally injects the old active state; updated-ledger SSA injects the revised active state.

Model	$n$	C1	C2	Joint	Recall	SIC +
GPT-5.4	4	100.0	100.0	100.0	100.0	0.0
GPT-5.4-mini	5	100.0	100.0	100.0	100.0	0.0
Claude-Sonnet-4-6	5	80.0	100.0	80.0	60.0	0.0
Gemini-3.1-Pro	5	20.0	0.0	0.0	0.0	0.0
DeepSeek-v4-Pro	5	100.0	80.0	80.0	60.0	20.0
DeepSeek-v4-Flash	5	80.0	100.0	80.0	100.0	20.0

Appendix

Table 19: Compact two-constraint diagnostic audit. C1/C2 are the two deterministic constraint checks; Joint requires both constraints; Recall is the forked recall probe.

Judge	Agreement w/ Primary	$\kappa$ (pairwise)	Self-Pref. $\Delta$	McNemar $p$	Bias?
GPT-5.4 (primary)	—	—	+1.2%	—	—
Claude-Sonnet-4-6	94.0%	0.87	+0.8%	0.42	No
Gemini-3.1-Pro	91.5%	0.82	$-$ 0.4%	0.68	No
Fleiss’ $\kappa$ (3 judges)	0.89

Appendix

Table 20: Multi-judge agreement and self-preference analysis on 200-output subset.

Model	Base Lat.	SSA Lat.	Lat. $\times$	Base Tok.	SSA Tok.	Tok. $\times$
GPT-5.4	2.1s	2.8s	1.34 $\times$	1,842	2,174	1.18 $\times$
Claude-Sonnet-4-6	2.4s	3.6s	1.50 $\times$	1,956	2,487	1.27 $\times$
Gemini-3.1-Pro	1.8s	3.8s	2.12 $\times$	1,724	2,534	1.47 $\times$
Qwen3.6-Plus	1.9s	2.7s	1.42 $\times$	1,801	2,198	1.22 $\times$

Appendix

Table 21: SSA inference overhead at depth 12. Latency is wall-clock time per response; token cost includes input+output tokens for all retries.

Model	Original	Variant A	Variant B	Variant C
GPT-5.4	94.0	93.2	94.8	92.4
Claude-Sonnet-4-6	90.0	88.8	91.2	89.6
Gemini-3.1-Pro	62.0	60.4	63.6	59.2

Appendix

Table 22: Retention (%) under alternative prompt formulations (50-scenario subset).

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 780 papers

StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue
Oct 24, 2026Samarth Varshney, Ritu YadavInstruction Following Multi-Turn Dialogue Management
Did You Forget What I Asked? Prospective Memory Failures in Large Language Models
Mar 7, 2026Avni MittalInstruction Following LLM Reliability
Quantifying Conversational Reliability of Large Language Models under Multi-Turn Interaction
Mar 2, 2026Jiyoon MyungTool Selection for Language Model Agents LLM Reliability
Samsung SDS

Oct 24, 2026J/K move · Enter open · S save

StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue

Samarth Varshney, Ritu Yadav

Instruction Following Multi-Turn Dialogue Management Language Model Self-Improvement Self-Training for Language Models

Open paper AskPDF arXiv

Users of conversational assistants rarely state all requirements up front: they add constraints, amend them, withdraw them, digress, and return. Instruction-tuned LLMs fail here in two ways: active constraints stop being honored as turns accumulate (persistence failure), and withdrawn constraints keep being enacted (relapse). The only existing repair we know of is an external ledger that re-injects a compiled specification every turn. We propose StateWise, a self-improving approach that teaches a single LLM to maintain its own revisable constraint state, a typed state with tombstones for revoked constraints, so persistence and relapse are measured by executable checkers and ground-truth states are compiled for free. StateWise fine-tunes the LLM on two complementary skills, summarizing a dialogue into its constraint state and responding conditioned on that state, then self-trains on its own trajectories, keeping only turns that pass an exact round-trip check against the compiled state. This teaches 7-8B models to track the state almost perfectly but, surprisingly, not to obey it: on Llama-3.1-8B every supervised variant roughly doubles relapse. We therefore add a preference stage on self-generated relapse contrasts, where the rejected response enacts a revoked rule and the chosen one passes the round-trip check. On two backbones with held-out constraint families and unseen phrasings, StateWise cuts relapse by 8-22 points relative to supervised self-training, lifts conversation success by 8-19 points over the backbone, and beats an oracle compiled-ledger baseline on relapse with no inference-time scaffolding. We also characterize the recipe's cost on general instruction-following benchmarks and study weight interpolation as a mitigation.

Abstract

LLM Reliability

Explore similar work

StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue

Did You Forget What I Asked? Prospective Memory Failures in Large Language Models

Quantifying Conversational Reliability of Large Language Models under Multi-Turn Interaction