StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue

Samarth Varshney; Ritu Yadav

cs.CLOct 24, 2026

StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue

Authors: Samarth Varshney, Ritu Yadav

Abstract

Users of conversational assistants rarely state all requirements up front: they add constraints, amend them, withdraw them, digress, and return. Instruction-tuned LLMs fail here in two ways: active constraints stop being honored as turns accumulate (persistence failure), and withdrawn constraints keep being enacted (relapse). The only existing repair we know of is an external ledger that re-injects a compiled specification every turn. We propose StateWise, a self-improving approach that teaches a single LLM to maintain its own revisable constraint state, a typed state with tombstones for revoked constraints, so persistence and relapse are measured by executable checkers and ground-truth states are compiled for free. StateWise fine-tunes the LLM on two complementary skills, summarizing a dialogue into its constraint state and responding conditioned on that state, then self-trains on its own trajectories, keeping only turns that pass an exact round-trip check against the compiled state. This teaches 7-8B models to track the state almost perfectly but, surprisingly, not to obey it: on Llama-3.1-8B every supervised variant roughly doubles relapse. We therefore add a preference stage on self-generated relapse contrasts, where the rejected response enacts a revoked rule and the chosen one passes the round-trip check. On two backbones with held-out constraint families and unseen phrasings, StateWise cuts relapse by 8-22 points relative to supervised self-training, lifts conversation success by 8-19 points over the backbone, and beats an oracle compiled-ledger baseline on relapse with no inference-time scaffolding. We also characterize the recipe's cost on general instruction-following benchmarks and study weight interpolation as a mitigation.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of StateWise. Stage I induces two complementary skills from scripted trajectories whose ground-truth states are compiled from the edit history. Stage II lets the model generate state and response on fresh trajectories and keeps only turns that pass an exact round-trip check. Stage III turns the same check into a preference signal: the model’s relapsed generations are rejected responses, its round-trip-clean generations are chosen responses. No proprietary model, judge or human annotation is used in training.

Arm (no training)	Persistence	Relapse	Conv. success	State match
Plain chat	0.793 [0.767, 0.820]	0.109 [0.080, 0.141]	0.275 [0.225, 0.333]	–
State-scratchpad prompt	0.587 [0.547, 0.629]	0.162 [0.125, 0.200]	0.088 [0.054, 0.125]	0.003 [0.001, 0.006]
Paired delta	$-0.206$ [-0.253, -0.160]	$+0.052$ [0.014, 0.090]	$-0.188$ [-0.250, -0.125]

Table 1: Inference-only diagnostic (Llama-3.1-8B-Instruct, 240 conversations). Asking the untrained model to maintain its own constraint state makes it worse on every metric.

		Held-out (300 conv., 6 unseen families)			In-distribution (240 conv.)
	Arm	Success	Relapse	Persist.	Success	Relapse	Persist.
none	A0 backbone	0.290 [0.240, 0.343]	0.130 [0.100, 0.165]	0.799	0.275 [0.221, 0.333]	0.110 [0.081, 0.141]	0.795
none	A1 single-turn constraint SFT	0.280 [0.230, 0.330]	0.164 [0.131, 0.197]	0.847	0.329 [0.271, 0.388]	0.148 [0.114, 0.183]	0.856
ledger	A6 oracle ledger incl. withdrawn rules	0.203 [0.160, 0.250]	0.380 [0.334, 0.424]	0.871	0.242 [0.188, 0.300]	0.333 [0.284, 0.383]	0.887
ledger	A6b oracle ledger, active rules only	0.390 [0.337, 0.443]	0.148 [0.117, 0.182]	0.877	0.463 [0.400, 0.525]	0.084 [0.058, 0.114]	0.890
self-training	A2 Stage I (state)	0.250 [0.203, 0.300]	0.254	0.841	0.325	0.281	0.926

Table 2: Main results on Llama-3.1-8B-Instruct (training seed 1, greedy decoding). Conversation-level 95% bootstrap intervals; A6 / A6b receive the compiled ground-truth state at inference and are oracle baselines. State exact-match for the state arms: A2 0.527 / 0.981 (held-out / in-dist.), A4 0.510 / 0.916, A7 0.500 / 0.893.

Setting	Arm	Success	Relapse	Persist.	Paired delta vs. A0 (success / relapse)
Llama-3.1-8B, seed 2, held-out	A0	0.250	0.124	0.788
	A6b oracle ledger	0.430	0.145	0.884
	A4 self-training	0.337	0.263	0.885
	A7 StateWise	0.443	0.097	0.860	$+0.193$ [0.133, 0.253] / $-0.026$ [-0.056, 0.004]
Qwen2.5-7B, held-out	A0	0.163	0.230	0.731
	A6b oracle ledger	0.260	0.244	0.865

Table 3: Replication of the main arms. Seed 2 repeats the Llama pipeline with a new training seed; the Qwen2.5-7B-Instruct pipeline uses Qwen as its own teacher; the paraphrase set uses author-written phrasings and new tasks. In-distribution: Appendix D .

		Held-out			In-distribution
Arm	Preference pairs	Success	Relapse	Persist.	Success	Relapse	Persist.
A4 (base for DPO)	none	0.327	0.246	0.853	0.350	0.258	0.937
A7 -shuf	A7 ’s 931 pairs, 50% labels swapped	0.320	0.242	0.849	0.379	0.240	0.936
A7 -act	active-violation pairs only (931)	0.317	0.284	0.881	0.412	0.308	0.979
A7 -rel	relapse pairs only (663)	0.343	0.142	0.827	0.463	0.102	0.896
A7 StateWise	relapse 663 + active 284	0.370	0.094	0.834	0.554	0.042	0.904

Table 4: Stage-III ablations (Llama-3.1-8B, same base, hyper-parameters and pair pool as A7 ; preregistered). Pair counts are before length filtering (Appendix B ).

	Held-out (ours)			IFEval		Multi-IF (prompt-strict)
Model	Success	Relapse	Persist.	strict	loose	T1	T2	T3
A0 backbone	0.290	0.130	0.799	0.725	0.771	0.735	0.557	0.400
A6b oracle ledger	0.390	0.148	0.877	–	–	–	–	–
A4 self-training	0.327	0.246	0.853	0.667	0.697	0.702	0.508	0.351
A7 StateWise ( $\alpha=1$ )	0.370	0.094	0.834	0.660	0.686	0.678	0.496	0.336
A7 , state-free inference	–	–	–	0.636	0.689	–	–	–

Table 5: Breadth. Multi-IF (909 English conversations) accumulates constraints without ever revoking them; IFEval (541 prompts) is single-turn.

\theta_{\alpha}=\theta_{\text{base}}+\alpha(\theta_{\textsc{A7}}-\theta_{\text{base}})

with no training; both

\alpha

values were fixed in advance.

Instruction Following

Instruction following studies how AI systems interpret and reliably carry out user goals, constraints, formats, and priorities.

See all 134 papers

Oct 22, 2026

When LLMs Forget What They Believe: Characterizing State Integrity Collapse in Long-Horizon Dialogues

Large language models can repeat an instruction while failing to keep obeying it. We study this recall-compliance dissociation in long-horizon dialogue and define State Integrity Collapse: a model preserves a stated constraint in memory but stops applying it to later turns. We introduce StateTrack-Bench, a controlled suite of conversations covering numerical, format, role/persona, factual-memory, and logical-rule constraints. Across seven provider-pinned model snapshots and five dialogue-depth probes, strict joint retention ranges from 62.4% to 100.0%, with degradation concentrated in symbolic constraints and in one especially brittle model. A forked recall audit shows that 88.6% of observed failures are behavioral-drift failures rather than ordinary forgetting; excluding the brittle outlier preserves this diagnosis (84.4% of failures). Extended evaluations at depths up to 50 turns confirm that the dissociation persists beyond the core benchmark, while Structured State Anchoring recovers up to 24.4 points on immutable constraints. The results show that reliability cannot be inferred from single-turn instruction following alone; evaluation must track whether remembered constraints remain operational as dialogue state grows. Code and artifacts are available at: https://github.com/xzzvsxd/StateTrack.

Yancheng Zhu, Yifan Yuan

School of Artificial Intelligence, Shenzhen University, Shenzhen, China

LLM Reliability Behavioral Consistency of LLM Agents Instruction Following Long-Context Reasoning in Language Models

Ask PDF

Aug 12, 2026

Dead text or binding clause? Measuring and restoring constraint influence in black-box LLM dialogues

Multi-turn dialogues let users revoke constraints as easily as impose them, but revocation does not reliably take effect: models keep enacting withdrawn requirements (occasionally beneath comments asserting their removal), a failure we call behavioral relapse, or revocation inertia. No existing instrument measures this influence per clause, predicts it before delivery, or repairs it under matched budgets. \sysname{} closes the three gaps through the model API alone: a contract ledger pairs every constraint with an executable checker, records revocations as tombstones, and compiles the net constraint state ahead of time into a single specification; a sequential ablation probe measures per-clause adherence and incremental behavioral effect; a repair ladder operates under token- and attempt-matched budgets. On \dataname{} (\NTasks{} HumanEval tasks, \NClauses{} verified checkers), relapse at an 8B operating point climbs from \ScaleDelayedMTwo{} to \ScaleDelayedMEight{} as constraint load grows, while stronger models sit at floor. Under matched checkers, model, and budget, ahead-of-time compilation significantly reduces relapse against a no-ledger verifier-retry baseline (\RestoreDiff{}, 95% CI \RestoreDiffCI{},

p

\RestoreDiffP{}); adaptive ladder interventions stacked on top add no detectable gain (95% confidence excludes gains

\geq

\LadderExcludedGain{}). The probe predicts relapse before delivery (AUROC \AurocPrimary{}); a one-sentence tombstone note recovers about a third of the compilation effect and survives a placebo control. At \CostDeliveryFactor{} delivery overhead and \CostTotalHedged{} of API compute for every result, revocation failure becomes a measurable, predictable, and repairable property of dialogue state rather than an invisible one.

Haoyuan Zhu

Department of Electronic and Electrical Engineering University of Sheffield

Instruction Following LLM Reliability Multi-Turn Dialogue Evaluation LLM Context Management

Ask PDF

Sep 27, 2026

Lost with a Map: Conversational State and Behavioral Reliability in Language Models

Task-oriented dialogue requires maintaining and updating information across turns, yet language models expose no explicit belief-state object. We study how conversational state is represented, updated, and used inside eight instruction-tuned language models from four families on MultiWOZ and SGD. Structure and values separate: which domains, slots, and requests are active is linearly readable just before the model acts, whereas exact values are far more readable where the user stated them. After a user changes a value, both values remain accessible at their mentions, and causal interventions show that both continue to influence the model's action. In natural closed-loop interaction, query failures separate into cases of weak structural support, incorrect value resolution, and failure to deploy otherwise-supported constraints, with targeted interventions producing systematically different repair behavior across these cases. These findings motivate a state-action controller that starts from the base model action and selectively edits it using structural readouts, without requiring a complete predicted belief state as an intermediate representation. On held-out MultiWOZ interaction across five models, it raises the base model exact-query accuracy from .318 to .621 and task success from .272 to .371 at negligible added cost. Overall, reliable interaction requires not only retaining conversational information, but resolving which available constraints currently apply and ensuring that they govern action.

Atahan Dokme, Larry Heck

Georgia Institute of Technology

Spoken Dialogue Systems LLM Reliability State Tracking Belief Updating in LLMs Task-Oriented Dialogue Systems

Ask PDF

Abstract

Instruction Following

Explore similar work

When LLMs Forget What They Believe: Characterizing State Integrity Collapse in Long-Horizon Dialogues

Dead text or binding clause? Measuring and restoring constraint influence in black-box LLM dialogues

Lost with a Map: Conversational State and Behavioral Reliability in Language Models