Users of conversational assistants rarely state all requirements up front: they add constraints, amend them, withdraw them, digress, and return. Instruction-tuned LLMs fail here in two ways: active constraints stop being honored as turns accumulate (persistence failure), and withdrawn constraints keep being enacted (relapse). The only existing repair we know of is an external ledger that re-injects a compiled specification every turn. We propose StateWise, a self-improving approach that teaches a single LLM to maintain its own revisable constraint state, a typed state with tombstones for revoked constraints, so persistence and relapse are measured by executable checkers and ground-truth states are compiled for free. StateWise fine-tunes the LLM on two complementary skills, summarizing a dialogue into its constraint state and responding conditioned on that state, then self-trains on its own trajectories, keeping only turns that pass an exact round-trip check against the compiled state. This teaches 7-8B models to track the state almost perfectly but, surprisingly, not to obey it: on Llama-3.1-8B every supervised variant roughly doubles relapse. We therefore add a preference stage on self-generated relapse contrasts, where the rejected response enacts a revoked rule and the chosen one passes the round-trip check. On two backbones with held-out constraint families and unseen phrasings, StateWise cuts relapse by 8-22 points relative to supervised self-training, lifts conversation success by 8-19 points over the backbone, and beats an oracle compiled-ledger baseline on relapse with no inference-time scaffolding. We also characterize the recipe's cost on general instruction-following benchmarks and study weight interpolation as a mitigation.
Figures & tables
Figure 1: Overview of StateWise. Stage I induces two complementary skills from scripted trajectories whose ground-truth states are compiled from the edit history. Stage II lets the model generate state and response on fresh trajectories and keeps only turns that pass an exact round-trip check. Stage III turns the same check into a preference signal: the model’s relapsed generations are rejected responses, its round-trip-clean generations are chosen responses. No proprietary model, judge or human annotation is used in training.
Arm (no training)
Persistence
Relapse
Conv. success
State match
Plain chat
0.793 [0.767, 0.820]
0.109 [0.080, 0.141]
0.275 [0.225, 0.333]
–
State-scratchpad prompt
0.587 [0.547, 0.629]
0.162 [0.125, 0.200]
0.088 [0.054, 0.125]
0.003 [0.001, 0.006]
Paired delta
−0.206 [-0.253, -0.160]
+0.052 [0.014, 0.090]
−0.188 [-0.250, -0.125]
Table 1: Inference-only diagnostic (Llama-3.1-8B-Instruct, 240 conversations). Asking the untrained model to maintain its own constraint state makes it worse on every metric.
Held-out (300 conv., 6 unseen families)
In-distribution (240 conv.)
Arm
Success
Relapse
Persist.
Success
Relapse
Persist.
none
A0 backbone
0.290 [0.240, 0.343]
0.130 [0.100, 0.165]
0.799
0.275 [0.221, 0.333]
0.110 [0.081, 0.141]
0.795
A1 single-turn constraint SFT
0.280 [0.230, 0.330]
0.164 [0.131, 0.197]
0.847
0.329 [0.271, 0.388]
0.148 [0.114, 0.183]
0.856
ledger
A6 oracle ledger incl. withdrawn rules
0.203 [0.160, 0.250]
0.380 [0.334, 0.424]
0.871
0.242 [0.188, 0.300]
0.333 [0.284, 0.383]
0.887
A6b oracle ledger, active rules only
0.390 [0.337, 0.443]
0.148 [0.117, 0.182]
0.877
0.463 [0.400, 0.525]
0.084 [0.058, 0.114]
0.890
self-training
A2 Stage I (state)
0.250 [0.203, 0.300]
0.254
0.841
0.325
0.281
0.926
Table 2: Main results on Llama-3.1-8B-Instruct (training seed 1, greedy decoding). Conversation-level 95% bootstrap intervals; A6 / A6b receive the compiled ground-truth state at inference and are oracle baselines. State exact-match for the state arms: A2 0.527 / 0.981 (held-out / in-dist.), A4 0.510 / 0.916, A7 0.500 / 0.893.
Setting
Arm
Success
Relapse
Persist.
Paired delta vs. A0 (success / relapse)
Llama-3.1-8B, seed 2, held-out
A0
0.250
0.124
0.788
A6b oracle ledger
0.430
0.145
0.884
A4 self-training
0.337
0.263
0.885
A7 StateWise
0.443
0.097
0.860
+0.193 [0.133, 0.253] / −0.026 [-0.056, 0.004]
Qwen2.5-7B, held-out
A0
0.163
0.230
0.731
A6b oracle ledger
0.260
0.244
0.865
Table 3: Replication of the main arms. Seed 2 repeats the Llama pipeline with a new training seed; the Qwen2.5-7B-Instruct pipeline uses Qwen as its own teacher; the paraphrase set uses author-written phrasings and new tasks. In-distribution: Appendix D .
Held-out
In-distribution
Arm
Preference pairs
Success
Relapse
Persist.
Success
Relapse
Persist.
A4 (base for DPO)
none
0.327
0.246
0.853
0.350
0.258
0.937
A7 -shuf
A7 ’s 931 pairs, 50% labels swapped
0.320
0.242
0.849
0.379
0.240
0.936
A7 -act
active-violation pairs only (931)
0.317
0.284
0.881
0.412
0.308
0.979
A7 -rel
relapse pairs only (663)
0.343
0.142
0.827
0.463
0.102
0.896
A7 StateWise
relapse 663 + active 284
0.370
0.094
0.834
0.554
0.042
0.904
Table 4: Stage-III ablations (Llama-3.1-8B, same base, hyper-parameters and pair pool as A7 ; preregistered). Pair counts are before length filtering (Appendix B ).
Held-out (ours)
IFEval
Multi-IF (prompt-strict)
Model
Success
Relapse
Persist.
strict
loose
T1
T2
T3
A0 backbone
0.290
0.130
0.799
0.725
0.771
0.735
0.557
0.400
A6b oracle ledger
0.390
0.148
0.877
–
–
–
–
–
A4 self-training
0.327
0.246
0.853
0.667
0.697
0.702
0.508
0.351
A7 StateWise ( α=1 )
0.370
0.094
0.834
0.660
0.686
0.678
0.496
0.336
A7 , state-free inference
–
–
–
0.636
0.689
–
–
–
Table 5: Breadth. Multi-IF (909 English conversations) accumulates constraints without ever revoking them; IFEval (541 prompts) is single-turn. θα=θbase+α(θ\textscA7−θbase) with no training; both α values were fixed in advance.