Delayed Supervision for Test-Time Language Models

Jinha Kim; Taksh Kothari

cs.AISep 26, 2026

Delayed Supervision for Test-Time Language Models

Authors: Jinha Kim, Taksh Kothari

Organizations: MIT CSAIL

Abstract

Test-time language models adapt a compact memory while processing the input sequence. This perspective encompasses nonlinear fast-weight learning in LaCT, associative delta-rule updates in DeltaNet, and generalized delta-rule state updates in RWKV-7. Training these models to predict the next token does not explicitly require a fact to remain accessible after many subsequent memory updates. We study delayed supervision for this test-time memory: during post-training, ask a simulator-grounded question only after a long interval of unrelated events, and supervise its answer alongside ordinary next-token prediction. Questions are evaluated on disposable branches, so their answers never enter the continuing event stream. The construction distinguishes retention from revision: a retained fact must remain valid throughout the delay, whereas a revised fact must be answered with its latest value. We evaluate this approach on LaCT-760M and plain DeltaNet-1.3B using TextWorld training trajectories and shared BABILong and RULER evaluation panels, and include a separately reported RWKV-7 comparison. Relative to event-only training, delayed QA improves BABILong by 5.48 percentage points for LaCT and 1.32 points for DeltaNet, and single-needle RULER by 1.45 and 3.27 points, respectively. The RWKV-7 comparison reports gains of 4.60 and 7.00 points on its own panels. These results support delayed semantic supervision as a practical outer training objective for usable test-time memory, while leaving open how much of the benefit derives specifically from delay rather than general question-answering and answer-termination supervision.

Figures & tables

Figure 1: Delayed semantic supervision. The event stream updates the model normally. A disposable branch asks a question about an earlier, still-valid fact or the latest revision. Only the branch’s answer and EOS receive auxiliary cross-entropy targets. The main trajectory continues without seeing that answer. The displayed delay threshold describes the LaCT/DeltaNet construction; the RWKV-7 report uses a longer minimum interval.

Backbone	Training	BABILong	RULER single-needle
LaCT	Original	21.48	85.21
	NTP	25.60	87.39
	Delayed QA	31.08	88.85
DeltaNet	Original	18.68	80.36
	NTP	21.16	80.24
	Delayed QA	22.48	83.52

Table 1: Official benchmark scores (%). Bold denotes the delayed-QA method, not a significance claim. LaCT and DeltaNet share the 2,500-example BABILong and 825-example RULER panels. RWKV-7 uses separate 500/600-example panels and is author-reported; its original-model result is unavailable. Cross-backbone scores should not be read as a controlled ranking.

Figure 2: Original, event-NTP, and delayed-QA scores. The gain over NTP is positive on both reported benchmarks for each backbone. RWKV-7 (*) is a separately reported comparison on different length panels; its original-model result is unavailable.

Figure 3: BABILong by context length on the shared panel, with 500 examples per point. LaCT improves consistently over event NTP; DeltaNet’s gains are concentrated at the longer evaluated lengths. Curves show one trained checkpoint per condition, not averages over seeds.

Task	Length	Original	NTP	Delayed QA
Multi-value retrieval	2K	36.00	40.00	37.00
Variable tracking	2K	12.80	6.80	13.20
Common-word extraction	2K	19.20	13.40	12.80
Frequent-word extraction	2K	25.33	21.33	39.33
Multi-value retrieval	4K	18.50	20.00	14.50
Variable tracking	4K	18.00	8.00	22.40

Table 2: Complementary DeltaNet RULER panel: 50 examples per cell. Scores are official task metrics, including partial-target credit for synthetic tasks; they are not whole-response exact match. The HotpotQA-based task belongs to RULER, not LongBench.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	LaCT	DeltaNet	RWKV-7
Parameters	760M	1.3B	1.5B
World exposures	1,024	1,024	64
Effective batch (worlds)	8	8	4
Outer updates	128	128	16
QA branches / world	12	8	2
Event coefficient $\alpha$	0.5	1	1

Appendix

Table 3: Training settings for the reported QA arms. NTP controls use the same event exposure within each comparison and omit the auxiliary loss.

Benchmark	Length	NTP	Delayed QA
BABILong	8K	24.00	29.20
BABILong	16K	21.20	25.20
RULER single-needle	8K	84.33	93.33
RULER single-needle	16K	53.33	58.33

Appendix

Table 4: RWKV-7 official scores (%) from the supplied corrective report. Each BABILong row contains 250 examples and each RULER row 300.

Language Model Post-Training

Language model post-training adapts pretrained models to follow instructions, reason, use tools, and meet task-specific or human-defined goals.

See all 197 papers

Test-Time Training with Next-Token Prediction
Jun 19, 2026Xuan Ouyang, Zefan Cai, Junjie HuLong-Context Language Modeling Next-Token Prediction
University of Wisconsin–Madison
Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training
Oct 27, 2026Yue Wu, Yang Zhang, Weinan Gan +4Test-Time Training Inference-Time Adaptation of Language Models
University of Science and Technology of China China · National University of Singapore Singapore · Huawei Noah’s Ark Lab China
Engram Injection: A Study of Temporal Validity in Parametric Language-Model Memory
Oct 11, 2026Sam Siavoshian, Omar Ramadan, Amir K. Saeed +2Temporal Reasoning in Language Models Persistent Memory for Language Models

Jun 19, 2026J/K move · Enter open · S save

Test-Time Training with Next-Token Prediction

Xuan Ouyang, Zefan Cai, Junjie Hu

University of Wisconsin–Madison

Long-Context Language Modeling Next-Token Prediction Test-Time Training Sequence Modeling Inference-Time Adaptation of Language Models

Open paper AskPDF arXiv

Next-token prediction is the self-supervised signal that trains language models, and every observed prompt token provides the same signal at test time. We study whether this signal can define the inner-loop objective for test-time training (TTT) in pretrained long-context language models. Many TTT architectures require models to be trained with test-time adaptation in mind, limiting their direct applicability to released LLM checkpoints. While recent in-place TTT methods make fast-weight adaptation possible for pretrained LLMs without redesigning the backbone, they leave a central question unresolved: what should each test-time write store? Existing recipes train the fast weight to match a learned local value proxy but they are not directly tied to the self-supervised next-token prediction signal. We introduce Test-Time Training with Next-Token Prediction (TTT-NTP), a drop-in fast-weight adaptation method for pretrained LLMs that instead supervises updates using the model's own next contextual hidden state. This makes each local write follow the same causal computation that supports next-token prediction: the value target is a pointwise linear projection of a single next-position contextual state. On RULER Full-13, averaged over 4k to 32k contexts, TTT-NTP is the only method that consistently improves the released backbone across four models spanning three families and a 0.6-8B size range, by 3.9 points on Llama-3.1-8B, 3.0 on Mistral-7B-v0.3, 4.1 on Qwen3-4B, and 2.9 on Qwen3-0.6B. On the real-world LongBench-v2 long-document QA benchmark, TTT-NTP improves over the base model by 5.6 points on Llama-3.1-8B and 3.7 on Mistral-7B-v0.3, while preserving commonsense and knowledge performance. Our code is publicly available at https://github.com/yancyou/TTT-NTP.

Abstract

Language Model Post-Training

Explore similar work

Test-Time Training with Next-Token Prediction

Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training

Engram Injection: A Study of Temporal Validity in Parametric Language-Model Memory