Fresh at Decision, Stale at Action: Routing New Information to LLM Decisions Already Made

Ayoub Merimi; Souhail Bakkali; Abdeljalil Sakat

cs.AIOct 13, 2026

Fresh at Decision, Stale at Action: Routing New Information to LLM Decisions Already Made

Authors: Ayoub Merimi, Souhail Bakkali, Abdeljalil Sakat

Abstract

Freshness shapes an LLM decision at the moment it is made; the action it commits to is latent, carried out later, in a changing world. We therefore introduce a timestamp theory of decision validity and a decision ledger that realises it. Six timestamps define a decision; a freshness ceiling bounds what up-to-date evidence guarantees at action time, placing the remaining risk in information published afterwards. Accuracy then factorises into two capabilities: routing, which delivers that information to affected decisions before their revision deadlines, and temporal judgment, which reads it for the time of action. The ledger is persistent state holding each decision's times, evidence and background knowledge, indexed by watch conditions written at decision time. Controlled decision streams with exact labels confirm four predictions across five LLMs, from 8B to frontier scale: freshness alone yields exactly the accuracy the ceiling predicts; models act on decisive news once it arrives, so accuracy follows routing; the ledger's index sets what routing reaches, and watch conditions reach indirect dependencies at low exposure; and the remaining errors concern when a change takes effect. Freshness makes a decision right when it is made; routing and temporal judgment keep it right when it is carried out.

Figures & tables

Panel of Figure 1 — Figure 1: A decision from the Indirect stream on its six timestamps. Retrieval at decision time shrinks the freshness gap $\Delta_{\mathrm{fresh}}$ . The notice that settles the decision is published inside the execution gap $\Delta_{\mathrm{act}}$ , is in effect at action time, and names only the substation that supplies the restaurant’s district; background knowledge in the ledger routes it to the decision before the revision deadline. Grey dots are other notices in the window.

Panel of Figure 2 — Figure 2: The decision ledger. Routing uses no model calls; the model is called again only for decisions that receive news. What the entry indexes sets what the router can reach.

	Direct	Stress	Indirect
Decisions	110	75	128
should switch	60	48	64
via name or stated dependency	60	48	32
via other name or upstream	0	0	32
should stand	50	27	64
News items	396	458	512

Table 1: The three conditions. A decision’s revision window holds dozens to hundreds of items, of which a handful concern it.

	Routing			Accuracy (%)
Router	Items	Rec.	Tok.	Claude	GPT	Qwen
None	0.0	–	0.0	36.0	36.0	36.0
Name match	3.5	56	1.2	73.3	72.0	73.3
BM25: decision	18.0	100	2.0	97.3	98.7	85.3
BM25: + time filter	14.3	100	1.8	97.3	98.7	92.0
Dense: decision	18.0	96	2.0	90.7	96.0	85.3

Table 2: Stress : final-decision accuracy (%) of Claude Sonnet 4.6, GPT-6.1 Sol and Qwen3-235B on the same 75 decisions. Items: new items delivered per decision; Rec.: share of decisive item–decision pairs delivered; Tok.: Claude Sonnet 4.6 input tokens per decision for the re-decision calls (thousands). Bold: best router without gold labels.

	Routing		Accuracy (%)
Router	Recall	Traps	Claude	Qwen-235B	Ministral 8B
None	–	0	50.0	50.0	50.0
Name match	25	0	62.5	57.0	52.3
BM25: decision	62	17	78.9	60.2	55.5
BM25: + dossier	100	89	93.0	61.7	54.7
Dense: + dossier	86	59	87.5	60.9	57.8

Table 3: Indirect : final-decision accuracy (%) on 128 balanced decisions. Recall: decisive notices delivered to their decision (%); Traps: notices delivered to a decision whose dossier mentions their entity in an irrelevant role (of 96). Claude writes the watch conditions for every model. Bold: best router without gold labels.

Figure 3: Decisive notices delivered on Indirect , by how the notice refers to the chosen option (16 decisions each; unlabelled bars reach 100%). The decision text reaches the option’s name and its stated dependency; the dossier and watch conditions extend the reach to other names and upstream entities.

Figure 4: The two factors of Eq. 3 on Indirect , one point per router. (a) Needed switches against routing recall: on the diagonal, the model switches exactly when decisive news arrives. (b) Decisions kept among those that should stand, against the items delivered to them; the points at two items are the related-news oracle.

	Direct	Indirect
Related news	Qwen3-8B	Ministral	Qwen-235B	Claude
starts after the activity	6	0.0	31.2	100.0
over before it starts	–	0.0	0.0	100.0
harmless	89	50.0	68.8	100.0
facility not needed	–	25.0	62.5	100.0
all that should stand	62.0	18.8	40.6	100.0

Table 4: Judgment in isolation (P4): decisions kept (%) when the oracle delivers exactly the related news and none of it changes the best option. On Indirect , each row except the last covers 16 decisions per model.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Router	Items	Decisive	Trap	Claude	Qwen	Ministral
	Delivered			Answer (gold: B)
None	0			A	A	A
Name match	0			A	A	A
BM25: decision	6			A	A	B
BM25: + dossier	5	✓	✓	B	B	B
Dense: + dossier	5			A	A	A

Appendix

Table 5: Decision D0011 under every router: notices delivered, whether they include the decisive notice and a trap, and each model’s answer (bold: correct).

Needed switches made, by how the notice refers to the option
	Name	BM25	+Doss.	+Watch	Oracle	All
option name	100.0	100.0	100.0	100.0	100.0	100.0
dependency	8.3	100.0	100.0	97.9	100.0	97.9
other name	6.2	54.2	95.8	97.9	97.9	93.8
upstream entity	4.2	35.4	93.8	75.0	100.0	89.6
Decisions kept, by the related news they received

Appendix

Table 6: Needed switches made and decisions kept on Indirect , by how the decisive notice refers to the option and by the related news a decision received (%, mean over the three models with complete answers; 16 decisions per row and model). Routing sets which switches are possible; judgment sets the rest.

		Claude		Qwen-235B		Ministral 8B		Pooled
Router A	Router B	W–L	$p$	W–L	$p$	W–L	$p$	W–L	$p$
BM25: decision	Name match	26–5	${<}.001$	30–26	.689	30–26	.689	86–57	.019
BM25: + dossier	BM25: decision	22–4	${<}.001$	17–15	.860	12–13	1.000	51–32	.048
BM25: + watch cond.	BM25: decision	21–5	.002	16–11	.442	15–7	.134	52–23	.001
BM25: + watch cond.	BM25: + dossier	5–7	.774	9–6	.607	12–3	.035	26–16	.164
Dense: watch cond.	Dense: + dossier	11–13	.839	12–16	.572	10–9	1.000	33–38	.635

Appendix

Table 7: Exact McNemar tests on Indirect . W–L: decisions router A gets right and router B gets wrong, and the reverse; 128 decisions per model, pooled over the three models with complete answers.

p

-values are two-sided and uncorrected.

Router	Recall	Implicit	Precision	Accuracy	Switch: named	Switch: dependency	Keep	Tokens (k)	$p$
	Routing (%)				Final decisions (%)
Qwen/Qwen3-8B
None (freshness at $t_{d}$ )	0.0	0.0	–	45.5	0.0	0.0	100.0	0.0	<0.001
Name match	39.7	0.0	100.0	70.0	100.0	18.2	88.0	23.9	0.851
Dense similarity	83.8	75.6	14.3	71.8	92.0	63.6	66.0	102.4	–
Watch conditions	77.9	65.9	14.2	62.7	80.0	57.6	56.0	146.9	0.076

Appendix

Table 8: Qwen3-8B on Direct . Recall : share of decision-changing items delivered to the decision they change; Implicit : the same for items that reach the option only through a dependency; Precision : share of delivered items that concern the chosen option. Switch: named / dependency : share of decisions that should switch and did, when the evidence names the option or reaches it only through a dependency; Keep : share of decisions that should stand and did. Tokens : prompt tokens spent on routing and re-deciding, including writing watch conditions.

p

: exact McNemar test on per-decision correctness against the most accurate practical router (bold; oracle excluded). Routing recall and precision for None, Name match, Oracle and All follow from the stream and are identical across models.

Related news	$n$	Oracle	Name	Sim.	Watch	All
Late change	18	94	28	39	67	33
Harmless only	19	11	5	37	37	32
None	13	0	0	23	23	31

Appendix

Table 9: Needless switches (%) on decisions that should stand, by the related news published in their revision window. Late change : a change to the option or its dependency that starts only after the activity ends; Harmless only : news about the option or its dependency that does not affect it. Name: name match; Sim.: similarity to the decision; Watch: watch conditions.

Router	Recall	Traps	Recall	Traps	Recall	Traps
	$K{=}1$		$K{=}3$		$K{=}5$
BM25: decision	51.6	4	62.5	17	68.8	35
BM25: + dossier	85.9	21	100.0	89	100.0	96
Dense: + dossier	70.3	11	85.9	59	90.6	74
BM25: + watch cond.	71.9	5	92.2	20	96.9	38
Dense: watch cond.	75.0	4	84.4	27	90.6	53

Appendix

Table 10: Indirect routing as the number

K

of decisions each notice is sent to varies: decisive pairs delivered (%) and lexical traps delivered.

Router	Accuracy	Keep	Needless
None	61.6	100.0	0
Name match	75.3	91.1	4
BM25: decision	89.0	91.1	4
BM25: + time filter	91.8	93.3	3
Dense: decision	91.8	93.3	3
Dense: watch cond.	89.0	86.7	6

Appendix

Table 11: Claude Sonnet 4.6 on a second Stress sample (73 decisions). Keep: accuracy on decisions that should stand (%); Needless: switches on those decisions. The table lists the routers run on this sample.

Explore similar work

StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams
Sep 29, 2026Jhen-Ke Lin, Chung Chun WangLLM Evaluation LLM Inference
National Yang Ming Chiao Tung University
Reading Too Much into Context: Passive Exposure Can Steer LLM Decisions
Sep 27, 2026Yuxiang Zheng, Lin Tian, Marian-Andrei RizoiuLLM Decision-Making LLM Reliability
Behavioral Data Science Lab University of Technology Sydney Sydney, Australia
StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
Oct 12, 2026Xialie Zhuang, Zunhai Su, Jia Pan +3LLM Decision-Making RL for Language Model Reasoning
StartLux · The University of Hong Kong · Southern University of Science and Technology

Sep 29, 2026J/K move · Enter open · S save

StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams

Jhen-Ke Lin, Chung Chun Wang

National Yang Ming Chiao Tung University

LLM Evaluation LLM Inference LLM Decision-Making

Open paper AskPDF arXiv

Language models increasingly make real-time decisions in applications that apply the latest answer until a newer one arrives. A late answer can prolong an outdated decision, such as a call recorder still running while a customer reads out card details, an error offline accuracy misses. We make three contributions. First, we release StreamDecisionBench (SDB), a dataset of eight streaming scenarios in four application families, with executable reference decisions derived from public rules. Second, we propose an evaluation protocol and a metric, in-force accuracy: the share of time the applied decision is correct across update intervals of 0.5-8 s. It reflects accuracy and latency jointly, attributing each error to judgment, latency or both. Third, we evaluate fifteen single-model settings, and this attribution separates speed-limited from judgment-limited models: slower, more accurate models lose 42-47% of the time to outdated answers, a fast model 35% to wrong ones. We therefore test hybrids in which a slow model corrects a fast one; with the right pairing and configuration, a hybrid outperforms every single model. However, even the best evaluated system keeps a correct decision in force only about two-thirds of the time, leaving a substantial gap for real-time use.

Abstract

Explore similar work

StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams

Reading Too Much into Context: Passive Exposure Can Steer LLM Decisions

StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions