Collection-Date Assignment Redirects Answers in Temporal QA

Minsu Oh; Seunghui Jwa; Soo Kyun Kim; Yeo-Chan Yoon

cs.CLOct 23, 2026

Collection-Date Assignment Redirects Answers in Temporal QA

Authors: Minsu Oh, Seunghui Jwa, Soo Kyun Kim, Yeo-Chan Yoon

Organizations: Department of Artificial Intelligence, Jeju National University · Department of Computer Engineering, Jeju National University

Abstract

A collection date records when a source revision was acquired, not when its contents changed. Can assigning this metadata redirect temporal question answering despite unchanged evidence? Swapping the question day and following day between two revisions lowers later-reference exact match (EM) by 37.6-51.4 percentage points across five open-weight LLMs on 595 Wikipedia pages, replicated on 128 disjoint pages. Previous-day swaps retain 29.0-39.6-point losses on 497 pages. Seven prompt variants, including explicit ignore-collection instructions and archived-revision provenance, retain the effect. Fictional answers and English month-name question dates show that remembered answers and identical date strings are unnecessary. Matched thinking reduces Qwen3.5's penalty from 30.1 to 11.1 points, while native reasoners vary. Two API LLMs show small Wikipedia swap effects and rarely shift to earlier answers on fictional records. Large swap-induced losses concentrate around the question date; swaps between distant dates have much smaller effects. Adding fields containing one observed acquisition date lowers EM relative to omission in three of five LLMs after correction. The diagnostic isolates metadata effects from changing evidence or missing the relevant revision, and provides a practical audit for temporal QA inputs.

Figures & tables

Figure 1: Evidence-fixed collection-date interventions. (a) Texts, revision timestamps and the question remain fixed. (b) For a question about

t_{1}

, collection dates

r_{0},r_{1}

can match

t_{1}

for the older revision, the newer revision, or both;

+1

means one day later. (c) Exact match and answer mentions separate selection from formatting; A/B answer selection probes shared dates, not swaps. Missing-value, absent-field and nondate controls test alternative explanations. The schematic is illustrative, not a result plot.

Question	Which team does Jordan Harris (ice hockey) play for?
	Older revision	Newer revision
Native revision	2024-08-30T01:23:19Z	2025-07-08T21:59:15Z
Recorded team	Columbus Blue Jackets	Boston Bruins
Question date ( $t_{0},t_{1}$ )	2024-09-01	2025-07-16
Collection date: question-matched	2024-09-01 ( $r_{0}=t_{0}$ )	2025-07-16 ( $r_{1}=t_{1}$ )
Collection date: shared question date	2025-07-16 ( $r_{0}=t_{1}$ )	2025-07-16 ( $r_{1}=t_{1}$ )

Table 1: Paired-revision inputs. Collection-date rows are assigned controls, not observed logs. The last two rows exchange the same dates between revisions, the headline manipulation. Question, team text and revision timestamps stay fixed. This is the original 595-page cohort’s shortest team-field pair by combined retrieved length, selected without LLM outputs. Official August 2024 and July 2025 transaction notices corroborate the team changes; they are not extra model inputs. Experiments use both complete snippets, with both source orders.

LLM	Earlier-reference EM ( $t_{0}$ )	Earlier answer only ( $t_{0}$ )	Later-reference EM ( $t_{1}$ )
595 pages, 113 predicates, raw display
Qwen2.5-7B	+38.7 [+29.9,+47.0]	+41.9 [+34.8,+48.9]	-40.1 [-49.4,-30.6]
Phi-4	+45.1 [+31.6,+58.3]	+65.2 [+56.8,+73.5]	-47.3 [-62.2,-32.0]
Mistral-24B	+49.8 [+39.0,+60.2]	+51.3 [+40.9,+61.2]	-51.4 [-62.6,-39.9]
Qwen3.5-35B	+37.6 [+30.6,+44.4]	+39.5 [+33.5,+45.2]	-37.6 [-44.1,-30.9]
Qwen2.5-72B	+44.9 [+34.0,+57.4]	+48.3 [+39.2,+59.4]	-48.7 [-60.2,-36.9]

Table 2: Question-date matching swap,

(o{=}0,n{=}1)-(o{=}1,n{=}0)

, for

t_{1}

questions. Values are percentage-point changes against

t_{0}

t_{1}

references with nominal 95% predicate-cluster intervals. The 128 pages are from the separate 596-page cohort, disjoint from the 595 pages. Scores average both revision orders per page. Earlier-answer-only inclusion uses the same text normalization as EM in both cohorts; follow-ups additionally use the Unicode word-matching rule described in Appendix A . All EM levels and proximity intervals: Appendix C .

Figure 2: Later-reference EM ( $t_{1}$ , %) on 595 Wikipedia pages. Older/newer revision collection offsets are days after question date $t_{1}$ , averaged over both source orders. Compare $(0,1)$ with $(1,0)$ for question-date matching and $(1,7)$ with $(7,1)$ for proximity without an exact date match. Texts and revision timestamps remain fixed. Q-match supplies an informative positive control; omit removes collection metadata.

LLM	After: later EM	Before: later EM	Before: earlier EM	Before $-$ after
Qwen2.5-7B	-40.4 [-50.6,-30.1]	-39.0 [-48.2,-29.5]	+37.1 [+27.9,+46.3]	+1.4 [-1.3,+4.5]
Phi-4	-46.9 [-61.5,-31.6]	-29.0 [-38.4,-19.1]	+22.5 [+14.2,+30.9]	+17.9 [+11.7,+24.2]
Mistral-24B	-52.0 [-63.8,-40.3]	-39.6 [-51.6,-27.7]	+32.4 [+22.5,+42.6]	+12.4 [+8.7,+15.6]
Qwen3.5-35B	-38.5 [-45.7,-31.4]	-34.3 [-42.6,-26.0]	+33.6 [+24.8,+42.5]	+4.2 [+0.7,+7.2]
Qwen2.5-72B	-49.6 [-61.9,-36.9]	-32.2 [-40.8,-23.6]	+29.6 [+21.4,+39.0]	+17.4 [+12.4,+23.3]

Table 3: Changes within one execution for questions about

t_{1}

on 497 eligible Wikipedia pages (103 predicate groups). After is

(0,+1)-(+1,0)

; before is

(0,-1)-(-1,0)

, always older-matching minus newer-matching assignment. Later/earlier EM uses the

t_{1}

t_{0}

reference. The last column compares later-reference EM effects. Units are percentage points; brackets give nominal 95% predicate-cluster intervals. Full levels: Appendix D .

Prompt contrast	Qwen2.5-7B	Phi-4	Mistral	Qwen3.5-35B	Qwen2.5-72B
Original	-37.9 / +36.3 [-48.1,-28.0]	-42.6 / +40.2 [-57.3,-27.9]	-47.7 / +45.7 [-63.1,-33.5]	-39.5 / +39.5 [-50.6,-28.4]	-52.7 / +47.3 [-67.0,-38.6]
Definition 1	-41.0 / +38.3 [-51.3,-30.1]	-46.9 / +45.3 [-61.4,-32.0]	-54.7 / +48.0 [-68.0,-41.5]	-44.9 / +44.1 [-57.2,-32.5]	-57.4 / +52.0 [-71.3,-42.6]
Definition 2	-41.8 / +38.3 [-51.9,-30.5]	-46.5 / +44.1 [-60.4,-31.8]	-44.5 / +43.4 [-59.0,-31.4]	-42.2 / +42.2 [-53.8,-30.5]	-54.7 / +50.4 [-70.6,-39.7]
Ignore collection	-42.2 / +39.5 [-52.2,-30.9]	-43.0 / +41.0 [-57.2,-28.1]	-47.3 / +43.0 [-60.5,-34.3]	-37.9 / +38.3 [-48.2,-28.4]	-50.8 / +46.1 [-65.0,-37.7]
Archive provenance	-43.4 / +41.0 [-54.0,-32.2]	-45.3 / +45.7 [-59.8,-29.9]	-57.8 / +55.9 [-71.3,-44.4]	-42.2 / +41.8 [-54.0,-31.0]	-52.3 / +47.3 [-67.5,-38.5]
Field name	-43.8 / +42.2 [-54.5,-32.1]	-44.9 / +42.6 [-59.4,-29.9]	-48.4 / +44.1 [-62.3,-34.9]	-46.5 / +46.5 [-59.8,-34.1]	-57.0 / +50.8 [-71.2,-42.4]

Table 4: Question-date matching swaps for

t_{1}

questions on the same 128 Wikipedia pages (45 predicate groups). Cells show later-reference (

t_{1}

) / earlier-reference (

t_{0}

) EM changes, older-matching minus newer-matching, in percentage points. Brackets give nominal 95% predicate-cluster intervals for later-reference EM; earlier-reference intervals: Table 22 . Levels and prompt interactions: Appendix D .

LLM	Synthetic	English date	English minus ISO
Qwen2.5-7B	-95.7 [-98.0,-93.4]	-34.8 [-45.2,-24.2]	+2.7 [-2.9,+8.3]
Phi-4	-94.5 [-97.7,-91.0]	-31.6 [-42.1,-20.6]	+11.3 [+4.3,+19.1]
Mistral	-94.1 [-96.9,-91.4]	-53.1 [-65.7,-38.9]	-5.5 [-13.4,+1.6]
Qwen3.5-35B	-92.6 [-95.3,-89.5]	-35.9 [-46.0,-25.9]	+3.5 [-0.9,+7.5]
Qwen2.5-72B	-92.2 [-95.3,-89.1]	-46.5 [-59.1,-34.7]	+6.6 [+2.7,+11.0]

Table 5: Changes in later-reference (

t_{1}

) EM after swapping collection dates, in percentage points. Synthetic uses 128 fictional records; English date uses 128 Wikipedia pages with month-name question dates. The last column is the English-date minus same-run ISO-date swap effect. Brackets: nominal 95% record intervals for synthetic data and predicate-cluster intervals for Wikipedia pages. Baselines: Appendix G .

LLM	Mode	Newer revision matches $t_{1}$	Older revision matches $t_{1}$	$\Delta$ later-reference EM ( $t_{1}$ )	$\Delta$ earlier-reference EM ( $t_{0}$ )
Qwen3.5-35B	sampled	59.6	29.5	-30.1 [-37.1,-23.1]	+30.5 [+23.5,+37.8]
Qwen3.5-35B	thinking	72.1	60.9	-11.1 [-16.3,-7.0]	+11.1 [+7.0,+16.3]
Ministral-3-14B	thinking	56.1	43.0	-13.1 [-18.1,-8.1]	+13.3 [+8.5,+18.1]
Nemotron-3-Nano	thinking	65.2	13.5	-51.8 [-64.3,-37.8]	+48.8 [+35.4,+61.6]

Table 6: Question-date matching swaps for

t_{1}

on the same 128 Wikipedia pages (45 predicate groups). Matching columns give later-reference EM (%); changes are older-matching minus newer-matching, in percentage points with nominal 95% predicate-cluster intervals. Scores average two seeds and both revision orders. Qwen modes share sampling; cross-family comparisons do not isolate reasoning. Settings and tests: Appendix F .

API LLM	Effort	CB EM	Swap: all 128	Miss $n$	Swap: CB-high misses
GPT-6 Astra	high	21.9	-1.2 [-4.8,+1.5]	100	-2.0 [-6.4,+0.6]
GPT-6 Astra	low	18.0	-0.8 [-2.2,+0.0]	100	-1.0 [-2.7,+0.0]
Claude Fable 5.1	high	7.8	-0.4 [-1.9,+0.9]	118	-0.4 [-2.0,+1.0]
Claude Fable 5.1	low	7.0	-2.3 [-6.3,+1.1]	118	-2.5 [-6.4,+0.9]

Table 7: Contemporaneous high/low API comparison. CB is closed-book reference EM (%). Swap: older-matching minus newer-matching EM percentage points (nominal 95% predicate-cluster intervals). Each LLM's high-effort closed-book EM misses define a subset reused at both efforts, not proof of absent knowledge. Levels and conditional sensitivity: Appendix F .

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Study	Pages	Rendering	LLMs	Relationship
Main date assignments	595	Raw	Core five	Original cohort
Question-day/previous-day swaps	497	Raw	Core five	All eligible pages of 595
Roles / display controls	595	Plain / both	Core five	Same original pages
Baseline decomposition	596	Plain	Core five	Disjoint second cohort
Grid / seven prompts	128	Plain	Core five	Fixed subset of 596
Expanded thinking	128	Plain	Qwen3.5-35B	Same subset; two seeds

Appendix

Table 8: Sample and rendering map. Raw preserves wiki markup; plain preserves its words/numbers while changing delimiters and whitespace. All contrasts hold evidence rendering fixed. Nested samples are not independent cohorts. The initial 32-page reasoning-setting comparison is included in the expanded 128-page experiment.

Study	Complete correction scope retained in the technical record
Original / role diagnostics	Original planned families and their 877-test union; retrospective content descriptions separate.
Main-grid swaps (post hoc)	5 matching and 5 proximity tests; family 10 (Table 13 ).
Observed dates (post hoc)	Original 2 contrasts $\times$ 5 LLMs = 10; expanded 4 contrasts $\times$ 5 LLMs = 20 (Table 14 ).
128-page grid / nested modes	51 EM tests plus 102 exclusive-alias tests; union 153.
Expanded reasoning / proximity	6 native-reasoner and 66 mode/proximity tests; union 72.
Past-side / seven prompts / API	80 local EM, 160 local alias, 22 API tests; union 262.

Appendix

Table 9: Multiplicity map. All listed unions use Holm; nominal intervals remain unadjusted. These are distinct historical/follow-up studies, not one prospective global family. Smaller original families, remaining secondary studies, all nulls and raw/adjusted values remain in the full technical record. Follow-up tests use exact paired-page sign flips; earlier Monte Carlo tests retain their recorded resolution.

Comparison	Family size	Tests	Adjusted $p$ (range)
Synthetic swaps (five LLMs)	60	5	$<10^{-6}$
English-date swaps (five LLMs)	60	5	$<10^{-6}$
Phi-4 English-minus-ISO	60	1	0.000457
Other spelling interactions	60	4	0.082 to 1.000
Claude low-effort synthetic swap	78	1	$<10^{-6}$
Claude low-minus-high interaction	78	1	$<10^{-6}$

Appendix

Table 10: Adjusted decisions referenced in the text. Holm corrections use the complete named study or sensitivity family, not only the rows displayed here. Ranges summarize every LLM in the indicated group. Nominal intervals in the effect tables are not multiplicity-adjusted intervals. The observed-date analyses are post hoc. The expanded family adds unknown-minus-omission and

(c,c)

-minus-unknown for all five LLMs to the original ten tests. The original family is retained here; other families are unchanged.

LLM	Public checkpoint / identifier	Main execution
Qwen2.5-7B / 72B	Qwen/Qwen2.5-7B-Instruct; Qwen/Qwen2.5-72B-Instruct	Greedy
Phi-4	microsoft/phi-4	Greedy
Mistral-24B	mistralai/Mistral-Small-24B-Instruct-2501	Greedy
Qwen3.5-35B	Qwen/Qwen3.5-35B-A3B	Greedy, thinking off
Ministral-3-14B	mistralai/Ministral-3-14B-Reasoning-2512	Native reasoning
Nemotron-3-Nano	nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16	Native reasoning

Appendix

Table 11: LLMs. Local checkpoint hashes, native templates and prompt/token receipts are preserved in the numerical archives. Expanded reasoning uses seeds 0/1 and a 32,768-token cap. API runs use independent requests; public identifiers do not guarantee immutable provider backends.

Collection-date condition	Qwen2.5-7B	Phi-4	Mistral	Qwen3.5-35B	Qwen2.5-72B
Question offsets (0, 0)	55.5	30.1	50.0	60.5	66.8
Question offsets (0, 1)	26.2	7.0	16.4	27.0	20.3
Question offsets (0, 7)	16.8	5.5	10.9	23.0	13.7
Question offsets (1, 0)	64.5	50.0	64.1	67.6	73.4
Question offsets (1, 1)	50.8	30.1	40.6	59.8	63.3
Question offsets (1, 7)	45.7	12.9	11.7	27.3	20.7

Appendix

Table 12: All 18 conditions from one execution on 128 pages (45 predicates), later questions: reference exact match (%), averaged over source orders. Offset pairs always list older/newer versions. Question offsets follow

t_{1}

; collection offsets follow the recorded acquisition day

c

. Only collection

(0,0)

reproduces the log. Appendix C defines the shared design and model names; no condition or model is omitted.

LLM	595 pages	Adjusted $p$	128-page rerun
Qwen2.5-7B	+1.2 [-0.5,+2.8]	0.207	-0.4 [-4.6,+3.9]
Phi-4	-30.8 [-43.0,-19.2]	$<10^{-6}$	-28.5 [-41.8,-15.6]
Mistral-24B	-31.6 [-36.8,-26.2]	$<10^{-6}$	-32.0 [-42.4,-22.5]
Qwen3.5-35B	-28.2 [-35.1,-21.5]	$<10^{-6}$	-30.9 [-42.1,-20.2]
Qwen2.5-72B	-38.9 [-48.1,-28.7]	$<10^{-6}$	-37.1 [-47.1,-27.2]

Appendix

Table 13: Near-question proximity swap

(1,7)-(7,1)

: later-reference EM points with nominal predicate-cluster 95% intervals (113/45 groups). The original-grid adjusted

p

values use its ten-test post hoc family, including the five matching swaps. The 128-page proximity estimates are descriptive. This after-question comparison alone does not separate closeness from earlier collection.

LLM	$(c,c)-$ omission	Adj. $p$	$(c,c)-$ definition only	Adj. $p$
Qwen2.5-7B	-4.7 [-10.6,+0.6]	0.654	-5.5 [-10.6,-1.0]	0.237
Phi-4	-11.3 [-17.7,-4.5]	0.000067	-8.6 [-14.2,-2.7]	0.006
Mistral-24B	-15.2 [-22.0,-9.2]	0.000002	-9.4 [-15.0,-4.3]	0.003
Qwen3.5-35B	-3.5 [-7.6,-0.3]	0.654	-3.1 [-7.2,+0.4]	0.698
Qwen2.5-72B	-8.6 [-15.1,-3.0]	0.009	-6.6 [-13.4,-0.5]	0.088
LLM	unknown $-$ omission	Adj. $p$	$(c,c)-$ unknown	Adj. $p$

Appendix

Table 14: Observed acquisition dates on 128 Wikipedia pages: later-reference (

t_{1}

) EM differences in points, with nominal 95% intervals over 45 predicate groups and both source orders averaged per page. Both revisions were acquired on October 11, 2026 (

c

). Adjusted

p

values use the expanded 20-test Holm family: all four contrasts in all five LLMs. Table 10 summarizes both families. Among the original ten tests, only Qwen2.5-72B versus definition-only changes its decision at

\alpha=.05

: adjusted

p=.034

becomes

.088

. Within each page,

(c,c)-

omission equals (unknown

-

omission) plus

((c,c)-

unknown

)

. This is a comparator decomposition, not identification of independent field-presence and date-value mechanisms: unknown is not neutral, and omission also changes the definition text.

LLM	Older date matches $t_{1}$	Older date closer to $t_{1}$	Shared dates	Newer date closer to $t_{1}$	Newer date matches $t_{1}$
Qwen2.5-7B	23.4	51.2	55.9	50.0	69.7
Phi-4	3.5	10.0	25.6	40.8	52.6
Mistral-24B	19.2	16.6	54.9	48.2	75.0
Qwen3.5-35B	32.2	35.5	65.4	63.7	72.6
Qwen2.5-72B	24.9	22.2	68.9	61.1	77.1

Appendix

Table 15: Descriptive later-reference (

t_{1}

) EM (%) from all nine date assignments on 595 Wikipedia pages. Older/newer date means the collection date assigned to that revision. Matching averages the two assignments with only that date equal to

t_{1}

; closer denotes

(1,7)

(7,1)

; shared dates average

(0,0),(1,1),(7,7)

. Assignments and revision orders have equal weights within each group. This describes behavior, not a fitted mechanism or deployment frequency.

LLM	Later-reference EM change	Earlier-reference EM change
Qwen2.5-7B	-49.3 [-58.8,-38.6]	+48.1 [+38.4,+56.6]
Phi-4	-56.7 [-71.6,-38.4]	+55.0 [+38.9,+67.8]
Mistral-24B	-59.4 [-71.0,-46.2]	+57.5 [+45.6,+68.0]
Qwen3.5-35B	-44.6 [-50.4,-38.6]	+44.4 [+38.2,+50.6]
Qwen2.5-72B	-55.7 [-68.5,-43.7]	+50.9 [+39.0,+66.1]

Appendix

Table 16: Question-date matching swap sensitivity on the 411 original-cohort pages with distinct reference aliases literally present in their corresponding excerpts. Both orders are averaged; changes are older-matching minus newer-matching points with nominal 95% predicate-cluster intervals. This input-defined post hoc subset is descriptive, not entailment validation or a replacement for all 595 pages.

Table 19

Condition	Qwen2.5-7B	Phi-4	Mistral	Qwen3.5-35B	Qwen2.5-72B
$(-1,0)$	70.3	42.3	73.8	75.1	76.6
$(0,-1)$	31.3	13.3	34.2	40.7	44.4
$(0,1)$	28.1	3.3	22.2	33.7	27.9
$(1,0)$	68.5	50.2	74.2	72.2	77.5

Appendix

Table 19: All date-direction condition levels for EM (%), averaging both source orders. Other metrics, page intervals, native token counts and individual responses are retained in the numerical archive.

Condition	Qwen2.5-7B	Phi-4	Mistral	Qwen3.5-35B	Qwen2.5-72B
Original: older matches	27.0	6.6	15.6	27.3	20.7
Original: newer matches	64.8	49.2	63.3	66.8	73.4
Definition 1: older matches	23.0	5.1	9.4	23.8	16.4
Definition 1: newer matches	64.1	52.0	64.1	68.8	73.8
Definition 2: older matches	25.0	6.2	18.8	25.8	19.5
Definition 2: newer matches	66.8	52.7	63.3	68.0	74.2

Appendix

Table 20: All prompt-robustness condition levels for EM (%), averaging both source orders. Other metrics, page intervals, native token counts and individual responses are retained in the numerical archive.

Contrast	Qwen2.5-7B	Phi-4	Mistral	Qwen3.5-35B	Qwen2.5-72B
Definition 1 minus original	-3.1 [-6.0,+0.0]	-4.3 [-6.9,-2.0]	-7.0 [-11.3,-2.8]	-5.5 [-8.8,-2.3]	-4.7 [-7.7,-1.4]
Definition 2 minus original	-3.9 [-7.1,-0.3]	-3.9 [-7.4,-0.7]	+3.1 [+0.4,+6.2]	-2.7 [-5.5,+0.0]	-2.0 [-5.8,+1.0]
Ignore collection minus original	-4.3 [-9.1,+0.9]	-0.4 [-2.7,+2.1]	+0.4 [-4.9,+6.1]	+1.6 [-3.8,+5.9]	+2.0 [-1.0,+4.7]
Archive provenance minus original	-5.5 [-9.5,-1.5]	-2.7 [-5.9,+0.5]	-10.2 [-15.2,-5.6] ∗	-2.7 [-6.7,+0.5]	+0.4 [-3.2,+3.1]
Field name minus original	-5.9 [-9.6,-1.7]	-2.3 [-5.0,+0.4]	-0.8 [-4.8,+3.8]	-7.0 [-11.3,-3.8] ∗	-4.3 [-7.1,-1.4]
Field position minus original	-12.5 [-19.5,-5.8] ∗	+6.6 [+2.0,+11.7]	-7.0 [-12.8,-1.0]	-6.2 [-11.7,-1.8]	+3.5 [-0.7,+8.6]

Appendix

Table 21: Complete prompt-robustness EM interactions, percentage points with nominal 95% predicate-cluster intervals. Stars use the full 262-test follow-up sensitivity; all results are retained. For EM, a positive variant-minus-original change indicates attenuation of the swap penalty, not an absolute accuracy gain. Prompt variants and directions are defined in Appendix D .

Prompt contrast	Qwen2.5-7B	Phi-4	Mistral	Qwen3.5-35B	Qwen2.5-72B
Original	+36.3 [+27.2,+45.3]	+40.2 [+26.8,+53.2]	+45.7 [+32.6,+60.6]	+39.5 [+28.3,+50.9]	+47.3 [+33.7,+62.5]
Definition 1	+38.3 [+27.8,+47.9]	+45.3 [+30.4,+59.6]	+48.0 [+36.4,+60.6]	+44.1 [+30.9,+57.0]	+52.0 [+38.5,+65.9]
Definition 2	+38.3 [+27.8,+47.5]	+44.1 [+30.3,+57.1]	+43.4 [+30.7,+57.7]	+42.2 [+30.4,+54.0]	+50.4 [+36.2,+66.4]
Ignore collection	+39.5 [+28.9,+49.0]	+41.0 [+27.5,+53.8]	+43.0 [+31.6,+55.2]	+38.3 [+28.5,+49.1]	+46.1 [+33.2,+61.4]
Archive provenance	+41.0 [+30.7,+50.9]	+45.7 [+31.1,+59.4]	+55.9 [+42.7,+68.8]	+41.8 [+29.9,+54.2]	+47.3 [+34.1,+61.8]
Field name	+42.2 [+31.0,+52.3]	+42.6 [+27.8,+56.7]	+44.1 [+32.5,+58.0]	+46.5 [+33.3,+60.1]	+50.8 [+36.9,+65.9]

Appendix

Table 22: Earlier-reference EM changes for all seven prompt variants and five LLMs on 128 pages, accompanying the later-reference EM changes in Table 4 . Each cell compares older-matching with newer-matching; nominal 95% predicate-cluster intervals retain both orders and all 45 groups.

LLM	Collection	Administrative	Publication	Last modified
Qwen2.5-7B	$-25.5/-13.9$	$-22.8/-19.9$	$-25.6/-18.2$	$-23.4/-18.4$
Phi-4	$-21.9/-25.4$	$-22.1/-25.4$	$-26.2/-32.5$	$-25.8/-28.9$
Mistral-24B	$-24.8/-17.7$	$-19.5/-18.4$	$-28.4/-20.6$	$-23.8/-16.6$
Qwen3.5-35B	$-14.3/-10.4$	$-13.2/-11.6$	$-17.9/-16.6$	$-14.2/-15.3$
Qwen2.5-72B	$-15.8/-5.5$	$-12.6/-7.6$	$-18.7/-7.3$	$-16.4/-7.0$

Appendix

Table 23: Date-role control on 595 pages with plain display: earlier (

t_{0}

)/later (

t_{1}

) EM changes (points), shared minus question-matched, averaging both orders. Administrative dates are explicitly unrelated to freshness or validity. All four roles and both question dates are shown; these are not matching-swap effects or pairwise role-equivalence tests. Full intervals and definitions are in the technical supplement.

Earlier question
LLM	Shared $-$ unknown	Channel placebo	Date $-$ placebo	Shared $-$ definition only	Shared $-$ channel cache
Qwen2.5-7B	-0.3 [-2.4,+1.5]	-0.2 [-1.0,+0.7]	-0.2 [-2.4,+1.8]	-2.2 [-3.7,-0.5]	-1.2 [-3.5,+1.0]
Phi-4	+1.5 [-0.2,+3.4]	-1.0 [-2.0,+0.1]	+2.5 [+0.4,+4.8]	-2.2 [-4.5,-0.2]	+2.6 [-0.2,+5.4]
Mistral-24B	-2.4 [-5.6,+0.1]	-1.1 [-2.1,-0.3]	-1.3 [-4.6,+1.4]	-3.9 [-8.1,-0.9]	-3.8 [-7.7,-1.0]
Qwen3.5-35B	-0.1 [-2.0,+1.9]	-1.3 [-2.3,-0.1]	+1.2 [-1.1,+3.4]	-3.6 [-5.9,-1.1]	+1.2 [-0.7,+3.3]
Qwen2.5-72B	+1.9 [-1.8,+5.8]	-0.3 [-0.8,+0.2]	+2.3 [-1.6,+6.2]	-0.3 [-3.8,+3.1]	+0.8 [-2.8,+4.3]

Appendix

Table 24: All 596 pages, both question dates and five LLMs: EM changes in percentage points with nominal 95% 107-predicate-cluster intervals. Channel placebo is channel-cache minus channel-unknown; date minus placebo subtracts that contrast from shared minus unknown within page. Definition-only retains the field definition but omits displayed collection values. The two direct comparisons avoid unknown but do not fully match field semantics or value length. These are post hoc analyses. Complete corrections and paired-page intervals are in the technical supplement.

LLM	Earlier ( $t_{0}$ )	Adj. $p$	Later ( $t_{1}$ )	Adj. $p$
Qwen2.5-7B	-1.0 [-3.7,+1.2]	1.000	-4.4 [-7.5,-1.5]	0.000359
Phi-4	+1.5 [-0.3,+3.8]	1.000	-14.5 [-20.8,-9.2]	$<10^{-6}$
Mistral-24B	-3.1 [-5.7,-0.8]	0.261	-3.5 [-7.0,-0.9]	0.071
Qwen3.5-35B	-1.2 [-2.8,+0.8]	1.000	-3.9 [-6.4,-2.3]	0.000048
Qwen2.5-72B	+1.3 [-1.6,+4.7]	1.000	-0.7 [-2.1,+0.9]	1.000

Appendix

Table 25: Separate 599-page cohort, shared-question-date minus unknown EM in points. Nominal 95% intervals resample 115 predicate groups; adjusted

p

values retain the original complete 90-test extension family. The five-LLM experiment has 47,920 responses across four conditions, both question dates and both source orders. These are shared-date comparisons, not matching swaps; earlier Qwen executions mean this was not an unexposed holdout.

LLM	Earlier ( $t_{0}$ )	Adj. $p$	Later ( $t_{1}$ )	Adj. $p$
Qwen2.5-7B	-2.7 [-5.2,-0.5]	0.837	-3.1 [-6.1,+0.6]	0.814
Phi-4	+7.4 [+2.8,+11.6]	0.012	-13.9 [-17.4,-9.8]	0.000400
Mistral-24B	+7.0 [+3.3,+10.9]	0.012	-1.6 [-5.7,+1.8]	1.000
Qwen3.5-35B	+4.1 [+0.3,+7.1]	0.312	-3.3 [-6.1,-1.1]	0.088

Appendix

Table 26: A/B answer selection accuracy: shared minus unknown on 128 pages from the original 595-page cohort. Changes are points with nominal 95% intervals over 45 predicate groups. Adjusted

p

values use all 40 original comparisons (four LLMs, two question dates, five contrasts), not only the eight shown. Both answer-label assignments and source orders are averaged.

LLM	Mode	Seed	$\Delta$ later EM ( $t_{1}$ )	$\Delta$ earlier EM ( $t_{0}$ )
Ministral-3-14B	thinking	0	-14.1 [-21.6,-6.2]	+12.9 [+7.6,+18.7]
Ministral-3-14B	thinking	1	-12.1 [-19.1,-6.2]	+13.7 [+7.7,+19.6]
Nemotron-3-Nano	thinking	0	-53.9 [-67.8,-37.1]	+50.8 [+36.5,+63.7]
Nemotron-3-Nano	thinking	1	-49.6 [-61.6,-37.4]	+46.9 [+33.1,+60.3]
Qwen3.5-35B	sampled	0	-29.3 [-36.9,-21.8]	+29.7 [+22.2,+37.6]
Qwen3.5-35B	sampled	1	-30.9 [-38.3,-23.7]	+31.2 [+23.9,+39.2]

Appendix

Table 27: Every seed-specific question-date matching swap estimate, descriptive rather than separate primary tests. Differences average both revision orders on all 128 Wikipedia pages, with nominal 95% intervals from 45 predicate groups. Aggregate tests average both seeds within page.

Question	Nonthinking sampled	Thinking	Thinking $-$ nonthinking interaction
Earlier	-1.17 [-2.92,+0.67]	+0.83 [-0.92,+2.50]	+2.00 [-0.58,+4.51]
Later	-1.59 [-3.42,+0.25]	-1.00 [-2.34,+0.25]	+0.58 [-1.67,+2.84]

Appendix

Table 28: Shared-date minus unknown EM on the 599-page matched Qwen3.5-35B execution. The last column computes the paired-page difference in this contrast between thinking and nonthinking sampled modes, with nominal retrospective 95% page-bootstrap intervals (seed 20261012). These are not comparisons with a separate greedy run, and they differ from shared-minus-question-matched sensitivity. Neither interaction interval excludes zero.

LLM	Earlier question	Later question
Qwen2.5-7B	+0.0 [-1.6,+1.7]	+2.7 [+0.0,+5.7]
Phi-4	+2.3 [-0.4,+5.0]	+2.3 [-0.5,+5.3]
Mistral-24B	+6.2 [+2.9,+9.6]	+2.0 [-1.3,+5.3]
Qwen3.5-35B	+1.6 [-0.9,+4.6]	+2.3 [+0.4,+4.3]
Qwen2.5-72B	+2.0 [-0.0,+4.1]	+0.0 [-2.2,+2.3]

Appendix

Table 29: Distant-date proximity changes: older-closer minus newer-closer requested-reference EM points on 128 DynaQuest pages, nominal 95% predicate-cluster intervals. Both source orders are retained. No EM effect survives the complete 72-test follow-up adjustment; this is not evidence of equivalence.

LLM	Earlier question	Later question
Mistral-24B	+0.4 [-0.6,+1.5]	+0.6 [+0.0,+1.3]
Qwen3.5-35B	-0.2 [-1.1,+0.8]	+0.8 [+0.2,+1.5]
Qwen2.5-72B	+0.2 [-0.4,+1.0]	+0.0 [-0.8,+0.8]

Appendix

Table 30: Distant-date proximity changes: older-closer minus newer-closer requested-reference EM points on 262 Temporal Wiki pages, nominal 95% paired-page intervals (nine relations). Both source orders are retained. No EM effect survives the complete 72-test follow-up adjustment; this is not evidence of equivalence.

API LLM	Effort	Condition	Later EM	Earlier EM	Later only	Earlier only
GPT-6 Astra	high	Newer matches	67.6	0.0	82.0	1.6
GPT-6 Astra	high	Older matches	66.4	0.0	80.9	1.6
GPT-6 Astra	high	Closed book	21.9	11.7	25.0	17.2
GPT-6 Astra	low	Newer matches	66.4	0.0	82.0	1.6
GPT-6 Astra	low	Older matches	65.6	0.0	81.6	1.6
GPT-6 Astra	low	Closed book	18.0	10.2	24.2	13.3

Appendix

Table 31: Complete API condition levels (%) on all 128 Wikipedia pages, for questions about

t_{1}

. Later/earlier EM uses the later (

t_{1}

)/earlier (

t_{0}

) reference; later-only/earlier-only include one reference answer but not the other. Source-present levels average both revision orders; closed book has one response per effort and page.

API LLM	Subset	Effort	$n$	Groups	EM swap effect
GPT-6 Astra	All pages	high	128	45	-1.2 [-4.8,+1.5]
GPT-6 Astra	All pages	low	128	45	-0.8 [-2.2,+0.0]
GPT-6 Astra	CB-high EM miss	high	100	39	-2.0 [-6.4,+0.6]
GPT-6 Astra	CB-high EM miss	low	100	39	-1.0 [-2.7,+0.0]
GPT-6 Astra	CB-high EM hit	high	28	18	+1.8 [+0.0,+6.2]
GPT-6 Astra	CB-high EM hit	low	28	18	+0.0 [+0.0,+0.0]

Appendix

Table 32: All frozen API subsets, including descriptive sensitivities, without outcome-based selection. Miss/hit use the complete high-effort closed-book EM result; both-CB-no-later-alias requires complete responses without a later alias at both efforts. Nominal 95% predicate intervals are conditional on subset membership;

p

marks a page-only interval when fewer than ten predicate groups remain. These lexical partitions do not certify knowledge or its absence.

LLM	Newer only	No field	Newer matches	Older matches	Swap: earlier EM
Qwen2.5-7B	99.2 / 0.0	97.3 / 2.3	98.4 / 0.0	2.7 / 96.1	+96.1 [+93.8,+98.0]
Phi-4	93.0 / 0.0	96.9 / 0.0	94.5 / 0.0	0.0 / 94.5	+94.5 [+90.6,+97.7]
Mistral-24B	99.2 / 0.0	99.6 / 0.0	99.6 / 0.4	5.5 / 94.5	+94.1 [+91.4,+96.9]
Qwen3.5-35B	100.0 / 0.0	100.0 / 0.0	100.0 / 0.0	7.4 / 92.6	+92.6 [+89.5,+95.3]
Qwen2.5-72B	100.0 / 0.0	100.0 / 0.0	99.6 / 0.0	7.4 / 91.4	+91.4 [+87.9,+94.5]

Appendix

Table 33: 128 fictional records, both revision orders except the single newer-only input. No-field controls retain the original collection definition. Levels give later-reference (

t_{1}

) / earlier-reference (

t_{0}

) EM (%). The final column gives the earlier-reference swap change, with a nominal 95% record interval conditional on eight templates.

LLM	Prose (64 records)	Key–value (64 records)
Qwen2.5-7B	-96.1 [-99.2,-92.2]	-95.3 [-98.4,-91.4]
Phi-4	-90.6 [-96.1,-84.4]	-98.4 [-100.0,-95.3]
Mistral-24B	-93.8 [-97.7,-89.1]	-94.5 [-97.7,-90.6]
Qwen3.5-35B	-96.9 [-99.2,-93.8]	-88.3 [-93.0,-82.8]
Qwen2.5-72B	-94.5 [-98.4,-90.6]	-89.8 [-94.5,-85.2]

Appendix

Table 34: Synthetic question-date matching swap effects on later-reference EM, in both render styles. Each style has 16 records per relation. Nominal record-bootstrap intervals condition on four relation templates; style results are descriptive, with no selected-style significance claims.

LLM	ISO: newer matches	ISO: older matches	English: newer matches	English: older matches	English: earlier EM
Qwen2.5-7B	64.5 / 7.0	27.0 / 42.2	66.0 / 5.5	31.2 / 40.2	+34.8 [+24.6,+44.7]
Phi-4	49.6 / 2.3	6.6 / 42.6	46.9 / 2.3	15.2 / 31.6	+29.3 [+19.2,+39.7]
Mistral-24B	63.7 / 4.7	16.0 / 50.0	69.1 / 2.7	16.0 / 53.1	+50.4 [+37.1,+63.0]
Qwen3.5-35B	68.0 / 5.9	28.5 / 45.7	68.4 / 5.9	32.4 / 41.0	+35.2 [+24.6,+45.5]
Qwen2.5-72B	73.4 / 1.2	20.3 / 48.8	75.0 / 0.4	28.5 / 41.8	+41.4 [+29.3,+55.4]

Appendix

Table 35: 128 Wikipedia pages, 45 predicate groups, both orders and all four spelling/assignment conditions. Levels give later-reference (

t_{1}

) / earlier-reference (

t_{0}

) EM (%). The final column gives the English-format earlier-reference swap change with a nominal 95% predicate-cluster interval. ISO controls are rerun alongside the English-date conditions.

API LLM	Effort	Newer matches	Older matches	$\Delta$ later EM	$\Delta$ earlier EM	Later alias
GPT-6 Astra	high	100.0	100.0	+0.0 [+0.0,+0.0]	+0.0 [+0.0,+0.0]	100.0
GPT-6 Astra	low	100.0	99.6	-0.4 [-1.2,+0.0]	+0.0 [+0.0,+0.0]	99.6
Claude Fable 5.1	high	100.0	100.0	+0.0 [+0.0,+0.0]	+0.0 [+0.0,+0.0]	100.0
Claude Fable 5.1	low	100.0	85.9	-14.1 [-18.0,-10.2]	+0.4 [+0.0,+1.2]	99.6

Appendix

Table 36: Question-date matching swaps on 128 fictional records, 2,048 API responses. Newer/older matching conditions give later-reference (

t_{1}

) EM (%) when the newer/older revision receives

t_{1}

. Changes are older-matching minus newer-matching percentage points; later/earlier EM uses

t_{1}

t_{0}

references. Later alias is later-answer inclusion in older-matching responses, not necessarily unambiguous selection. Nominal 95% paired-record intervals condition on eight templates, averaging both orders; [0,0] means no observed variation, not equivalence. Both efforts use reasoning.

Temporal QA

Temporal question answering determines what happened, or what was true, at particular times by reasoning over events, intervals, and changing evidence.

See all 54 papers

Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding
Aug 9, 2026Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Fahmid Hasan Chowdhury +4LLM Evaluation Document QA
1BRAC University · 2United International University · 3North South University +1
How often do Answers Change? Estimating Recency Requirements in Question Answering
Mar 17, 2026Bhawna Piryani, Zehra Mert, Adam JatowtTemporal QA Temporal Reasoning in Language Models
University of Innsbruck Innsbruck, Austria · MEF University Istanbul, Turkey
Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers
Sep 25, 2026Md Shamim Ahmed, Lukas Galke Poech, Richard RöttgerRAG Poisoning Attacks RAG Robustness
University of Southern Denmark

Aug 9, 2026J/K move · Enter open · S save

Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding

Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Fahmid Hasan Chowdhury, Md Adnan Arefeen+3

1BRAC University · 2United International University · 3North South University+1

LLM Evaluation Document QA Legal QA Temporal QA

Open paper AskPDF arXiv

Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates. In contrast to encyclopedic knowledge, where an old fact is simply overwritten, an amendment is itself an official text that states what it replaces and when it takes effect, and the earlier version stays correct for its validity period. The central challenge is therefore version resolution, that is, identifying the version in force on the queried date. Existing temporal QA datasets treat time only as an annotation, so version resolution stays untested. We present TIDE, an expert-verified benchmark of 3,050 QA pairs over 644 official customs instruments issued between 1969 and 2025 by the Government of Bangladesh, covering eight task types over deeply code-mixed documents that are heterogeneous in layout and dated in two calendars. In addition, we evaluate nine recent LLMs under a single protocol across parametric, gold-context, and retrieval access, scored by a three-judge LLM council with a hard date gate separating correct meaning from correct time. The best macro-averaged accuracy is only 68.5%. Resolving a version from an implicit date reaches 59.7%, and detecting that the supplied version does not govern the query reaches only 26.7%. Models are more likely to find correct versions than to reject incorrect ones, and they tend to follow a confident parametric answer over the supplied authoritative text. All code and data are available at https://github.com/icsetepa44/TIDE

Abstract

Temporal QA

Explore similar work

Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding

How often do Answers Change? Estimating Recency Requirements in Question Answering

Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers