A consistent concept of the current time is important for temporal reasoning, yet how language models represent the current time is not well understood. We contribute two tasks that probe the current year in conceptually distinct ways: an associative task, which infers the current year from verb tense, and a declarative task, which directly queries for the current year. Both tasks estimate current years within one year of the post-training data cutoff of instruction-tuned language models. For base models, predictions on the associative task serve as a strong proxy for the pre-training data cutoff, with an average error of only 10 months across 13 models. However, their internal mechanisms diverge: the associative task uses mechanisms similar to factual recall, while the declarative task lacks consistent causal pathways. This divergence poses a challenge for updating the current year in language models. None of prompting, SFT, or weight editing succeed in shifting the associative and declarative years simultaneously. Prompting updates the declarative year (94.6% success across 351 target years) but leaves the associative year nearly unchanged (1.7% success). Year-shifted SFT also fails to shift the associative year, matching the target year in only one of eight models. Weight editing, while effective for both tasks individually, does not generalize across both. Overall, our results show that the current year is not consistently encoded in language models: The associative notion, deeply ingrained in linguistic structures learned in pre-training, uses different causal mechanisms and resists the same modifications that easily shift the declarative notion learned in post-training.
Figures & tables
Figure 1Figure 2Figure 3
Prompted with Target Year
Standard SFT
Sys-Prompt SFT
Target
Decl.
Assoc.
ACE
Decl.
Assoc.
ACE
1988
∗ 1992 (+4)
2022 (+34)
0.64
∗ 1988 (+0)
2022 (+34)
0.96
2005
∗ 2007 (+2)
2020 (+15)
0.43
∗ 2005 (+0)
2020 (+15)
0.67
2008
∗ 2008 (+0)
2020 (+12)
0.40
∗ 2008 (+0)
2020 (+12)
0.63
2017
∗ 2017 (+0)
2020 (+3)
0.36
∗ 2017 (+0)
2020 (+3)
0.63
Table 4: None of prompting, SFT, or weight editing shifts the associative and declarative current year simultaneously. We attempt to update the ACY and DCY on versions of OLMo2-1B . +/- values are signed differences of predicted year to the target year. The associative cross-entropy (ACE) is calculated on the range (1900, 2250). Asterisks (*) indicate the LM responses to at least 6 out of 10, but not all declarative prompts with a year.
Model
DCY accuracy
ACY accuracy
OLMo2-1B Standard SFT
0.513
0.011
OLMo2-1B Sys-prompt SFT
0.940
0.020
OLMo2-7B Standard SFT
1.000
0.157
Gemma3-1B-it
0.288
0.014
Gemma3-4B-it
0.231
0.399
Gemma3-12B-it
0.986
0.311
Table 5: Shifting the ACY through prompting fails across model families and scales. We attempt to update the DCY and ACY of nine additional LMs by specifying the target current year in the system prompt. DCY and ACY accuracy are the fraction of target years for which the predicted year exactly matches the target, computed over 351 target years in 1900–2250.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Various LMs demonstrate a coherent notion of current year. The prompt used is In [year] there .
Figure 6: OLMo2-7B demonstrates a coherent notion of the current year on various prompts and output verbs.
#
Sentence template (past / present)
1
“The party [was / is] fun in {year}.”
2
“The meeting [was / is] productive in {year}.”
3
“The mood [was / is] celebratory in {year}.”
4
“The weather [was / is] unusually warm in {year}.”
5
“The economy [was / is] strong in {year}.”
6
“The city [was / is] crowded in {year}.”
Appendix
Table 6: Sentence templates for the alternative ACY setup. We score each template in its past and present form (bracketed), sweeping {year} over 1900–2250. Because the verb comes early and the year comes last, the model must settle on a tense before it sees the year. Templates 1–8 use singular agreement and templates 9–12 use plural agreement.
Figure 7: The alternative ACY setup reproduces the past-to-present shift when tense precedes the year. For each of the six LMs, we plot the mean sentence log-probability of the past (orange) and present (blue) tense templates from Table 6 across years, with the training data cutoff marked (dotted line).
#
Year
Prompt
1
2010
“During the FIFA World Cup in South Africa, there”
2
2010
“During the earthquake response in Haiti, there”
3
2011
“During the Arab Spring protests across the Middle East, there”
4
2011
“At the time of Osama bin Laden’s killing, there”
5
2012
“At the time of the Higgs boson discovery at CERN, there”
6
2013
“During the Edward Snowden NSA surveillance revelations, there”
Appendix
Table 7: Implicit contextual association prompts. Each prompt names a real-world event tied to a ground-truth year, without stating the year explicitly. We use 2–3 events per year over 2010–2028, spanning events before, at, and after the training data cutoff.
Figure 8: The implicit contextual association task does not recover the ACY signal. For each event prompt (ordered by year on the x-axis), we plot the next-token probability mass on past verbs (orange) versus present+future verbs (purple), for base and SFT versions of OLMo2-1B and OLMo2-7B . Past tense dominates across all events with no clear rise in present+future probability toward the training data cutoff.
Fact
Continuation form (base LMs)
Question form (SFT LMs)
Summer Olympics (Tokyo)
(1) “The most recent Summer Olympics was held in the city of” (2) “The host city of the most recent Summer Olympics was” (3) “The most recent Summer Olympic Games took place in”
(1) “What city hosted the most recent Summer Olympics?” (2) “Which city most recently hosted the Summer Olympics?” (3) “In what city did the most recent Summer Olympic Games take place?”
Winter Olympics (Beijing)
(1) “The most recent Winter Olympics was held in the city of” (2) “The host city of the most recent Winter Olympics was” (3) “The most recent Winter Olympic Games took place in”
(1) “What city hosted the most recent Winter Olympics?” (2) “Which city most recently hosted the Winter Olympics?” (3) “In what city did the most recent Winter Olympic Games take place?”
Nobel Peace Prize (Mohammadi)
(1) “The most recent Nobel Peace Prize was awarded to” (2) “The most recent winner of the Nobel Peace Prize is” (3) “The person who most recently received the Nobel Peace Prize is”
(1) “Who most recently won the Nobel Peace Prize?” (2) “Who is the most recent Nobel Peace Prize laureate?” (3) “Who was the most recent recipient of the Nobel Peace Prize?”
Nobel Physics (Krausz)
(1) “The most recent Nobel Prize in Physics was awarded to” (2) “The most recent winner of the Nobel Prize in Physics is” (3) “The most recent Nobel Prize in Physics laureate is”
(1) “Who most recently won the Nobel Prize in Physics?” (2) “Who is the most recent Nobel Prize in Physics winner?” (3) “Who is the most recent Nobel Physics Prize laureate?”
Nobel Chemistry (Bawendi)
(1) “The most recent Nobel Prize in Chemistry was awarded to” (2) “The most recent winner of the Nobel Prize in Chemistry is” (3) “The most recent Nobel Prize in Chemistry laureate is”
(1) “Who most recently won the Nobel Prize in Chemistry?” (2) “Who is the most recent Nobel Prize in Chemistry winner?” (3) “Who is the most recent Nobel Chemistry Prize laureate?”
Nobel Literature (Fosse)
(1) “The most recent Nobel Prize in Literature was awarded to” (2) “The most recent winner of the Nobel Prize in Literature is” (3) “The most recent Nobel Prize in Literature laureate is”
(1) “Who most recently won the Nobel Prize in Literature?” (2) “Who is the most recent Nobel Prize in Literature winner?” (3) “Who is the most recent Nobel Literature Prize laureate?”
Appendix
Table 8: Factual recency probe: the 18 recurring facts and all prompts. Each fact (with its answer as of the 2023 cutoff in parentheses) has a well-defined “most recent” answer that changes over time. We write three phrasings per fact, each realized as a sentence-completion (continuation) prompt for base LMs and a matching question prompt for SFT LMs.
Figure 9: Factual recency does not cleanly encode the current year. Each row is one of 18 recurring facts; for its three phrasings we draw the year range over which the LM’s named answer is correct; current (green) if the range covers the ∼ 2023 cutoff (dashed line), behind (orange) if the answer is real but older. Alternatively, we draw a marker for a hallucinated ( ⋄ ) or refused ( × ) response. The gold band is the correct-answer range; a left-pointing mark in the margin flags an answer from before 2008 (labeled with its year). Columns are the base and SFT versions of OLMo2-1B and OLMo2-7B .
Figure 10: Relative perplexities of LMs per month using the WIKISPAN dataset.
Figure 11: Next token predictions on OLMo2 base stage1 LM. 11(a) and 11(b) show next-token predictions of the LM after stage2, 11(c) shows the normalized co-occurrence model created from 10k steps of olmo-mix-1124, 11(d) shows the normalized n-gram model created from 10k steps of olmo-mix-1124. The associative task follows the training data distribution.
Figure 12: Next token predictions on OLMo2 base stage2 LM. 12(a) and 12(b) show next-token predictions of the LM after stage2, 12(c) shows the normalized co-occurrence model created from 10k steps of dolmino-mix-1124, 12(d) shows the normalized n-gram model created from 10k steps of dolmino-mix-1124. The associative task follows the training data distribution.
Count model
LM
Stage 1
Stage 2
N-gram
OLMo2-1B
0.34 ( − 0.40)
0.34 ( − 0.19)
OLMo2-7B
0.35 ( − 0.28)
0.36 ( − 0.12)
Co-occurrence
OLMo2-1B
0.61 ( − 0.42)
0.68 ( − 0.21)
OLMo2-7B
0.78 ( − 0.33)
0.70 ( − 0.16)
Appendix
Table 9: OLMo2 LMs have lower cross-entropy with corpus-count models than baselines. The first two sections show the CE difference against the N-gram and co-occurrence baselines, with the delta versus the corresponding random baseline in parentheses. The last two sections show the CE difference against random token selections drawn from N-gram and co-occurrence count distributions, respectively.
Layer
Random sub- space
Randomly initialized model
Non-year non-number entities
Non-year num- bers
Different template structure
31
0.14
0.00
0.00
0.00
0.24
30
0.14
0.00
0.00
0.00
0.24
28
0.14
0.00
0.00
0.08
0.28
26
0.14
0.00
0.00
0.10
0.26
24
0.14
0.00
0.00
0.15
0.28
22
0.14
0.00
0.39
0.35
0.39
Appendix
Table 10: The year-type subspace is non-trivial and specific to year–like representations. IIA across layers on the second token of the year for various setups.
Table 11: All prompts used for the associative interchange robustness experiments.
Figure 13: Attention and MLP knockout corroborate associative task’s causal abstraction results. We show results for the associative task, measured by tense shift magnitude (TSM), averaged across 35 past-year (top) and 35 future-year (bottom) prompts. We see large TSM in early layers on year tokens and late layers on the final token, the same as in causal abstraction. Future-year prompts show stronger dependence on the second year token (last digit) than past-year prompts.
#
Prompt
Continuation
1
“Finish the sentence: The current year (in YYYY format) is”
2
“What day of the week is Valentines day this year? For the year”
3
“Today, the year is”
4
“Write a story about the current year. Story: The year is”
5
“Write a yearly summary, log the current year at the top. Year:”
Appendix
Table 12: Declarative prompts . Prompts grouped by continuation and instruction. These prompts were used in Section 4 .
Figure 14: Year counts for training datasets olmo-mix, dolmino, and tulu3. Normalized year frequency in pre-training (olmo-mix, dolmino-mix) and SFT (tulu3) datasets, counted as raw 4-digit occurrences (top) and in the In [year] context (bottom). The In [year] counts are used as the primary measure since they reflect temporal usage better than arbitrary 4-digit numbers.
Figure 15: Mean and variance of causal tracing magnitude shift for declarative prompts …(continued)
Figure 16: Mean and variance of causal tracing magnitude shift for declarative prompts …(continued)
Figure 17: Mean and variance of causal tracing magnitude shift for declarative prompts …(continued)
Figure 18: Mean and variance of causal tracing magnitude shift for declarative prompts , shown across four prompt types ([1]–[4]) in question and continuation formats. Unlike the associative task, there is no consistent layer or token position across prompt types, reflecting the lack of a stable causal pathway for the declarative current year. [1], [2], [3], and [4] are used to refer to the varying few-shot examples in the prompt, which differ in token length and therefore cannot be shown in the plots.
Table 13: Declarative prompts used for current-year tasks. We use few-shot prompts to elicit a current year with both a question and continuation format. Month and day vary across prompts to prevent the LM from pattern-matching on a fixed date.
Table 14: Declarative prompts used for today-date tasks. We use few-shot prompts to elicit a current year with both a question and continuation format. We fix the month and day as June 14 and vary the historical examples.
Figure 19: Training dynamics across training checkpoints for OLMo2-7B . Points represent the declarative year, and bars represent the standard deviation. The horizontal line indicates the data cutoff year. This is the OLMo2-7B counterpart of Figure 3 .
Language models are appealing tools for research on the past. But to trust the evidence a model provides, researchers need to know whether its responses fit the period represented. Validation is challenging, because this is not a task living people ordinarily perform, and because many questions have multiple correct answers. We use historical texts to develop a benchmark for a model's representation of English-language contexts 1831-1930, relying on pairwise comparisons to multiple ground truths and strong distractors to score the hardest questions in an appropriately graduated way. We find that generative tasks are harder than discriminative ones; in fact, reasoning models can typically discern the weakness of their own generated answers. While models pretrained exclusively on historical text lead the pack when evaluated by answer likelihood, they cannot compete with commercial models in free generation. None of the models we tested represent historical contexts in a fully persuasive way yet, but progress toward that goal is evident.
Ted Underwood, Ziliang Qiu, Sarah Griebel +4
University of Illinois Urbana-Champaign · University of British Columbia · University of Missouri +1
Model editing keeps large language models (LLMs) up to date without retraining, but temporal facts expose a limitation of the prevailing locate-and-edit paradigm: an update is not always a replacement. When a fact changes, the new answer should become current while the old answer may remain correct in historical time contexts. Building on this insight, we use causal tracing to show that LLMs already support this distinction via a two-stage internal computation: early MLP layers retrieve a time-agnostic subject representation, and later layers modulate it with temporal context to yield the time-correct answer. Motivated by this finding, we introduce PRISM Edit, which optimizes a single polysemous representation across temporal contexts and leverages the model's inherent modulation pathway to route it to temporally correct predictions without requiring any architectural modification. We evaluate on TimeConflict, a newly introduced temporal editing benchmark, and on temporally augmented CounterFact. PRISM Edit improves multiple core metrics over the best baseline, most notably +23.3 Temporal Consistency (TC) and +33.7 Current Relative-time Score (CRS) on LLaMA-3, while being more than 2x faster. Code and data are publicly available at https://github.com/CheerCHuang/PRISM-Edit.
Cultural attitudes and social norms evolve over time, yet most cultural benchmarks for large language models (LLMs) treat culture as a static attribute of a country or region. We introduce ChronoCulture, a benchmark of 10,360 temporal pairs built from longitudinal survey data across 72 countries. Each pair contains the same survey question asked in the same country in an older and a newer year, with full population response distributions as ground truth. Across six LLMs, we find that models can often predict a country's attitudes at one point in time but do not reliably capture how those attitudes change. Their predictions for different years tend to stay close together, failing to capture much of the observed change. Specifying the intended year in the prompt also offers little overall improvement in alignment with survey responses. These findings suggest that models often present a largely static picture of cultural attitudes, highlighting the need to evaluate cultural awareness across time.