Identifying Temporal Features within Transcoders for Time Sensitive Factual Recall

Sanjay Govindan; Yang Song; Maurice Pagnucco

cs.CLSep 27, 2026

Identifying Temporal Features within Transcoders for Time Sensitive Factual Recall

Authors: Sanjay Govindan, Yang Song, Maurice Pagnucco

Organizations: University of New South Wales, Sydney

Abstract

Large Language Models (LLMs) suffer from temporal misalignment, often due to the contradictory nature of their training corpora. While current mitigation strategies rely on computationally expensive fine-tuning or context-heavy retrieval augmented generation (RAG), the internal mechanisms governing time-sensitive recall remain under-explored. Unlike prior studies that identify temporal components such as attention heads and MLP layers, we provide the first feature-level map of temporal recall by isolating individual MLP features via transcoder circuit tracing. We identify three node categories (common temporal, common to the year, and chrono-semantic) which interact to generate a temporal filter during factual recall. By analysing Gemma 2 2B, LLaMA 3.2 1B, and Qwen3-4B, we show that these features do not follow a simple linear pipeline but represent time through a parallel and mixed syntactic-semantic interplay across layers. We additionally discover a class of higher-layer temporal components invisible to existing EAP-IG methods, establishing transcoders as a more complete lens for temporal interpretability in time-sensitive factual recall. These findings present MLP components for potential targeted interventions in time-sensitive factual recall

Figures & tables

Figure 1: The overview of our experimental setup. (a) Using a new dataset, ChronosAlign, to generate question answer pairs over time. (b) illustrates the construction of transcoder circuits across a range of time sensitive question answers. This enables a refined understanding of which layers and transcoder features are selectively important for specific years ( common to the year features), or temporal recall ( common temporal features). (c) We validate our findings by testing certain nodes against time sensitive subjects (people, places and events) and noting the average token activation for specific nodes and years. (d) Finally, we demonstrate the importance of these nodes on time sensitive recall through ablation and steering studies, using logit probability differences and F1 scores to measure outcomes - aligning to 2022 in this example

Figure 4: Gemma 2 2B activation heatmap for chrono-semantic features. The graph is normalised for feature ids to highlight the continuous nature of each chrono-semantic feature across a period of time. Notably there are a range of features that cover the temporal range between 2015 up until 2024. Note that the invariant measure is 0 (the last column), demonstrating that the activation is time period specific.

	F1 Scores					RVE Scores			Non-Target Ans.
Year	Inv. F1	Rel. F1	Exp. F1	Temp. Ab. F1	Rand. Ab. F1	Prior rve	Ab. new rve	Rand. new rve	Avg. Logit Diff.
2020	0.446	0.514	0.908	0.381	0.736	-0.052	0.035	-0.048	0.025
2021	0.509	0.477	0.914	0.379	0.828	-0.062	0.053	-0.059	0.010
2022	0.390	0.425	0.917	0.302	0.813	-0.051	0.045	-0.047	0.041
2023	0.444	0.487	0.925	0.357	0.803	-0.072	0.042	-0.059	0.049
2024	0.635	0.549	0.903	0.493	0.779	-0.012	0.012	-0.006	0.001

Table 1: Ablation successfully removes temporal grounding, leading degraded F1 due to an increase in non-target (neither relative, invariant nor explicit answers) answer probabilities. Temporal ablation (Temp. Ab. F1) shifts responses toward relative answers (positive Ab. new rve) but achieves F1 scores below even the relative baseline, indicating incomplete factual knowledge. Random ablation maintains baseline rve but reduces F1, confirming our targeted features affect temporal considerations. Gemma 2 2B results on ChronosAlign dataset.

	F1 Scores			RVE Scores
Year	Rel. F1	Exp. F1	Steer F1	Prior rve	Steer new rve
2020	0.071	0.351	0.177	0.063	-0.030
2021	0.085	0.370	0.157	0.073	0.002
2022	0.020	0.416	0.151	0.069	0.013
2023	0.046	0.320	0.209	0.068	-0.068
2024	0.081	0.148	0.093	0.041	0.041

Table 2: F1 and RVE scores for 2020-2024, ChronosAlign dataset, Gemma 2 2b. Using layers above 7, m=5, a=1. Temporal Steering F1 (Steer F1) present F1 scores for each intervention. A score closer to explicit F1 is better. Prior rve and Steer new rve show probability differences between relative and explicit answers before and after intervention. A smaller new rve value relative to prior rve value is better

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: F1 scores for ChronosAlign. We run explicit and relative scores. Gemma 2 (Blue), Red Qwen 3 (Red), LLaMA 3.2B (Orange).

Gemma	LLaMA	Qwen
0.9	0.7	0.9

Appendix

Table 3: Extremely high confidence thresholds for each model

		2020	2021	2022	2023	2024	Combined
Gemma 2 2B	Repl.	0.732	0.730	0.730	0.729	0.733	0.731
Gemma 2 2B	Comp.	0.929	0.928	0.929	0.928	0.929	0.929
LLaMA 3.2 1B	Repl.	0.590	0.610	0.594	0.610	0.612	0.599
LLaMA 3.2 1B	Comp.	0.874	0.879	0.874	0.879	0.880	0.876
Qwen3-4b	Repl.	0.462	0.461	0.462	0.458	0.464	0.461
Qwen3-4b	Comp.	0.884	0.884	0.884	0.883	0.885	0.884

Appendix

Table 4: Average replacement (Repl.) and completeness (Comp.) scores across 2020–2024 for all models. The combined score is the average across all years.

Model	Ablation			Steering
	a	m	l	a	m	l
Gemma 2 2B	1	-2	4	1	5	4
LLaMA 3.2 1B	0	-1	1	0.5	5	2
Qwen3-4b	0.5	-1	4	1	2	4

Appendix

Table 5: parameters used for causal intervention. From Eq. 11 where

a

and

m

are the additive and multiplier coefficients, and

l

denotes the initial layer intervention unless otherwise specified. The intervention was applied to all tokens except the last two which are the answer block “Answer:”.

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.324	0.373	0.298	0.359	0.441
	Relative F1	0.514	0.477	0.425	0.487	0.549
	Explicit F1	0.908	0.914	0.917	0.925	0.903
	Invariant F1	0.446	0.509	0.390	0.444	0.635
Logit Diff	Explicit Logit Diff	-0.151	-0.129	-0.140	-0.137	-0.051
	Relative Logit Diff	-0.064	-0.014	-0.044	-0.023	-0.027

Appendix

Table 6: F1 scores, with logit probability difference for invariant and relative answers. Gemma 2 2B, ChronosAlign ablation using common temporal nodes

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.745	0.799	0.791	0.778	0.698
	Relative F1	0.514	0.477	0.425	0.487	0.549
	Explicit F1	0.908	0.914	0.917	0.925	0.903
	Invariant F1	0.446	0.509	0.390	0.444	0.635
Logit Diff	Explicit Logit Diff	-0.005	0.006	0.003	-0.002	-0.005
	Relative Logit Diff	0.000	0.009	0.007	0.011	0.001

Appendix

Table 7: F1 scores, with logit probability difference for invariant and relative answers. Gemma 2 2B, ChronosAlign ablation using time invariant nodes.

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.231	0.149	0.163	0.201	0.158
	Relative F1	0.071	0.085	0.020	0.046	0.081
	Explicit F1	0.351	0.370	0.416	0.320	0.148
	Invariant F1	0.083	0.076	0.056	0.089	0.118
Logit Diff	Explicit Logit Diff	0.079	0.051	0.039	0.086	0.019
	Relative Logit Diff	-0.042	-0.039	-0.031	-0.060	-0.066

Appendix

Table 8: F1 scores, with logit probability difference for invariant and relative answers. Gemma 2 2B, ChronosAlign Steering using common to the year nodes

	Metric	2004	2015	2020	2021	2022
F1	Ablation F1	0.168	0.226	0.157	0.159	0.140
	Relative F1	0.124	0.232	0.153	0.174	0.173
	Explicit F1	0.199	0.260	0.207	0.223	0.231
Logit Diff	Explicit Logit Diff	-0.054	-0.066	-0.100	-0.119	-0.128
Logit Diff	Relative Logit Diff	-0.008	-0.035	-0.027	-0.016	-0.015
Probs. Change	Prior rve	-0.082	-0.032	-0.039	-0.056	-0.065

Appendix

Table 9: F1 scores, with logit probability difference for invariant and relative answers. Gemma 2 2B, Taqa ablation using common temporal nodes

Dataset	Ablation F1	Original F1	Original Answer Logit Diff
Commonsense	0.377	0.370	0.007
GSM8K	0.307	0.262	-0.002

Appendix

Table 10: F1 Score and logit probabilities for ablation results on time-invariant datasets, Gemma 2 2B

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.442	0.571	0.525	0.467	0.682
	Relative F1	0.514	0.477	0.425	0.487	0.549
	Explicit F1	0.908	0.914	0.917	0.925	0.903
	Invariant F1	0.446	0.509	0.390	0.444	0.635
Logit Diff	Explicit Logit Diff	-0.095	-0.076	-0.075	-0.096	-0.014
	Relative Logit Diff	-0.015	0.012	-0.010	0.014	-0.004

Appendix

Table 11: F1 scores, with logit probability difference for invariant and relative answers. Ablation using higher temporal representation, layers equal to or greater than 12. Gemma 2 2B, ChronosAlign

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.321	0.377	0.298	0.321	0.440
	Relative F1	0.514	0.477	0.425	0.487	0.549
	Explicit F1	0.908	0.914	0.917	0.925	0.903
	Invariant F1	0.446	0.509	0.390	0.444	0.635
Logit Diff	Explicit Logit Diff	-0.148	-0.119	-0.122	-0.127	-0.033
	Relative Logit Diff	-0.067	-0.022	-0.042	-0.033	-0.019

Appendix

Table 12: Semantically different questions, where time is presented at the end of the question. F1 scores, with logit probability difference for invariant and relative answers.

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.404	0.491	0.395	0.437	0.409
	Relative F1	0.557	0.549	0.539	0.617	0.460
	Explicit F1	0.896	0.897	0.887	0.907	0.911
	Invariant F1	0.577	0.587	0.554	0.628	0.507
Logit Diff	Explicit Logit Diff	-0.245	-0.260	-0.238	-0.211	-0.194
	Relative Logit Diff	-0.140	-0.119	-0.136	-0.087	-0.077

Appendix

Table 13: F1 scores, with logit probability difference for invariant and relative answers. Qwen3-4b, ChronosAlign ablation using common temporal nodes

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.475	0.538	0.383	0.490	0.421
	Relative F1	0.557	0.549	0.539	0.617	0.460
	Explicit F1	0.896	0.897	0.887	0.907	0.911
	Invariant F1	0.577	0.587	0.554	0.628	0.507
Logit Diff	Explicit Logit Diff	-0.189	-0.207	-0.228	-0.190	-0.218
	Relative Logit Diff	-0.105	-0.130	-0.123	-0.068	-0.081

Appendix

Table 14: F1 and cumulative probability scores for 2020-2024 given the top 100 (Qwen3-4b, ChronosAlign random invariant)

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.099	0.099	0.072	0.074	0.104
	Relative F1	0.086	0.073	0.079	0.085	0.089
	Explicit F1	0.256	0.325	0.241	0.268	0.288
	Invariant F1	0.113	0.113	0.128	0.098	0.152
Logit Diff	Explicit Logit Diff	0.011	0.013	0.000	0.002	0.002
	Relative Logit Diff	0.000	0.004	0.000	0.002	-0.003

Appendix

Table 15: F1 scores, with logit probability difference for invariant and relative answers. Qwen3-4b, ChronosAlign steering using common to the year nodes

	Metric	2004	2015	2020	2021	2022
F1	Ablation F1	0.168	0.226	0.157	0.159	0.140
	Relative F1	0.124	0.232	0.153	0.174	0.173
	Explicit F1	0.199	0.260	0.207	0.223	0.231
Logit Diff	Explicit Logit Diff	-0.054	-0.066	-0.100	-0.119	-0.128
Logit Diff	Relative Logit Diff	-0.008	-0.035	-0.027	-0.016	-0.015
Probs. Change	Prior rve	-0.082	-0.032	-0.039	-0.056	-0.065

Appendix

Table 16: F1 scores, with logit probability difference for invariant and relative answers. Qwen3-4b, Taqa ablation using common temporal nodes

Dataset	Ablation F1	Original F1	Original Answer Logit Diff
Commonsense	0.457	0.572	-0.087
GSM8K	0.310	0.515	-0.024

Appendix

Table 17: F1 Score and logit probabilities for ablation results on time-invariant datasets, Qwen3-4b

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.573	0.755	0.558	0.683	0.554
	Relative F1	0.557	0.549	0.539	0.617	0.460
	Explicit F1	0.896	0.897	0.887	0.907	0.911
	Invariant F1	0.577	0.587	0.554	0.628	0.507
Logit Diff	Explicit Logit Diff	-0.108	-0.091	-0.084	-0.057	-0.140
	Relative Logit Diff	-0.051	-0.043	-0.019	-0.003	-0.023

Appendix

Table 18: F1 scores, with logit probability difference for invariant and relative answers. Ablation using higher temporal representations, L11F148703 , L15F127839 , L16F45664 . Qwen3 4b, ChronosAlign

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.356	0.402	0.324	0.493	0.359
	Relative F1	0.557	0.549	0.539	0.617	0.460
	Explicit F1	0.896	0.897	0.887	0.907	0.911
	Invariant F1	0.577	0.587	0.554	0.628	0.507
Logit Diff	Explicit Logit Diff	-0.266	-0.260	-0.255	-0.165	-0.278
	Relative Logit Diff	-0.096	-0.086	-0.125	-0.011	-0.124

Appendix

Table 19: Semantically different questions, where time is presented at the end of the question. F1 scores, with logit probability difference for invariant and relative answers.

Figure 6: Qwen3-4b, higher layer nodes. Mean edge weights (y-axis), by years (x-axis) highlighting temporal anchors across a 2015-2024 period. Demonstrates how a variety of nodes regarding time have a larger (or lower) edge weight compared to other temporal nodes. Specifically L11F148703 (yellow), L10F125625 (orange), L16F45664 (pink) and L15F127829 (grey)

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.496	0.383	0.393	0.425	0.653
	Relative F1	0.584	0.560	0.469	0.525	0.493
	Explicit F1	0.988	0.967	0.969	0.981	0.965
	Invariant F1	0.464	0.408	0.365	0.393	0.599
Logit Diff	Explicit Logit Diff	-0.129	-0.146	-0.128	-0.114	-0.026
	Relative Logit Diff	-0.082	-0.078	-0.039	-0.038	-0.008

Appendix

Table 20: F1 scores, with logit probability difference for invariant and relative answers. LLaMA 3.2 1B, ChronosAlign ablation using common temporal nodes

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.003	0.003	0.002	0.005	0.014
	Relative F1	0.584	0.560	0.469	0.525	0.493
	Explicit F1	0.988	0.967	0.969	0.981	0.965
	Invariant F1	0.464	0.408	0.365	0.393	0.599
Logit Diff	Explicit Logit Diff	-0.318	-0.301	-0.236	-0.259	-0.177
	Relative Logit Diff	-0.264	-0.245	-0.144	-0.182	-0.129

Appendix

Table 21: F1 and cumulative probability scores for 2020-2024 given the top 100 (LLaMA 3.2 1B, ChronosAlign random invariant)

	Metric	2020	2021	2022	2023	2024
F1	Steer F1	0.089	0.136	0.154	0.109	0.033
	Relative F1	0.126	0.149	0.158	0.172	0.108
	Explicit F1	0.252	0.400	0.425	0.204	0.070
	Invariant F1	0.053	0.069	0.107	0.058	0.076
Logit Diff	Explicit Logit Diff	0.030	0.000	0.023	0.012	-0.069
	Relative Logit Diff	-0.025	-0.020	0.001	-0.011	-0.047

Appendix

Table 22: F1 scores, with logit probability difference for invariant and relative answers. LLaMA 3.2 1B, ChronosAlign steering using common to the year nodes

	Metric	2004	2015	2020	2021	2022
F1	Ablation F1	0.014	0.032	0.028	0.032	0.031
	Relative F1	0.016	0.029	0.025	0.032	0.021
	Explicit F1	0.022	0.032	0.027	0.048	0.036
Logit Diff	Explicit Logit Diff	-0.014	-0.030	-0.057	-0.061	-0.055
Logit Diff	Relative Logit Diff	-0.009	-0.015	-0.012	-0.010	-0.007
Probs. Change	Prior rve	-0.028	-0.013	-0.028	-0.032	-0.029

Appendix

Table 23: F1 scores, with logit probability difference for invariant and relative answers. LLaMA 3.2 1B, Taqa ablation using common temporal nodes

Dataset	Ablation F1	Original F1	Original Answer Logit Diff
Commonsense	0.304	0.310	0.003
GSM8K	0.045	0.043	-0.001

Appendix

Table 24: F1 Score and logit probabilities for ablation results on time-invariant datasets, LLaMA 3.2 1B

	Metric	2020	2021	2022	2023	2024
F1	Ablation F1	0.497	0.332	0.337	0.347	0.569
	Relative F1	0.584	0.560	0.469	0.525	0.493
	Explicit F1	0.988	0.967	0.969	0.981	0.965
	Invariant F1	0.464	0.408	0.365	0.393	0.599
Logit Diff	Explicit Logit Diff	-0.119	-0.121	-0.109	-0.105	-0.028
	Relative Logit Diff	-0.073	-0.050	-0.037	-0.033	-0.013

Appendix

Table 25: Semantically different questions, where time is presented at the end of the question. F1 scores, with logit probability difference for invariant and relative answers. LLaMA 3.2 1B, ChronosAlign dataset.

Figure 7: LLaMA 3.2 1B, layers between 4 and 8. Mean edge weights (y-axis), by years (x-axis) highlighting temporal anchors across a temporal continuum. Demonstrates a convoluted representation of time nodes. Disentangling temporal representations for LLaMA 3.2 1B is difficult

Figure 8: Gemma 2 2B activation heatmap for chrono-semantic specialised features. Note that the highest values for each feature span one to two years with no activation for the invariant data (last column)

Figure 9: Qwen3-4b activation heatmap for chrono-semantic specialised features. The highest activations occur for the invariant column, with varying activations across yearly representations. This graph is the inverse of Gemma 2 2B, whereby the darker colours highlight which years are to be aligned to.

Figure 10: LLaMA 3.2 1B activation heatmap for chrono-semantic specialised features. The activation pattern is jagged, and not as smooth as Gemma 2 2B and Qwen3 4b across any span of time, potentially highlighting the lack of temporal consideration, or the limited ability of the transcoder to pick up on temporal features.

Year	Ablation	Relative	Explicit	Explicit Logit Diff	Relative Logit Diff	Prior rve	New rve
2020	0.562	0.514	0.908	-0.078	-0.023	-0.052	0.003
2021	0.630	0.477	0.914	-0.060	-0.006	-0.060	-0.006
2022	0.620	0.425	0.917	-0.066	-0.011	-0.052	0.003
2023	0.614	0.487	0.925	-0.077	-0.003	-0.072	0.001
2024	0.672	0.549	0.903	-0.015	-0.004	-0.011	0.000

Appendix

Table 26: Gemma 2 2B, Top 5 contributing common temporal nodes.

	Overall Node Count		Higher Layer Node Count
Year	High Conf.	Low Conf.	High Conf.	Low Conf.
2020	56	37	6	1
2021	52	50	3	3
2022	44	46	5	3
2023	81	46	6	2
2024	106	64	4	1

Appendix

Table 27: Node counts by year comparing overall and high layer representations.

Figure 11: Simplified temporal representations for 2020–2024.

Figure 12: Original steering (blue), average activation (orange), subtractive (red). Demonstrates the improvement of the explicit answer logit probabilities with a scaled or selective approach, when compared to the original steering methodology

	Metric	2020	2021	2022	2023	2024
F1	Steer F1	0.189	0.197	0.142	0.194	0.202
	Relative F1	0.071	0.085	0.020	0.046	0.081
	Explicit F1	0.351	0.370	0.416	0.320	0.148
	Invariant F1	0.083	0.076	0.056	0.089	0.118
Logit Diff	Explicit Logit Diff	0.031	0.064	0.016	0.045	0.023
	Relative Logit Diff	-0.028	-0.043	-0.029	-0.033	-0.034

Appendix

Table 28: F1 scores, with logit probability difference for invariant and relative answers. Gemma 2 2B, ChronosAlign dataset using Average activation steering intervention with Chrono-semantic nodes at layer 12 and above.

	Metric	2020	2021	2022	2023	2024
F1	Steer F1	0.161	0.058	0.143	0.097	0.123
	Relative F1	0.071	0.085	0.020	0.046	0.081
	Explicit F1	0.351	0.370	0.416	0.320	0.148
	Invariant F1	0.083	0.076	0.056	0.089	0.118
Logit Diff	Explicit Logit Diff	0.036	0.015	0.042	0.012	-0.000
	Relative Logit Diff	-0.013	-0.019	-0.027	-0.018	-0.018

Appendix

Table 29: F1 scores, with logit probability difference for invariant and relative answers. Gemma 2 2B, ChronosAlign dataset using Subtractive steering intervention with Chrono-semantic nodes, at layer 12 and above.

Figure 13: Gemma 2 sample EAP-IG circuit. Note the layer 7 head 4 attention, and M14 nodes

Figure 14: Gemma 2 2B SAE circuit tracing example showing residual stream and MLP feature interactions.

Temporal Reasoning in Language Models

Temporal reasoning in language models studies how models represent, order, update, and apply facts and events across time.

See all 99 papers

The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations
May 9, 2026Rania Elbadry, Ahmed Heakl, Fan Zhang +4LLM Interpretability LLM Reliability
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · INSAIT, Sofia University
PRISM Edit: One Vector for All Temporal Answers
Jul 13, 2026Chen Huang, Qi Zheng, Ruiqin Zheng +2Temporal Reasoning in Language Models Model Editing
Tsinghua University · ByteDance
Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models
Oct 26, 2025Anooshka Bajaj, Deven Mahesh Mistry, Sahaj Singh Maini +2In-Context Learning Induction Heads
Department of Computer Science Indiana University Bloomington

May 9, 2026J/K move · Enter open · S save

The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations

Rania Elbadry, Ahmed Heakl, Fan Zhang, Dani Bouch+3

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · INSAIT, Sofia University

LLM Interpretability LLM Reliability Factual Knowledge in Language Models Drifting Models Language Model Probing

Open paper AskPDF arXiv

Large language models confidently produce outdated answers, and no existing method can detect them. We show this is not an engineering failure but a structural one: temporal drift, whether a stored fact has changed since training, is encoded as a direction in the residual stream geometrically orthogonal to both correctness and uncertainty. Any method operating on correctness or uncertainty signals is therefore blind to drift by construction. We verify this across six instruction-tuned models. A linear probe trained directly on drift labels achieves AUROC

0.83

0.95

; methods based on token entropy, semantic entropy, CCS, and SAPLMA all remain near chance (

0.49

0.57

). Five tests confirm the geometric orthogonality: weight cosines (

|\cos| \leq 0.14

), score correlations (

|r| \leq 0.20

), bidirectional null-space projection (

|Δ| \leq 0.008

), iterative null-space projection with

k{=}10

, and difference-of-means dissociation. Mechanistically, the MLP retrieval circuit produces identical dynamics for stale recall and confabulation (

r > 0.81

, six models), explaining why output confidence cannot separate them. A cross-cutoff experiment holds inputs constant and varies only the model: the probe fires on the model whose training predates the fact's transition and stays silent otherwise (

P(A{>}B) = 0.975

0.998

, twelve model pairs), confirming it reads model-internal knowledge state rather than input properties. Our code and datasets will be publicly released.

Abstract

Temporal Reasoning in Language Models

Explore similar work

The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations

PRISM Edit: One Vector for All Temporal Answers

Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models