Engram Injection: A Study of Temporal Validity in Parametric Language-Model Memory

Sam Siavoshian; Omar Ramadan; Amir K. Saeed; Benjamin A. Johnson; Benjamin M. Rodriguez

cs.AIOct 11, 2026

Engram Injection: A Study of Temporal Validity in Parametric Language-Model Memory

Authors: Sam Siavoshian, Omar Ramadan, Amir K. Saeed, Benjamin A. Johnson, Benjamin M. Rodriguez

Abstract

A stored fact may stop being valid when a user changes a preference or an event replaces an earlier one. We study whether a language model can learn these changes in its parameters and answer correctly from only a question and the current time. Engram Injection stores facts in LoRA adapters and supplies the clock through FiLM conditioning on a frozen Qwen2.5-3B-Instruct model. We compare it with additive clock conditioning, prompting with the full dated history, and retrieving four dated records. We train six matched adapters across three seeds after fixing a sampler that could admit test-age examples into training. The training audit reports no such examples among 127,760 evaluated training inputs. On 62 pairs of questions asked before and after an update, at an elapsed age of seven days excluded from training, both adapter methods answer zero pairs correctly under the primary blinded assistant review with order code 1 held fixed. Full-history prompting answers 59/62 pairs correctly, and retrieval answers 62/62. Separate experiments show that adapters can store facts and that removable memory units can support local deletion. Combining fact selection with time-based version selection remains unsuccessful. These results separate learning a fact from learning when to use it.

Figures & tables

Capability	Shared Parametric	Addressable Bank	Joint Bank
Prompt-free parametric recall	Demonstrated	Demonstrated	–
Clock-gated supersession	Diagnostic only	–	Failed
Semantic addressing	–	Demonstrated	Partial
Local structural deletion	–	Demonstrated	Demonstrated
Unified semantic + temporal routing	–	–	Failed

Table 1: Capabilities tested in each system. “Demonstrated” refers to a successful component test. “Diagnostic only” refers to answer preference at trained times. “Failed” marks an evaluated capability that did not succeed. “–” marks a capability not tested by that system.

Figure 1: Engram Injection architecture. (a) Dated updates are written into LoRA memory. (b) A frozen backbone receives the question and FiLM clock features. Clock conditioning applies to every decoder layer except the last. (c) Schematic old/new preference reversal; these bars are not measured accuracies.

Pairs	Training arm	Seeds	$\mathrm{new\_after}$	$\mathrm{old\_before}$	Recall
25	Time channel only	4	0.38	0.65	0.77
25	Order tag only	2	0.30	0.76	0.77
25	Margin only	2	0.82	0.72	0.81
25	Order + margin	10	0.79 [0.72,0.92]	0.88	0.87
100	Time channel only	2	0.675	0.38	0.63
100	Order + margin	5	0.948	0.99	0.945

Table 2: Temporal supersession across training arms and update-set sizes. The range is shown for the earlier ten-seed configuration.

Figure 2: Earlier answer-preference tests. (a) The target margin favors the old value before an update and the new value afterward. (b) Training-arm scores measure post-update likelihood preference;

n

is the number of seeds. (c) Frozen and shuffled controls lower this score. These earlier controls can also change the order code. Section 4.3 tests generated answers with the code fixed.

Benchmark	Metric	Base	Injected	Intervention
CounterFact	Efficacy	0.14	0.96–1.00	0.00–0.12
CounterFact	Fair-locality	0.87	0.80	0.81
CounterFact	Pre-update old preference	0.86	0.99	0.99
TOFU	Forget-set recall	0.01	0.98	0.01
TOFU	Retain-set recall	0.00	0.97	0.97

Table 3: External editing and deletion evaluation. CounterFact evaluates the clock-conditioned editing configuration; TOFU evaluates structural deletion in the separate addressable bank.

Size	Controlled flip	Recall	CounterFact efficacy
3B	0.813	0.89	1.00
7B	0.760	0.85	1.00
14B	0.747	0.86	1.00

Table 4: Selected configurations at three Qwen2.5-Instruct sizes. These runs are separate from the ten-seed experiment in Table 2 .

System	Recall (%)	Before (%)	After (%)	Both (%)	Tokens
P0	61.67	30.67	86.67	25.33	3087.3
P0-asof	62.08	89.33	92.00	89.33	1717.9
P1	61.25	10.67	89.33	9.33	148.0
P1-asof	62.50	88.00	92.00	88.00	145.8
Additive	97.92	77.33	36.00	25.33	6.0
FiLM	97.50	81.33	41.33	33.33	6.0

Table 5: Three-seed literal-match comparison at training endpoints. Before, after, and both use 75 pairs; recall uses 240 questions. Mean prompt tokens are descriptive query costs.

System	Before	After	Both (%)	Judge both
P0	33/55	48/55	54.5	35/55
P0-asof	49/55	49/55	85.5	53/55
P1	11/55	49/55	18.2	10/55
P1-asof	48/55	51/55	87.3	55/55
Additive	35/55	10/55	10.9	1/55
FiLM	43/55	14/55	12.7	2/55

Table 6: Earlier code-0 temporal-offset tests with unverified training/test separation. Literal scores are pooled counts out of 55 eligible pairs. Judge scores are supplementary, unvalidated semantic assessments of the same generated answers.

System	Code	Clock	Literal	Reviewed	Upper bound
P0-asof	–	asof	52/62	59/62	59/62
P1-asof	–	asof	54/62	62/62	62/62
Additive	1	frozen	12/62	0/62	1/62
Additive	1	live	11/62	0/62	1/62
Additive	1	shuffled	11/62	0/62	2/62
Additive	2	frozen	6/62	0/62	3/62

Table 7: Corrected held-out 7-day temporal-offset tests on stored facts. All entries are pooled counts out of 62 pairs. Strict blinded assistant review rejects ambiguity; upper bounds accept it. Both baselines are rerun on the same queries.

System	Seed 0	Seed 1	Seed 2	Pooled
FiLM	0/18	0/20	0/24	0/62
Additive	0/18	0/20	0/24	0/62
P0-asof	18/18	20/20	21/24	59/62
P1-asof	18/18	20/20	24/24	62/62

Table 8: Primary code-1 live-clock reviewed counts by seed. Both temporal answers must be correct.

Figure 3: Primary reviewed scores on 62 before/after pairs at a seven-day age excluded from training. Code 1 stays fixed across query times and clock controls. Both as-of baselines are rerun on these queries. With all adapter scores at zero, the controls cannot determine internal clock use.

	FiLM (%)		Additive (%)
Seed	Live	Frozen	Live	Frozen
0	64	24	32	4
1	56	12	56	16
2	0	0	52	8

Table 9: Literal pairs correct at both training endpoints with ordinal code one fixed. Each seed contains 25 pairs. Frozen controls hold the runtime clock constant.

Metric	Score
Semantic top-1	0.7292
Temporal top-1	0.4167
Joint attribute/version top-1	0.2188
Answer exact match	0.3681
Clock-shift success	0.0000
Deleted-version/attribute recall	0.0000

Table 10: Three-seed joint-bank replication; means across seeds. These are separate metrics from shared-adapter literal-span matching.

Figure 4: Attribute selection, version selection, and deletion. (a) The question selects an attribute. (b) The clock should select its valid version. (c) An earlier bank with 30 forget and 30 retain memories illustrates deletion, separately from TOFU and the joint-bank replication. The dashed panel shows the intended combination. Table 10 reports the joint system’s measured performance.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Arm	Intervention	Interpretation
P0	Facts as JSON in prompt	Recurring long-context prefill cost
P1	Retrieve dated chunks	Text-based temporal reasoning baseline
A1	Zero $\chi$ channel	Removes runtime clock contribution
Order only	Add order tag, no temporal margin	Tests whether order representation alone drives supersession
Margin only	Add temporal margin, no order tag	Tests whether validity supervision drives supersession
Shuffled time	Permute timestamps	Breaks fact–time alignment; $\mathrm{new\_after}=0.28$

Appendix

Table 11: Temporal controls and ablations.

Family	Question type	Intended diagnostic
T-ABS	Absolute anchoring	Calendar/elapsed categories from hidden clock state
T-REL	Relative age	Query time relative to stored write time
T-ORD	Ordering	Earlier/later relation between stored events
T-CONF	Recency conflict	Old/new resolution across supersession
T-AGE	Age/staleness	Coarse age and stale/current judgments

Appendix

Table 12: ChronoRecall-125 evaluation families. Evaluation questions omit timestamp text.

Setting	$\mathrm{new\_after}$	$\mathrm{old\_before}$	Recall
$\lambda=0.1$	0.84	1.00	0.90
$\lambda=0.5$	0.84	0.96	0.91

Appendix

Table 13: One-seed sensitivity checks for the temporal-margin coefficient.

Lever	Value	$\mathrm{new\_after}$
Learning rate	$1\times 10^{-4}$	0.68
Learning rate	$3\times 10^{-4}$	0.79
Learning rate	$1\times 10^{-3}$	0.88
LoRA rank	4	0.76
LoRA rank	8	0.79
LoRA rank	16	0.76

Appendix

Table 14: One-factor temporal sweeps.

Memory representation	Prompt tokens/query	Prefill time (s)
Facts in prompt	2442	0.362
Shared Parametric Memory	6	0.042

Appendix

Table 15: Measured query-time cost at 140 stored facts.

Temporal Reasoning in Language Models

Temporal reasoning in language models studies how models represent, order, update, and apply facts and events across time.

See all 98 papers

User as Engram: Internalizing Per-User Memory as Local Parametric Edits
Jun 17, 2026Bojie LiKnowledge Editing LLM Personalization
Pine AI
Right Knowledge, Wrong Answer: Test-Time Steering for Temporal Fact Conflicts in Open-Weight Language Models
Jun 18, 2026Elias Hossain, Sourav Saha, Umesh Chandra Biswas +1Knowledge Conflicts in Language Models Temporal Reasoning in Language Models
University of Central Florida · 2Mississippi State University
Can a Language Model Learn Facts Continually in Its Weights?
Jul 13, 2026Charles O'Neill, Max Kirkby, Jonathon Liu +1Knowledge Editing Continual Learning for LLMs
Baseten

Jun 17, 2026J/K move · Enter open · S save

User as Engram: Internalizing Per-User Memory as Local Parametric Edits

Bojie Li

Pine AI

Knowledge Editing LLM Personalization Memory-Augmented Language Models Persistent Memory for Language Models LLM Memory Personalized Language Models+2

Open paper AskPDF arXiv

Personal memory in a language model is two problems: content and reasoning skill. The brain keeps the two apart (a sparse, local engram in the hippocampus for each episode, a slow neocortex for the shared skills that interpret it), so a new fact need not overwrite everything else. Most personalization today keeps a user's facts outside the weights, in a natural-language memory file or a retrieval index. When facts are written into the model instead, the standard recipe is the per-user LoRA adapter, which does the opposite of the brain, folding content and skill into one global weight delta. Writing a user's facts as a LoRA contaminates text unrelated to them; writing the same facts as local Engram rows leaves it mathematically untouched, resulting in a roughly 33,000x smaller memory footprint. We therefore propose User as Engram: store a user's content as surgical edits to the hash-keyed memory table of an Engram model, and carry the reasoning skill in one shared adapter. This layered design matches per-user LoRA's direct recall while delivering 5.6x higher indirect-reasoning accuracy on average, and never makes a single user worse at reasoning than the untouched base. The edit is a glass box: writing a fact switches on its lookup at exactly the trigger, adds the value the answer needs, leaves every other position unchanged to the last bit, and fails if written into the wrong layer. Because different users' facts land in disjoint hash slots, their edits compose: many users live in one shared table at once, stacking additively and losslessly, where a per-user LoRA, a single global weight delta, admits only one. Upon retrieval, a per-user Engram table does not grow with the population the retriever must search, so past ~100 facts it overtakes a retrieval pipeline on a 2.5x larger model.

Abstract

Temporal Reasoning in Language Models

Explore similar work

User as Engram: Internalizing Per-User Memory as Local Parametric Edits

Right Knowledge, Wrong Answer: Test-Time Steering for Temporal Fact Conflicts in Open-Weight Language Models

Can a Language Model Learn Facts Continually in Its Weights?