SchemaMem: Schema-Indexed Recurrent Memory for Delayed State Retrieval

Sungwoo Goo; Hwi-yeol Yun; Sangkeun Jung

cs.LGSep 27, 2026

SchemaMem: Schema-Indexed Recurrent Memory for Delayed State Retrieval

Authors: Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung

Organizations: College of Pharmacy, Chungnam National University Daejeon, Republic of Korea · Department of Computer Science & Engineering, Chungnam National University Daejeon, Republic of Korea

Abstract

Attention provides direct access to past representations, but retaining an ever-growing history is costly. Recurrent models bound persistent state, yet must preserve selected information while processing subsequent inputs. We introduce SchemaMem, an attention-based recurrent memory architecture combining chunk-local attention with a persistent, schema-indexed phase state. Learned schema embeddings provide a shared representational reference for reading and writing. Reads use the current state, whereas writes use the layer input and static schema embeddings, excluding direct feedback from that layer's own state. Chunk-boundary commits aggregate bounded phase increments through forward computation. The same parameters also support full-history attention training before and during recurrent training. We studied selective updates, preservation, and delayed retrieval in a controlled address--value task, comparing three-layer models with approximately matched parameter counts and persistent-state dimensions. Across nine address/value settings and three training seeds, SchemaMem has higher mean written-value retention at four times the maximum training delay than both baselines, which are trained toward a higher in-range accuracy target. Updated-value recovery favors SchemaMem in all nine settings against Mamba-3 and seven against Gated DeltaNet. Defaults consistently favor Gated DeltaNet over SchemaMem at that delay, and SchemaMem requires substantially more optimization steps. These results identify a promising retention--optimization trade-off in schema-indexed recurrence.

Figures & tables

Model	Parameters	Persistent elements	Final target
SchemaMem	54,336	16,896	95%
Mamba-3	55,128	16,752	99%
Gated DeltaNet	55,260	17,280	99%

Table 1: Resource counts for A3V3. A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). All models have three layers and hidden width 32. Across A

a

v

settings, only the input projection

(a+v+1)\to 32

and output bit classifier

32\to v

change dimensions; backbone dimensions and persistent-state sizes remain fixed. The extra input bit indicates write versus NULL query. Persistent elements exclude temporary buffers and local attention storage. Final target is the final-stage validation exact-value accuracy threshold, required for every evaluated semantic group on two consecutive checks; it is not the long-delay evaluation accuracy.

	Retained-I			Updated-U
Setting	SM	M3	GDN	SM	M3	GDN
A3V3	78.9	21.3	21.6	74.6	26.9	27.1
A3V4	71.2	12.5	30.2	66.3	20.5	34.0
A3V5	70.2	10.9	25.0	65.8	13.9	20.9
A4V3	78.1	28.8	31.5	77.7	38.7	36.4
A4V4	79.3	12.6	33.0	75.6	20.5	42.0

Table 2: All nine long-delay comparisons: mean exact accuracy (%) at delay 1,536. A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). SM: SchemaMem; M3: Mamba-3; GDN: Gated DeltaNet. I: retained after disjoint writes; U: overwritten values. Complete means, SDs, default predictions, and per-seed results are in the appendix.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Model	Retained-R	Retained-I	Updated	Default
A3V3	SM	$81.4\,\pm\,5.3$	$78.9\,\pm\,3.6$	$74.6\,\pm\,5.8$	$71.3\,\pm\,10.6$
A3V3	M3	$24.3\,\pm\,7.9$	$21.3\,\pm\,5.6$	$26.9\,\pm\,10.0$	$76.0\,\pm\,19.3$
A3V3	GDN	$21.4\,\pm\,21.6$	$21.6\,\pm\,18.6$	$27.1\,\pm\,22.7$	$86.5\,\pm\,23.3$
A3V4	SM	$71.9\,\pm\,9.3$	$71.2\,\pm\,9.1$	$66.3\,\pm\,8.8$	$71.7\,\pm\,13.5$
A3V4	M3	$15.0\,\pm\,3.7$	$12.5\,\pm\,3.7$	$20.5\,\pm\,2.5$	$90.3\,\pm\,3.8$
A3V4	GDN	$29.7\,\pm\,8.7$	$30.2\,\pm\,5.8$	$34.0\,\pm\,8.3$	$91.3\,\pm\,6.1$

Appendix

Table 3: Delay 1,536: exact-value accuracy (%, mean

\pm

sample SD, three seeds). A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). SM: SchemaMem; M3: Mamba-3; GDN: Gated DeltaNet. Retained-R is the retention-only group; Retained-I is the retained group after disjoint writes; Updated and Default are from the overwrite scenario. All 81 endpoints are included.

Setting	SchemaMem	Mamba-3	Gated DeltaNet
A3V3	88,667	3,917	3,750
A3V4	58,000	4,083	4,333
A3V5	58,417	3,833	5,583
A4V3	65,667	4,917	6,667
A4V4	36,917	4,833	6,000
A4V5	63,000	5,083	5,917

Appendix

Table 4: Total optimizer steps (transaction plus maintenance), mean over three seeds. A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). SchemaMem totals include full-history training; these are not matched-FLOP budgets.

Setting	Model	Seed	Tx	Maint	I (%)	U (%)	Pass
A3V3	SM	42	8,750	42,250	76.37	68.16	yes
A3V3	SM	43	14,500	97,000	83.01	76.46	yes
A3V3	SM	44	18,000	85,500	77.34	79.30	yes
A3V3	M3	42	1,500	2,000	25.78	35.45	yes
A3V3	M3	43	1,500	2,250	14.94	15.92	yes
A3V3	M3	44	1,500	3,000	23.05	29.30	yes

Appendix

Table 5: Per-seed endpoints. A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). SM: SchemaMem; M3: Mamba-3; GDN: Gated DeltaNet. Tx/Maint: transaction/maintenance optimizer steps. I/U: exact accuracy at delay 1,536 for retained interference/overwritten values. Pass refers to the configured final target, not to long-delay accuracy.

Setting	Model	Scenario	Retained	Updated	Default
A3V3	SM	retention	$81.4\,\pm\,5.3$	—	$71.7\,\pm\,10.2$
A3V3	SM	interference	$78.9\,\pm\,3.6$	$84.8\,\pm\,4.6$	$71.1\,\pm\,10.8$
A3V3	SM	overwrite	$80.2\,\pm\,1.4$	$74.6\,\pm\,5.8$	$71.3\,\pm\,10.6$
A3V3	M3	retention	$24.3\,\pm\,7.9$	—	$75.1\,\pm\,20.1$
A3V3	M3	interference	$21.3\,\pm\,5.6$	$28.9\,\pm\,10.6$	$75.3\,\pm\,19.9$
A3V3	M3	overwrite	$20.9\,\pm\,6.6$	$26.9\,\pm\,10.0$	$76.0\,\pm\,19.3$

Appendix

Table 6: All maintenance scenario groups at delay 1,536: exact-value accuracy, mean

\pm

sample SD (%). A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). SM: SchemaMem; M3: Mamba-3; GDN: Gated DeltaNet. A dash means the group is absent by construction.

Setting	State	Retained-R	Retained-I	Updated	Default
A3V3	Normal	$99.1\,\pm\,1.4$	$99.1\,\pm\,1.1$	$95.3\,\pm\,0.3$	$99.9\,\pm\,0.1$
A3V3	$S=0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$100.0\,\pm\,0.0$
A3V4	Normal	$99.4\,\pm\,0.7$	$99.1\,\pm\,1.5$	$96.1\,\pm\,1.3$	$99.9\,\pm\,0.0$
A3V4	$S=0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$100.0\,\pm\,0.0$
A3V5	Normal	$99.9\,\pm\,0.2$	$99.9\,\pm\,0.2$	$96.4\,\pm\,1.0$	$100.0\,\pm\,0.1$
A3V5	$S=0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$100.0\,\pm\,0.0$

Appendix

Table 7: SchemaMem dynamic-state ablation at delay 384: exact-value accuracy (%, mean

\pm

sample SD over three training seeds). A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). Normal and

S=0

use the same checkpoints and episodes.

S=0

clears all layers’ dynamic states after every commit. Retained-R: retention-only; Retained-I: retained after disjoint writes; Updated and Default: overwrite scenario.

Setting	State	Retained-R	Retained-I	Updated	Default
A3V3	Normal	$81.4\,\pm\,5.3$	$78.9\,\pm\,3.6$	$74.6\,\pm\,5.8$	$71.3\,\pm\,10.6$
A3V3	$S=0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$100.0\,\pm\,0.0$
A3V4	Normal	$71.9\,\pm\,9.3$	$71.2\,\pm\,9.1$	$66.3\,\pm\,8.8$	$71.7\,\pm\,13.5$
A3V4	$S=0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$100.0\,\pm\,0.0$
A3V5	Normal	$72.5\,\pm\,25.3$	$70.2\,\pm\,27.5$	$65.8\,\pm\,21.8$	$64.8\,\pm\,21.7$
A3V5	$S=0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$0.0\,\pm\,0.0$	$100.0\,\pm\,0.0$

Appendix

Table 8: SchemaMem dynamic-state ablation at delay 1,536: exact-value accuracy (%, mean

\pm

sample SD over three training seeds). A

a

v

denotes

a

-bit addresses (

2^{a}

addresses) and

v

-bit values (

2^{v}

possible values). Normal and

S=0

use the same checkpoints and episodes.

S=0

clears all layers’ dynamic states after every commit. Retained-R: retention-only; Retained-I: retained after disjoint writes; Updated and Default: overwrite scenario.

Explore similar work

From Unstructured Recall to Schema-Grounded Memory: Reliable AI Memory via Iterative, Schema-Aware Extraction
Apr 30, 2026Alex Petrov, Alexander Gusak, Denis Mukha +1Agentic Memory Semantic Memory
1xmemory
Kathleen Remembers: Length-Invariant One-Shot Recall Without Attention
Aug 31, 2026George FountzoulasFactual Recall Recurrent Model
Department of Computer Engineering & Informatics Frederick University, Nicosia, Cyprus
MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
Aug 12, 2026Ming Zhang, Kaisen Yang, Shu Yu +6Efficient Long-Context Inference Recurrent Model
Shanghai AI Laboratory · Tsinghua University · Fudan University

Apr 30, 2026, cs.AIJ/K move · Enter open · S save

From Unstructured Recall to Schema-Grounded Memory: Reliable AI Memory via Iterative, Schema-Aware Extraction

Alex Petrov, Alexander Gusak, Denis Mukha, Dima Korolev

1xmemory

Agentic Memory Semantic Memory Factual Recall Peak Memory Iterative

Open paper AskPDF arXiv

Persistent AI memory is often reduced to a retrieval problem: store prior interactions as text, embed them, and ask the model to recover relevant context later. This design is useful for thematic recall, but it is mismatched to the kinds of memory that agents need in production: exact facts, current state, updates and deletions, aggregation, relations, negative queries, and explicit unknowns. These operations require memory to behave less like search and more like a system of record. This paper argues that reliable external AI memory must be schema-grounded. Schemas define what must be remembered, what may be ignored, and which values must never be inferred. We present an iterative, schema-aware write path that decomposes memory ingestion into object detection, field detection, and field-value extraction, with validation gates, local retries, and stateful prompt control. The result shifts interpretation from the read path to the write path: reads become constrained queries over verified records rather than repeated inference over retrieved prose. We evaluate this design on structured extraction and end-to-end memory benchmarks. On the extraction benchmark, the judge-in-the-loop configuration reaches 90.42% object-level accuracy and 62.67% output accuracy, above all tested frontier structured-output baselines. On our end-to-end memory benchmark, xmemory reaches 97.10% F1, compared with 80.16%-87.24% across the third-party baselines. On the application-level task, xmemory reaches 95.2% accuracy, outperforming specialised memory systems, code-generated Markdown harnesses, and customer-facing frontier-model application harnesses. The results show that, for memory workloads requiring stable facts and stateful computation, architecture matters more than retrieval scale or model strength alone.

Abstract

Explore similar work

From Unstructured Recall to Schema-Grounded Memory: Reliable AI Memory via Iterative, Schema-Aware Extraction

Kathleen Remembers: Length-Invariant One-Shot Recall Without Attention

MARCH: Scaling Recurrent Memory with Content-Routed State Anchors