ATANT: An Evaluation Framework for AI Continuity

Samuel Sameer Tanguturi

cs.AIApr 8, 2026

ATANT: An Evaluation Framework for AI Continuity

Authors: Samuel Sameer Tanguturi

Organizations: Kenotic Labs

Abstract

We present ATANT (Automated Test for Acceptance of Narrative Truth), an open evaluation framework for measuring continuity in AI systems: the ability to persist, update, disambiguate, and reconstruct meaningful context across time. While the AI industry has produced memory components (RAG pipelines, vector databases, long context windows, profile layers), no published framework formally defines or measures whether these components produce genuine continuity. We define continuity as a system property with 7 required properties, introduce a 10-checkpoint evaluation methodology that operates without an LLM in the evaluation loop, and present a narrative test corpus of 250 stories comprising 1,835 verification questions across 6 life domains. We evaluate a reference implementation across 5 test suite iterations, progressing from 58% (legacy architecture) to 100% in isolated mode (250 stories) and 100% in 50-story cumulative mode, with 96% at 250-story cumulative scale. The cumulative result is the primary measure: when 250 distinct life narratives coexist in the same database, the system must retrieve the correct fact for the correct context without cross-contamination. ATANT is system-agnostic, model-independent, and designed as a sequenced methodology for building and validating continuity systems. The framework specification, example stories, and evaluation protocol are available at https://github.com/Kenotic-Labs/ATANT. The full 250-story corpus will be released incrementally.

Figures & tables

#	Property	Testable Requirement
1	Persistence Beyond Session	After ingesting facts, terminate and restart the process. All facts must be retrievable with identical accuracy.
2	Update Handling	Ingest a fact, then an update. The system must return the current state and distinguish it from the previous state.
3	Temporal Ordering	Ingest facts with temporal references. The system must return correctly resolved dates, sequencing, and status.
4	Disambiguation	Ingest overlapping narratives. The system must return correct facts for the correct narrative without cross-contamination.
5	Reconstruction	After multi-turn ingestion, the system must retrieve connected facts sufficient to reconstruct a situation, not isolated fragments.
6	Model Independence	Ingest with one model (or none). Retrieve with another. Accuracy must not degrade.

Table 1: The 7 Required Properties of Continuity

CP	Path	Name	Pass Criteria
1	Write	Input Classification	Classification matches expected type
2	Write	Fact Extraction & Storage	All expected keywords in storage
3	Write	Predictive Indexing	$\geq$ 1 predicted query per stored fact
4	Write	Type Tagging	Type tags match expected categories
5	Read	Query Classification	Query type matches expected
6	Read	Structural Matching	Correct fact in top- $k$ candidates

Table 2: ATANT Checkpoint System

Level	Requirement	What It Proves
ATANT-Core	50 stories, isolated	Basic continuity across 6 life domains
ATANT-Stress	250 stories, isolated	Continuity generalizes to novel patterns
ATANT-Cumulative	50 stories, cumulative	Disambiguation when narratives coexist
ATANT-Scale	250 stories, cumulative	Disambiguation at scale

Table 3: ATANT Compliance Levels

Phase	Stories	Questions	Purpose
Phase 1 (Core)	50	304	6 life domains
Phase 2 Round 2	50	367	Generalization
Phase 2 Round 3	50	386	Novel patterns
Phase 2 Round 4	50	380	Edge cases
Phase 2 Round 5	50	398	Adversarial
Total	250	1,835

Table 4: ATANT Corpus Statistics

Suite	Date	Stories	Questions	CP8 Rate	Key Change
Legacy	Jan 2026	50	N/A	58%	Legacy scoring (with LLM)
Legacy+	Feb 2026	50	N/A	72%	Tuning gains
Legacy++	Feb 2026	50	N/A	58%	Regression from over-tuning
1.0	Mar 8	50	304/304	100%	New architecture (no LLM)
1.1	Mar 9	100	671/671	100%	Stress round 2
1.2	Mar 10	150	1,057/1,057	100%	Stress round 3

Table 5: Reference Implementation Progression

Mode	Stories	Questions	CP8 Rate
Isolated (250)	250/250	1,835/1,835	100.0%
Cumulative (50)	50/50	304/304	100.0%
Cumulative (250)	$\sim$ 210/250	1,761/1,835	96.0%

Table 6: Cumulative Mode Results

Level	Status	Tier
ATANT-Core	Pass	Gold
ATANT-Stress	Pass	Gold
ATANT-Cumulative	Pass	Gold
ATANT-Scale	In progress	Silver (96%)

Table 7: Reference Implementation Compliance

LLM Memory

Large language model memory studies how models store, retrieve, update, and forget information across prompts, tasks, and interactions.

See all 95 papers

ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks
Apr 13, 2026Samuel Sameer TanguturiMemory Agent Benchmarks Agent Benchmarks
Kenotic Labs
The Continuity Layer: Why Intelligence Needs an Architecture for What It Carries Forward
Apr 19, 2026Samuel Sameer TanguturiMemory-Augmented Language Models Persistent Memory for Language Models
Kenotic Labs
Persistent Identity in AI Agents: A Multi-Anchor Architecture for Resilient Memory and Continuity
Mar 2, 2026Prahlad G. MenonAgent Memory Persistent Memory for Language Models
The Menon Lab

Apr 13, 2026J/K move · Enter open · S save

ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks

Samuel Sameer Tanguturi

Kenotic Labs

Memory Agent Benchmarks Agent Benchmarks Long-Context Language Model Evaluation

Open paper AskPDF arXiv

ATANT v1.0 (arXiv:2604.06710) defined continuity as a system property with 7 required properties and introduced a 10-checkpoint, LLM-free evaluation methodology validated on a 250-story corpus. Since publication, a recurring reviewer and practitioner question has concerned not the framework itself but its relationship to a wider set of memory evaluations: LOCOMO, LongMemEval, BEAM, MemoryBench, Zep's evaluation suite, Letta/MemGPT's evaluations, and RULER. This companion paper, v1.1, does not modify the v1.0 standard. It closes a related-work gap that v1.0 left brief under page limits. We show by structural analysis that none of these benchmarks measures continuity as defined in v1.0: of the 7 required properties, the median existing eval covers 1 property, the mean covers 0.43 when partial credit is scored at 0.5, and no eval covers more than 2. We provide a cell-by-cell property-coverage matrix, identify methodological defects specific to each benchmark (including an empty-gold scoring bug in the LOCOMO reference implementation that renders 23% of its corpus unscorable by construction), and publish our reference implementation's LOCOMO score (8.8%) alongside the structural reason that number is uninformative about continuity. We publish our 8.8% LOCOMO score alongside our 96% ATANT cumulative-scale score as a calibration pair: the 87-point divergence is evidence that the two benchmarks measure different properties, not that one system is an order of magnitude better than another. The position v1.1 takes is not adversarial: each benchmark measures a real capability. The claim is that none of them can adjudicate continuity, and conflating them with continuity evaluation has led the field to under-invest in the properties v1.0 names.

Abstract

LLM Memory

Explore similar work

ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks

The Continuity Layer: Why Intelligence Needs an Architecture for What It Carries Forward

Persistent Identity in AI Agents: A Multi-Anchor Architecture for Resilient Memory and Continuity