Time and Space: A Dual-Axis, Evidence-Grounded Context Substrate for Personal Agents
Abstract
We argue that a binding constraint for personal agents is not model capacity but the form in which context is represented: the agent does not know which entity a phrase denotes, when a fact held, or which source record licenses a claim. We call the missing representation spatio-temporal context (STC) and define it on two axes. The temporal axis carries event time, validity intervals and ingestion time. The spatial axis is a relational space in which entities are points, typed relations are edges, and distance is semantic rather than geographic; physical places are one entity type among many. Every relation is anchored to immutable source records, and every answer must be closed over those records. We formalise the substrate and six invariants, a context operator that composes temporal intersection with relational expansion, an ambiguity-preserving resolution state machine, a conflict detector that converts mutually exclusive facts into user adjudication rather than silent overwrite, and a four-regime consumption taxonomy spanning query, insight, triage and proactive reminder. We describe a local-first prototype of the retrieval layer built on SQLite with on-device embeddings and hybrid lexical-dense recall, report measurements from a deployed store of 341 documents, 340,079 chunks and 190,271 embedded vectors, and specify, but do not yet implement, the analytic layer on which the framework depends. We close with an evaluation protocol with per-invariant metrics for evidence coverage, temporal correctness, conflict precision and interruption cost. The proposal targets the representation of context rather than the volume of it.
Figures & tables
| System | Time model | Unit of "space" | Provenance | Identity ambiguity | Scope | Beyond question answering |
|---|---|---|---|---|---|---|
| MemGPT (Packer et al., 2023) | Memory timestamps | None (tiered text) | Partial | Not modelled | Per agent | No |
| Generative Agents (Park et al., 2023) | Observation time | None (memory stream) | Observation | Not modelled | Single world | No |
| Zep / Graphiti (Rasmussen et al., 2025) | Valid and transaction time | Entity graph | Episodes | Entity nodes | Tenant | No |
| Bitemporal store (Niksarli & Baheti, 2026) | Valid and transaction time | Versioned graph | Versions | Entity nodes | Per conversation | No |
| SodaMem (Wan et al., 2026) | Mention, occurrence, validity | Fact-event graph | Mandatory spans | Entity nodes | User | No |
| REMem (Shu et al., 2026) | Time-scoped facts and gists | Episodic graph | Gist and fact links | Not central | User | No |
| Invariant | Statement | Failure prevented |
|---|---|---|
| I1 Record primacy | Records are immutable and are the only ground truth; entities and relations are derived and rebuildable | Derived errors becoming indistinguishable from observations |
| I2 Multi-time | Event time, validity and ingestion time are stored separately and never collapsed into one timestamp | Confusing when something happened with when it was learned |
| I3 Provenance totality | Every relation cites at least one record; every quoted claim cites a span inside a record | Unsupported assertions and un-auditable answers |
| I4 Open-world validity | An open right endpoint means "still true as far as known", never "current" or "invalid" | Stale facts silently persisting or expiring |
| I5 Ambiguity preservation | A mention may map to a candidate set; resolution requires added evidence, not preference | Premature identity commitments that later evidence cannot repair |
| I6 Scope isolation | Facts do not cross scope boundaries during retrieval unless the user widens the scope | Plausible but incoherent answers assembled across domains |
| Stage | Operation | Output | Cost note | Status |
|---|---|---|---|---|
| 1 Decompose | Split the query into anchors, temporal focus and scope | Query frame | Linear in query length | Implemented, by heuristics |
| 2 Anchor resolution | Alias match plus dense top- , fused by reciprocal rank fusion | Candidate entities and mentions | Two index lookups | Implemented |
| 3 Relational expansion | -hop neighbourhood over entity and relation edges | Candidate relation set | Bounded by degree and | Proposed |
| 4 Temporal filter | Interval intersection with the focus | Time-valid relation set | Interval index | Proposed |
| 5 Evidence assembly | Replace relations by their record pointers and spans | Evidence bundle | Linear in selected relations | Proposed |
| 6 Answer | Generate with records only, verify spans, abstain on failure | Evidence-closed answer | Model-bound | Generation implemented, closure not verified |
| Regime | Question shape | Operator composition | Output | Failure guarded |
|---|---|---|---|---|
| Query | "When, where, with whom" | Anchor resolution, temporal filter, evidence assembly | Evidence-closed answer | Confident answers without support |
| Insight | "Is anything inconsistent or changing" | Relational expansion plus conflict detection over a window | Tension report or trend | Silent averaging of incompatible facts |
| Triage | "Does this deserve the user’s attention" | Scoped lookup plus policy evaluation | Silent handling, reply, or escalation | Notification flooding |
| Reminder | "What must be decided before a deadline" | Open-relation scan plus decision window | Proactive message with actions | Deadlines discovered too late |
| Table | Fields | Role |
|---|---|---|
| records | id, source_id, kind, content, happened_at, captured_at, scope, span | Immutable observations; adjudications and retractions are records of their own kind |
| entities | id, type, name, aliases, first_seen, embedding, status | Persistent things; candidate sets live in an association table keyed by mention |
| mentions | mention_id, record_id, surface, candidate_entity_id, evidence_record_id | Ambiguity as data rather than as a decision |
| relations | id, statement, participants, valid_from, valid_to, scope, decision_deadline, status, functional_key, embedding | Queryable facts with provenance pointers, validity, scope and the decision window |
| relation_records | relation_id, record_id, span, weight | The supporting set of each relation; invariant I3 is enforced here |
| adjudications | relation_a, relation_b, decision, decided_at, note, record_id | User decisions as records with provenance |
| Stage | Operation on the vignette | Result |
|---|---|---|
| Anchor resolution | "mom" yields two candidates: the speaker’s mother and the mother-in-law named on the photograph | Candidate set of size two; the transfer memo raises the posterior for one member but does not separate them |
| Background | The registration slip lists the patient and the speaker as accompanying person | A kinship statement that names the patient as the speaker’s mother, which separates the candidate set |
| Temporal filter | Focus is January | The slip survives as a one-granule interval on 12 January; the ticket is a point event on 28 January and falls outside |
| Conflict detection | Location is functional and exclusive | The accompaniment on 12 January and the travel on 28 January are sixteen days apart, so no conflict is raised |
| Evidence assembly | Relations are replaced by their records | Slip, transfer and chat message, each with its span |
| Answer | The question asks where the speaker was on 27 January | The answer states the 12 January facts and abstains on 27 January, citing the three records that support the kinship |
| Metric | Definition | Primary instrument |
|---|---|---|
| Citation coverage | Fraction of factual claims with an entailing span | Constructed claim-level annotations |
| Verifier accuracy | Agreement of the span-claim entailment check with human labels | Held-out span-claim pairs |
| Abstention rate | Fraction of answerable queries answered with a refusal | Annotated query set |
| Temporal accuracy | Correctness on queries requiring interval reasoning | LongMemEval temporal split plus constructed set |
| Resolution accuracy | Correct entity chosen when evidence separates candidates | Constructed ambiguity set |
| Conflict precision at budget | Precision of adjudications at a fixed rate of interruptions | Human labels on conflicting pairs |
LLM Context Management
Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.
See all 265 papers