We study whether geometric relationships between conversation turns, memory texts, and queries can predict answer correctness before generation. We evaluate four memory implementations with dense or hybrid retrieval and one observation-summary comparator. We embed these texts in a shared space. A logistic classifier uses 25 features describing query proximity, memory-conversation correspondence, and conversation structure. On LongMemEval-small, pooled AUROC is 0.640 over 2,367 question-system pairs, below a question-difficulty baseline of 0.655. Performance varies by question type: geometry reaches 0.841 on single-session-assistant questions, but system identity alone reaches 0.826. After controlling for difficulty and identity, geometry adds no statistically established pooled improvement (delta = +0.002, 95% CI [-0.010, 0.014]). A fixed-store context-selection experiment finds higher accuracy with proximity-ranked memories than with random subsets, consistent with ordinary retrieval relevance. On LoCoMo, nearest-turn similarity has modest predictive value for single-hop questions and near-chance estimates for several other categories. Training uses correctness labels; prediction uses pre-answer features and no additional LLM generation call. The results characterize associations between memory representations and correctness, rather than establish a reliable deployment diagnostic. In particular, embedding proximity does not verify fact coverage or distinguish memory-construction, retrieval, and generation errors.
Figures & tables
Figure 1: Conversation turns, memories, and queries for two selected examples. The correctly answered query is closer to memory points than the incorrectly answered query. The t-SNE projection is illustrative; all measurements use original-space cosine distance. The examples do not establish fact coverage or the cause of an error.
Question type
n
AUROC
95% CI
% wrong
single-session-assistant
265
0.841
[0.802,0.881]
27.5
temporal-reasoning
623
0.628
[0.567,0.683]
27.8
single-session-user
333
0.607
[0.478,0.762]
0 8.1
single-session-preference
137
0.567
[0.485,0.659]
48.2
multi-session
639
0.566
[0.514,0.618]
33.3
knowledge-update
370
0.542
[0.459,0.617]
21.1
Table 1: Geometry AUROC by type, with separately fitted classifiers and question-group bootstrap intervals. n counts question–system pairs. These results do not classify the causes of individual errors.
Figure 2: AUROC within each type (five implementations). Bars show full geometry; diamonds show a system-identity-only classifier under the same protocol. Intervals are question-group bootstrap 95% intervals conditional on fixed out-of-fold predictions.
Question type
memory–conv. correspondence
query reachability
full geometry
single-session-assistant
0.843
0.825
0.841
single-session-preference
0.605
0.478
0.567
temporal-reasoning
0.590
0.612
0.628
multi-session
0.562
0.517
0.566
knowledge-update
0.547
0.512
0.542
single-session-user ∗
0.645
0.475
0.607
Table 2: Feature-block AUROCs under grouped CV. Full geometry includes both blocks and five conversation covariates. Paired pooled comparisons are reported in the text; per-type block comparisons are point estimates. ∗ Single-session-user has few errors and substantial uncertainty.
Question type
correct
wrong
gap
AUROC
single-session-assistant
0.377
0.618
+ 0.241
0.841
single-session-user
0.533
0.579
+ 0.046
0.607
temporal-reasoning
0.526
0.570
+ 0.044
0.628
multi-session
0.534
0.567
+ 0.033
0.566
knowledge-update
0.456
0.454
− 0.003
0.542
single-session-preference
0.570
0.541
− 0.029
0.567
Table 3: Mean nearest-memory cosine distances by judged correctness. AUROC is from the full per-type classifier in Table 1 . The large-gap category has the highest AUROC, but the smaller-gap categories do not show a precise monotonic ordering.
Question type
# fail
answer-distant
query-distant
neither
single-session-assistant
0 73
62%
0 5%
33%
multi-session
213
37%
21%
43%
knowledge-update
0 78
31%
0 8%
62%
temporal-reasoning
173
25%
13%
61%
single-session-user
0 27
19%
19%
63%
single-session-preference
0 66
0 0%
0 3%
97%
Table 4: Reference-answer proximity among 630 failed rows. Thresholds are the 75th percentiles among correct rows. These are proxy bins, rather than verified extraction, retrieval, or reasoning failure rates. Percentages are rounded independently.
Memories shown to the model
gpt-4o-mini
gpt-4o
Farthest-15 (cosine floor)
0.068
0.060
Random-15 (5 seeds)
0.114
0.091
Closest-15 (pure cosine)
0.752
0.790
Native-15 (system’s own retriever)
0.738
0.822
Table 5: Interventional accuracy when the answer model is given different memory subsets from the same store (Ext-Hybrid, K=15 , two answer models).
LoCoMo category
n
% wrong
gap
AUROC
single-hop
841
17.7
+ 0.026
0.603
adversarial (no answer)
446
62.6
+ 0.032
0.612
multi-hop
282
43.3
− 0.001
0.520
open-domain
96
65.6
− 0.003
0.487
temporal
321
81.3
− 0.007
0.468
Table 6: LoCoMo category results for a fixed nearest-turn similarity score. n counts questions. Gaps use full-context answer correctness. These descriptive point estimates have no category confidence intervals.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Family / feature
Definition
Correspondence: coverage (7)
cov_mean_nn , cov_max_nn
mean / worst nearest-memory distance over turns
cov_hausdorff
Hausdorff (T,M)
cov_mean_session , cov_worst_session
mean / worst per-session coverage
cov_at_eps_p10 , cov_at_eps_p5
fraction of turns with a memory within εp
Correspondence: alignment (2)
Appendix
Table 7: The 25 scalar features, grouped by the two relationships of §1.