Fidelity Before Structure: Verbatim Chunks Beat Lossy Artifact Extraction in Long-Conversation LLM Memory
Organizations: Hawaii Pacific University
Abstract
A growing class of conversational-memory systems compresses dialogue history into structured artifacts (extracted facts, decisions, or events). The premise is that distilled structure retrieves better than raw text. We test this premise with a controlled ablation. Within one fixed retrieval--rerank--reasoning pipeline, we swap only the stored representation (LLM-extracted typed artifacts versus verbatim conversation chunks) and hold the model, retriever, reranker, and judge constant. Verbatim chunks win by 15.9 points on LoCoMo (43.9% vs. 28.0%) and by 22.0 points on LongMemEval-S (67.4% vs. 45.4%). A semantic graph does not recover the gap, and six confound controls reproduce the effect. The mechanism is lossy distillation rather than structure per se. Accuracy tracks how much source text survives in the store, and the extracted-artifact pipeline does not beat naive RAG in overall accuracy (chunks do abstain worse; see Limitations). For the extraction designs we test, structured memory should augment verbatim text rather than replace it. Adding artifacts alongside chunks costs no detected accuracy, while substituting them forfeits the gap. Code and data are available at https://github.com/tao-hpu/cog-canvas.
Figures & tables
| Stored representation | Overall | Multi-hop | Temporal | Open-domain |
|---|---|---|---|---|
| Reference ceiling (no compression; conversations fit the context window) | ||||
| Full context (entire conversation in prompt) | 59.9 | 57.4 | 62.3 | 59.4 |
| Verbatim conversation chunks | ||||
| Chunks, no graph | 43.9 | 35.5 | 50.2 | 47.9 |
| Chunks graph | 43.1 | 36.5 | 47.7 | 46.9 |
| Chunks, zero overlap (control ii) | 42.8 | 34.8 | 49.2 | 44.8 |
| Condition | Chunks | Artif. | Gap |
|---|---|---|---|
| Headline (Table 1 ) | 43.9 | 28.0 | 15.9 |
| Zero-overlap chunks | 42.8 | 28.0 | 14.7 |
| Budget-matched artifacts ( ) | 43.9 | 29.2 | 14.7 |
| Dense-only retrieval | 41.8 | 27.5 | 14.3 |
| Session-level extraction | 43.9 | 29.2 | 14.7 |
| Decontaminated extraction prompt | 43.9 | 28.5 | 15.5 |
| Store | IE | MSR | KU | TR | Abs | All | T-Avg |
|---|---|---|---|---|---|---|---|
| n | 150 | 121 | 72 | 127 | 30 | 500 | — |
| Chunks | 84.7 | 62.8 | 77.8 | 50.4 | 46.7 | 67.4 | 64.5 |
| Union | 80.7 | 58.7 | 76.4 | 51.2 | 36.7 | 64.6 | 60.7 |
| RAG+RR | 85.3 | 61.2 | 72.2 | 38.6 | 70.0 | 64.8 | 65.5 |
| RAG | 86.7 | 58.7 | 72.2 | 37.0 | 70.0 | 64.2 | 64.9 |
| Artifacts | 50.7 | 34.7 | 69.4 | 36.2 | 43.3 | 45.4 | 46.9 |
| Method | Extract | Gen | Total | Total † | Ratio |
|---|---|---|---|---|---|
| Truncation | $0.00 | $1.25 | $1.25 | $0.09 | 13.2 |
| RAG | $0.00 | $3.50 | $3.50 | $0.23 | 15.3 |
| RAG+Rerank | $0.00 | $3.50 | $3.50 | $0.23 | 15.3 |
| Chunks (no graph) § | $0.00 | $3.84 | $3.84 | $0.23 | 16.7 |
| Summarization | $0.67 | $1.74 | $2.41 | $0.17 | 14.2 |
| Artifacts (CogCanvas) § | $0.13 | $2.20 | $2.33 | $0.27 | 8.7 |
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Recall | Exact | Cmplx. |
|---|---|---|---|
| Lossy stores | |||
| Truncation | 12.5% | 7.0% | Low |
| Summarization | 19.0% | 14.0% | Low |
| GraphRAG | 83.5% | 70.0% | High |
| Verbatim stores | |||
| RAG (k=10) | 93.5% | 89.5% | Low |
| Method | Pass | KW | Causal | Impact |
|---|---|---|---|---|
| Summarization | 0.0% | 42.5% | 48.3% | 35.8% |
| Chunks graph | 24.5% | 60.8% | 50.6% | 71.0% |
| Chunks, no graph | 25.0% | 60.8% | 50.6% | 71.0% |
| GraphRAG | 40.0% | 64.8% | 62.5% | 67.0% |
| RAG+Rerank | 45.0% | 68.2% | 64.2% | 73.3% |
| RAG | 55.5% | 74.8% | 68.8% | 79.5% |
| Method | TF | State | Infra | P&P | Incr. |
|---|---|---|---|---|---|
| Truncation | ✓ | ✗ | ✗ | ✓ | ✓ |
| Summarization | ✓ | ✗ | ✗ | ✓ | ✓ |
| RAG | ✓ | ✗ | ✗ | ✓ | ✓ |
| RAG+Rerank | ✓ | ✗ | ✗ | ✓ | ✓ |
| GraphRAG | ✓ | ✗ | ✗ | ✓ | ✗ |
| Letta ( Letta AI, 2025 ) | ✓ | ✓ | ✓ | ✗ | ✓ |
| Archival representation | Overall | M-hop | Temp. | O-dom. |
|---|---|---|---|---|
| Verbatim turns | 24.0 | 27.7 | 13.4 | 49.0 |
| Extracted artifacts | 13.2 | 14.5 | 7.2 | 29.2 |
| (verbatim artifacts) | +10.9 | +13.1 | +6.2 | +19.8 |
| Configuration | All | M-hop | Temp. | O-dom. |
|---|---|---|---|---|
| Full System | 28.6% | 20.9% | 30.5% | 44.8% |
| Graph Expansion | 25.8% | 19.5% | 29.0% | 33.3% |
| Reranking | 20.9% | 16.7% | 23.1% | 26.0% |
| Minimal (Graph only) | 14.3% | 11.7% | 13.7% | 24.0% |
| compress_rate | Overall | M-hop | Temp. | O-dom. |
|---|---|---|---|---|
| 1.00 (verbatim segments) | 47.4 | 37 | 56 | 47 |
| 0.75 (SeCom default) | 46.9 | 39 | 53 | 49 |
| 0.50 (aggressive) | 36.9 | 30 | 41 | 44 |
| ref: verbatim chunks | 45.2 | |||
| ref: full context | 59.9 |
| Store (all same backbone) | Tok/q | Acc. |
|---|---|---|
| Verbatim chunks (coarse) | 2,127 | 42.8 |
| Sentence-verbatim, (2.2 budget) | 4,674 | 39.1 |
| Sentence-verbatim, (budget match) | 2,140 | 36.2 |
| Sentence-verbatim, | 1,010 | 32.2 |
| Typed artifacts (lossy, fine) | 1,159 | 26.5 |
| Store (same pipeline) | GPT pair | Claude pair |
|---|---|---|
| Verbatim chunks | 43.9 | 53.9 (54.7) |
| Artifacts graph | 28.0 | 33.2 (34.0) |
| Chunks Artifacts | 15.9 | 20.7 |
| Prompt tok/q, chunks / artifacts | 2,127 / 1,191 | 2,400 / 1,518 |
| Store | Tok/q | All | M-hop | Temp. | O-dom. | |
|---|---|---|---|---|---|---|
| Chunks | 15 | 2,127 | 47.4 | 37.9 | 53.9 | 53.1 |
| EDU | 90 | 2,700 e | 36.5 | 23.4 | 46.7 | 40.6 |
| EDU | 60 | 1,900 e | 36.3 | 22.0 | 47.4 | 41.7 |
| EDU | 15 | 700 e | 29.8 | 17.0 | 40.5 | 31.2 |
| Artifacts | 15 | 1,159 | 30.2 | 26.2 | 30.8 | 39.6 |
| Store (same backbone, Chinese) | Acc. |
|---|---|
| Naive RAG (verbatim) | 86.1 |
| Verbatim chunks | 82.2 |
| Typed artifacts (lossy) | 35.6 |
| Direct experimental finding | Interpretation we draw |
|---|---|
| Chunks beat artifacts by 15.9pp (LoCoMo) and 22.0pp (LongMemEval-S) in one fixed pipeline; both gaps significant (Tables 1 , 3 ). | The stored representation, not the retrieval stack, is what loses. |
| The gap survives six confound controls, three retriever families and a second answerer/extractor vendor (§ 4 , Appendices P , H ). | The effect is not an artifact of one configuration. |
| Accuracy falls monotonically as verbatim tokens are removed, at random or by a real compressor (Figure 2 , Appendix F ). | Fidelity is causal for accuracy in this pipeline. |
| At equal budget, a sentence-verbatim store trails chunks by 6.6pp and beats artifacts by 5.7pp at matched granularity (Appendix G ). | Granularity explains part of the gap; fidelity explains more of it. |
| Union store: 42.5 vs. chunks 43.9, (no significant difference detected; not an equivalence test). | Adding structure to verbatim text costs no detected accuracy; replacing text does. |
| Of 58 diagnosable discordant questions, 29.3% have no trace in any artifact and 39.7% survive only in a raw-context field (Appendix D ). | Extraction commits to relevance before the question exists; only the first class is irreversible loss. |
| Result | Batch | Aggregation | Answerer |
|---|---|---|---|
| Tab. 5 (synthetic std.) | 2025-12/06 d | per-fact | gpt-4o-mini |
| Tab. 6 (multi-hop) | 2025-12/06 d | conv. macro | gpt-4o-mini |
| Tab. 1 (LoCoMo) | 2026-06 | question micro | gpt-4o a |
| cat. 4/5 paragraphs | 2026-06 | question micro | gpt-4o |
| Tab. 3 (LME-S) | 2026-06 | question micro | gpt-4o |
| Tab. 4 (cost) | 2026-05 b | usage logs | both tiers |
| Config. | Overall | M-hop | Temp. | O-dom. |
|---|---|---|---|---|
| Full System | 71.8% | 87.3% | 63.2% | 55.0% |
| – Graph Exp. | 25.8% | 32.3% | 20.5% | 24.7% |
| Improvement | +45.9pp | +55.0pp | +42.6pp | +30.3pp |
| Chunk store ∗ | 59.0% | 56.1% | 67.6% | 38.7% |
| rerank ∗ | 61.6% | 61.2% | 68.9% | 38.4% |
| Chunk size | Overlap | ||
|---|---|---|---|
| 256 | 50 | 43.5% | 50.5% |
| 512 | 100 | 49.0% | 52.5% |
| 1024 | 200 | 63.0% | 63.0% |
| Retriever family | Chunks | Artifacts | Gap |
|---|---|---|---|
| bge-m3 (dense, default) | 43.1 | 28.5 | 14.6 |
| text-embedding-3-small (dense) | 43.5 | 29.9 | 13.6 |
| BM25 (sparse lexical) | 43.8 | 29.0 | 14.7 |
| Cat-5 | Cat 1–3 | |
| Refusal mechanism | abstention | answerable |
| Score-threshold gate (no gate: 10.8 / 43.1) | ||
| 55.6 | 25.3 | |
| 59.2 | 21.9 | |
| 63.5 | 17.9 | |
| 65.5 | — | |
| Store | Cat 1–3 | Cat 5 |
|---|---|---|
| Chunks (union’s chunk subset) | 47.6 | 4.7 |
| Artifacts (union’s artifact subset) | 30.3 | 10.3 |
| Union (chunks artifacts) | 46.4 | 7.0 |
| Artifacts, independent extraction | 29.3 | 11.7 |
| Configuration | Overall | M-hop | Temp. | O-dom. |
|---|---|---|---|---|
| cs=800, g=3, cl=2, local | 13.5 | 16.2 | 2.2 | 52.4 |
| cs=800, g=1, cl=2, local † | 13.0 | 18.9 | 2.2 | 38.1 |
| cs=800, g=2, cl=2, local | 12.4 | 16.2 | 2.2 | 42.9 |
| cs=800, g=1, cl=3, local | 12.4 | 16.2 | 2.2 | 42.9 |
| cs=800, g=1, cl=2, drift | 11.9 | 13.5 | 4.4 | 38.1 |
| cs=400, g=1, cl=2, local | 11.9 | 16.2 | 1.1 | 42.9 |
| Configuration | Overall | M-hop | Temp. | O-dom. |
|---|---|---|---|---|
| cs=800, g=1, cl=2, drift (tuned) | 13.0 | 12.8 | 7.5 | 32.3 |
| cs=800, g=3, cl=2, local | 11.4 | 13.8 | 3.4 | 31.2 |
| cs=800, g=1, cl=2, local † | 11.2 | 13.5 | 3.1 | 31.2 |
| Verbatim chunks, no graph ∗ | 43.9 | 35.5 | 50.2 | 47.9 |
| Artifacts graph ∗ | 28.0 | 22.3 | 31.2 | 34.4 |
| Gap: tuned GraphRAG chunks | –30.9 | –22.7 | –42.7 | –15.6 |
| Config | Ref. | Causal | Pass |
|---|---|---|---|
| Low | 0.3 | 0.25 | 62.5% |
| Default | 0.5 | 0.45 | 67.5% |
| High | 0.7 | 0.6 | 60.0% |
| Very High | 0.8 | 0.7 | 57.5% |
Long-Term Conversational Memory
Long-term conversational memory helps AI assistants retain, update, and use information from past conversations across sessions.
See all 287 papers