Retrieval Is Not Enough: Why Organizational AI Needs Epistemic Infrastructure
Organizations: Kakashi Ventures Accelerator (KVA) · Massachusetts Institute of Technology
Abstract
Organizational knowledge used by AI agents typically lacks epistemic structure: retrieval systems surface semantically relevant content without distinguishing binding decisions from abandoned hypotheses, contested claims from settled ones, or known facts from unresolved questions. We argue that the ceiling on organizational AI is not retrieval fidelity but epistemic fidelity--the system's ability to represent commitment strength, contradiction status, and organizational ignorance as computable properties. We present OIDA, a framework that structures organizational knowledge as typed Knowledge Objects carrying epistemic class, importance scores with class-specific decay, and signed contradiction edges. The Knowledge Gravity Engine maintains scores deterministically with proved convergence guarantees (sufficient condition: max degree ; empirically robust to degree 43). OIDA introduces QUESTION-as-modeled-ignorance: a primitive with inverse decay that surfaces what an organization does not know with increasing urgency--a mechanism absent from all surveyed systems. We describe the Epistemic Quality Score (EQS), a five-component evaluation methodology with explicit circularity analysis. In a controlled comparison ( response pairs), OIDA's RAG condition (3,868 tokens) achieves EQS 0.530 vs.\ 0.848 for a full-context baseline (108,687 tokens); the token budget difference is the primary confound. The QUESTION mechanism is statistically validated (Fisher , OR). The formal properties are established; the decisive ablation at equal token budget (E4) is pre-registered and not yet run.
Figures & tables
| Type | Initial priority | Temporal behavior | Role |
|---|---|---|---|
| DECISION | 1.00 | Stable until superseded | Binding choice or commitment |
| CONSTRAINT | 0.90 | Stable until changed | Hard structural boundary |
| EVIDENCE | 0.80 | Slow decay | Verifiable supporting or refuting material |
| NARRATIVE | 0.70 | Stable contextual anchor | Persistent interpretive context |
| PLAN | 0.65 | Time-bounded decay | Structured intention with horizon |
| EVALUATION | 0.55 | Moderate decay | Informed qualitative assessment |
| Corpus | Domain | Files / size | Intended stress tests |
|---|---|---|---|
| A / ClearPath | Consulting / operations | 46 / 318 KB | Process redesign, bottleneck evidence, unresolved audit workflow, and conflicting organizational observations |
| B / FireGlass | IoT / product development | 47 / 566 KB | Architecture decisions, latency constraints, technical debt, negative knowledge, and integration risk |
| C / Vertex Minds | Venture capital | 77 / 376 KB | Investment decisions, due-diligence gaps, prior-deal lessons, and market-size contradiction |
| Corpus | Status here | Use in this paper | Not established |
|---|---|---|---|
| A / ClearPath | Reported pilot | Aggregate in-window comparison; knowledge-gap surfacing | No longitudinal dynamics |
| B / FireGlass | Released dataset | Product-development stress corpus for future tests | Not analyzed here |
| C / Vertex Minds | Reported diagnostic | Quality–cost frontier; contradiction-sensitive failure mode | No graph-retrieval win |
| Sub-score | Weight | Structured | Full context | |
|---|---|---|---|---|
| ECA | 0.20 | 0.667 | 0.807 | +0.140 |
| CP | 0.25 | 0.550 | 0.862 | +0.312 |
| CR | 0.20 | 0.418 | 0.860 | +0.442 |
| EC | 0.20 | 0.578 | 0.788 | +0.210 |
| DE | 0.15 | 0.582 | 0.843 | +0.262 |
| EQS | 1.00 | 0.5574 | 0.8329 | +0.2755 |
| Metric | Structured | Full context | Ratio / gap |
|---|---|---|---|
| Mean input tokens | 2,623 | 110,539 | 42.1 |
| Aggregate EQS | 0.5574 | 0.8329 | +0.2755 |
| Contextual recall | 0.418 | 0.860 | +0.442 |
| Condition | Gap section | Interpretation |
|---|---|---|
| Structured graph retrieval | 10 / 10 | Open questions are foregrounded by the representation |
| Full-context baseline | 5 / 10 | Gaps appear when salient in the raw context |
| Sub-score | Weight | Structured | Full context | |
|---|---|---|---|---|
| ECA | 0.20 | 0.617 | 0.817 | +0.200 |
| CP | 0.25 | 0.550 | 0.865 | +0.315 |
| CR | 0.20 | 0.455 | 0.863 | +0.408 |
| EC | 0.20 | 0.548 | 0.822 | +0.273 |
| DE | 0.15 | 0.520 | 0.848 | +0.328 |
| EQS | 1.00 | 0.5395 | 0.8438 | +0.3043 |
| Corpus | Struct. tok. | Full tok. | Struct. EQS | Full EQS |
|---|---|---|---|---|
| A / ClearPath | 2,623 | 110,539 | 0.5574 | 0.8329 |
| C / Vertex Minds | 1,944 | 109,303 | 0.5395 | 0.8438 |
| Query | Structured retrieval behavior | Full-context behavior | Diagnostic lesson |
| NovaTech market size and reliability | May present one TAM figure and miss the pitch-deck vs. dossier contradiction | Recovers both figures and treats them as unverified hypotheses | Graph retrieval helps only if both sides of a tension are extracted and retrieved |
| Quantity | How to estimate it | Why it matters |
|---|---|---|
| Window pressure | Tokens in the task-relevant corpus divided by usable context budget | If pressure is low, full context may be best |
| Epistemic heterogeneity | Fraction of units that are decisions, plans, hypotheses, evidence, or open questions | Flat chunks are less harmful when all records play the same role |
| Conflict / supersession rate | Fraction of claim-like units that contradict or replace earlier units | Signed edges matter only when conflicts or changes are common |
| Open-question rate | Number of unresolved information needs per document or per query | Open-question nodes matter only if uncertainty must be surfaced |
| Multi-document dependency | Number of source documents needed for a correct answer | Graph state helps most when answers require chains or tensions |
| Extraction quality | Precision and recall of node types and edge types on a small audit set | Bad graph structure can be worse than no graph structure |
| Condition | What it tests | Decision rule |
|---|---|---|
| Full context | Best in-window answer quality when the corpus fits | Use if cost and latency are acceptable |
| Dense / hybrid RAG | Standard low-cost retrieval without typed graph state | Use if queries are mostly factual and single-hop |
| Static typed graph | Value of typed nodes and signed edges without time dynamics | Use if state-sensitive queries improve at equal budget |
| Dynamic typed graph | Added value of time-dependent node weights | Use only if it beats the static typed graph |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Type | Coeff. | Semantics ( : A acts on B) |
|---|---|---|
| SUPPORTS | A provides evidence strengthening B | |
| BASED_ON | A is the logical grounding of B | |
| IMPLEMENTS | A operationally realizes B | |
| SUPERSEDES | A replaces B; B is demoted, not deleted | |
| REFINES | A narrows B without contradiction | |
| DERIVES_FROM | A follows logically from B |
| Symbol | Meaning |
|---|---|
| Attributed organizational memory graph at time | |
| Memory nodes and directed edges | |
| Node attributes and edge attributes | |
| Node-priority weight for node | |
| Type prior / initial priority | |
| Open-question urgency term |