Decoupling Error Attribution in Cloud-Native Graph-RAG: A Data Integrity Diagnostic Framework

Shuai Yan; Yuhang Wu; Xiaodong Huang; Ke Wang

cs.IRSep 10, 2026

Decoupling Error Attribution in Cloud-Native Graph-RAG: A Data Integrity Diagnostic Framework

Authors: Shuai Yan, Yuhang Wu, Xiaodong Huang, Ke Wang

Organizations: College of Computer and Software Chengdu Jincheng College Chengdu 611731, China

Abstract

Graph-RAG systems often assume pristine data quality, overlooking the severe impact of perturbations in cloud-native databases. This paper proposes a three-layer decoupled diagnostic framework to orthogonally attribute system errors to reasoning loss, Knowledge Graph (KG) defects, and Cypher generation errors. Evaluated on a spatio-temporal ecological KG of the Southeastern Tibet region with eight defect types, results reveal that data integrity, rather than algorithmic reasoning, is the dominant performance bottleneck, with structural defects degrading system accuracy from 0.93 to 0.39. Crucially, we observe a masking-like phenomenon termed the Parametric Knowledge Masking Effect (PKME), suggesting LLMs compensate for broken retrieval paths using internal memory. This shrinks apparent query generation errors by over 70 percent, obscuring actual storage deterioration and increasing the risk of false negatives for automated monitoring. This work provides a quantitative foundation for auditing and optimizing data integrity in cloud-based information fusion systems.

Figures & tables

Defect Type	Category	Primary Impact & Cloud Mapping
Structural Sparsity	Structural	Query path interruption (e.g., shard loss)
Structural Noise	Structural	Retrieval results point to incorrect entities
Temporal Drift	Structural	Temporal logic confusion (e.g., sync delays)
Evidence Gap	Structural	Evidence exists but content is missing
Motif Artifact	Numeric	Uniform value distribution, misleading statistical reasoning
Factual Noise	Numeric	Attribute values exceed ecologically plausible ranges

TABLE I: Eight Controlled Defect Types

Fig. 1: System Architecture and Three-Layer Decoupled Evaluation Flowchart

Layer	Experimental Design	Measurement Goal
Oracle Evidence	gold_cypher $\rightarrow$ Clean Graph $\rightarrow$ Perfect Text Evidence $\rightarrow$ LLM	Upper bound of LLM reasoning capability under ideal data provenance
Oracle Cypher	gold_cypher $\rightarrow$ Defective Graph $\rightarrow$ Graph Retrieval Results $\rightarrow$ LLM	Impact of KG quality defects on storage integrity and retrieval paths
End-to-End	Generated Cypher $\rightarrow$ Defective Graph $\rightarrow$ Graph Retrieval Results $\rightarrow$ LLM	Performance of the complete system under real data defect scenarios

TABLE II: Definition of the Three-Layer Decoupled Evaluation Framework

Fig. 2: Three-Layer Baseline Accuracy Comparison

Fig. 3: Error Attribution Stacked Analysis

Panel of Figure 6 — Fig. 4: System Robustness Evaluation (E2E vs. Oracle Cypher)

Graph Retrieval-Augmented Generation

Graph Retrieval-Augmented Generation connects language models to knowledge graphs to retrieve relational evidence for grounded, multi-hop answers.

See all 232 papers

MissDiag: Diagnostic Evaluation of Incomplete-Knowledge Robustness in KGQA and KG-RAG
Aug 19, 2026Hang Wang, Hang Dong, Lu Liu +1Knowledge Graph Question Answering RAG Robustness
Why Retrieval-Augmented Generation Fails: A Graph Perspective
May 13, 2026Kai Guo, Xinnan Dai, Zhibo Zhang +5Retrieval-Augmented Generation LLM Grounding
Michigan State University · Massachusetts Institute of Technology
TRIAGE: Trustworthy Retrieval Instrumentation And Graph Evaluation
Jul 3, 2026Axel TahmasebiMoradi, Lucas Schott, Martin RoyerKG Construction RAG Evaluation
IRT-SystemX

Aug 19, 2026J/K move · Enter open · S save

MissDiag: Diagnostic Evaluation of Incomplete-Knowledge Robustness in KGQA and KG-RAG

Hang Wang, Hang Dong, Lu Liu, Chuanru Ren

Knowledge Graph Question Answering RAG Robustness Graph Retrieval-Augmented Generation

Open paper AskPDF arXiv

Knowledge graph question answering (KGQA) and knowledge-graph-based retrieval-augmented generation (KG-RAG) aim to ground answers in explicit graph evidence, but real-world knowledge graphs are often sparse, outdated, and incomplete. Existing robustness evaluations usually report aggregate changes in answer quality after evidence is removed or perturbed, which measures sensitivity to incomplete support but leaves the source of degradation under-specified: the same score change can conflate the type of missing evidence, the response of the evaluated system, and the sensitivity of the answer-matching protocol. To address this gap, we propose MissDiag, a diagnostic evaluation framework for incomplete-knowledge robustness in KGQA and KG-RAG. MissDiag keeps the question and gold answer fixed while applying structurally typed missingness interventions to benchmark-provided support graphs, enabling paired comparisons that decompose robustness changes by evidence type, system response, and evaluation protocol rather than reducing them to a single aggregate score drop. Experiments across multiple system families show that incomplete-knowledge robustness is better understood as a typed degradation phenomenon than as a uniform property: answer-adjacent evidence loss produces the largest observed degradation, source-context removal is often neutral and can be beneficial, and semantic answer matching changes absolute scores while preserving the main typed degradation patterns. By transforming aggregate robustness measurement into typed diagnostic attribution, MissDiag provides a more interpretable basis for comparing, diagnosing, and stress-testing KGQA and KG-RAG systems under incomplete knowledge.

Abstract

Graph Retrieval-Augmented Generation

Explore similar work

MissDiag: Diagnostic Evaluation of Incomplete-Knowledge Robustness in KGQA and KG-RAG

Why Retrieval-Augmented Generation Fails: A Graph Perspective

TRIAGE: Trustworthy Retrieval Instrumentation And Graph Evaluation