cs.AIAug 22, 2026

From Association to Causation: Improving Retrieval Precision of Retrieval-Augmented Generation via Causal Relations and an Attention Mechanism

Authors: Jing Liu, Yongxing Qi, Muchen Jiang, Chengnan Hu, Qingqing Peng, Haoming Wang, Yuqing Wang, Yang Yu, 

Organizations: Hangzhou Innovation Institute, Beihang University, Hangzhou, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Retrieval-Augmented Generation (RAG) grounds LLM generation on retrieved documents, but the standard terminal retrieval stage--dense-vector similarity, optionally followed by reranking--often returns documents that share keywords with the query without containing the needed information, a failure mode that grows with the knowledge base. We trace it to a conceptual gap: similarity captures only associational relations, whereas the documents that matter are linked to the query causally. We model the terminal retrieval stage with a causal graph grounded in Reichenbach's common cause principle: the keywords shared by the query and a retrieved document form a latent common cause A, and the document's residual keywords form a latent set B linking the document to the ideal output. Since a retrieved document is a collider (A -> d <- B), retrieval itself opens an associational path between the query and B, which licenses a training-free, attention-style re-scoring rule: the cosine similarity between the query embedding and the weighted centroid embedding of B. Unlike causality-enhanced RAG variants that model causal relations inside the knowledge content, our graph models the causal structure of the retrieval process itself. On a real 471-document enterprise knowledge base, the method promotes a relevant guideline from rank 6 to the top 3; on a controlled diagnostic corpus reproducing the keyword-stuffing regime, it improves the mean target rank from 2.88 to 1.25, while a trained cross-encoder reranker barely helps (2.63). Conversely, on three BEIR benchmarks the score underperforms the similarity baseline, delineating the applicability boundary: the method guards the keyword-stuffing regime of growing proprietary knowledge bases and complements neural rerankers; a corpus-level calibration gate selects the correct regime with >= 95% reliability. A fully local testbed demonstrates deployability.

Figures & tables

Figure 1: Unified view of the proposed method at the terminal retrieval stage of RAG (redrawn in English from the flow–causal diagram in the underlying invention disclosure [ 1 ] ). Solid arrows: data flow. Red dashed arrows: causal directions among the query qq , the first-round candidate set S1S_{1} , the ideal output yy , and the two latent keyword sets AA and BB (cf. the abstract causal graph in Figure 2 ). Step 1 scores each candidate by the attention-style dot product between the query embedding and the centroid of its residual keyword set BB ; step 2 constructs the prompt from the re-ranked set. The candidate set S1S_{1} is a collider ( A ⁣→ ⁣d ⁣← ⁣BA\!\rightarrow\!d\!\leftarrow\!B ), which justifies the score of step 1 (Observation 1 ).
Figure 2: Causal graph of the terminal retrieval stage. qq : user query; dd : a document of the initially retrieved set S1S_{1} ; yy : the ideal output text (unobserved); AA , BB : latent keyword sets. dd ( A ⁣→ ⁣d ⁣← ⁣BA\!\rightarrow\!d\!\leftarrow\!B ) and yy ( q ⁣→ ⁣y ⁣← ⁣Bq\!\rightarrow\!y\!\leftarrow\!B ) are colliders. Marginally q⊥Bq\perp B ; conditioning on the retrieved document dd opens the path q ⁣← ⁣A ⁣→ ⁣d ⁣← ⁣Bq\!\leftarrow\!A\!\rightarrow\!d\!\leftarrow\!B (Observation 1 ), licensing the attention-style score of Section 3.4 .
RankDocument
Baseline (similarity + rerank), top-8
1Measures for Security Assessment of Cross-Border Data Transfer (draft)
2Measures for Security Assessment of Personal Information and Important Data
 Cross-Border Transfer (draft)
3Measures for Security Assessment of Personal Information Cross-Border Transfer (draft)
4Interpretation of Technical Guidelines for Automotive Collected-Data
Table 1: Retrieval results for the query “How to assess the security of cross-border data transfer?”. Bold: the document that actually contains the assessment procedure (《信息安全技术 数据出境安全评估指南》). The baseline buries it at rank 6, behind keyword-matching automotive-data documents; our method ranks it within the top 3. Document titles translated from Chinese; the draft measures at ranks 1–3 (baseline) and 1–2 (ours) are also relevant regulations, while ranks 4, 5, and 7 of the baseline concern a different domain (automotive data).
Theme (target rank ↓\downarrow )BaselineCE rerankerscs_{c} exactscs_{c} semanticHybrid
T1 data-transfer security (zh)55622
T2 TCM cough treatment (zh)22111
T3 Wason selection task (en)33422
T4 LLM pretraining data (zh)33211
T5 PI protection assessment (zh)42212
T6 ColBERT late interaction (en)22211
Table 2: Controlled diagnostic experiment (synthetic corpus, known ground truth; ranks, lower is better). Each theme contains one answer-bearing target document and two keyword-stuffed distractors. A strong trained cross-encoder reranker (BGE-reranker-v2-m3) barely improves over the similarity baseline and leaves the stuffing distractors at the top; the exact-match variant of our A/BA/B construction is inconsistent (T1, T3); semantic absorption wins on all eight themes, and the hybrid configuration never ranks the target below the baseline.
DatasetMethodnDCG@10P@5MRR@10
SciFactBaseline ssims_{\mathrm{sim}}0.6420.1590.608
CE reranker0.7160.1670.693
Ours scs_{c}0.1570.0330.115
Ours-fb (post-hoc)0.1570.0320.115
Hybrid0.5500.1350.510
NFCorpusBaseline ssims_{\mathrm{sim}}0.3170.3050.523
Table 3: Public BEIR benchmark results (re-ranking the baseline top-20; best per dataset in bold). The trained cross-encoder reranker (BGE-reranker-v2-m3) is the strongest method on all three datasets, while our causal-attention score underperforms the similarity baseline; the fallback variant is indistinguishable from the plain score, and the hybrid recovers part but not all of the gap. Compare Table 2 , where the ordering reverses: the cross-encoder fails to repair the stuffing failure that our method fixes.
CorpusRegimeProbes nnCorrect-decision reliabilityGated outcome
SciFactfactoid30100% / 99.2%baseline retained (0.642)
NFCorpusfactoid30100% / 100%baseline retained (0.317)
ArguAnafactoid3099.9% / 95.5%baseline retained (0.398)
Diagnostic (8 themes)stuffing4100% / 100%enabled; rank 2.88 ⁣→ ⁣1.252.88\!\rightarrow\!1.25
Table 4: Corpus-level regime gating by offline calibration. Reliability is the fraction of bootstrap resamples (2,000 draws) in which the probe-based decision matches the correct one (disable on BEIR, enable on the diagnostic corpus); the two values per cell are for gating scs_{c} and the hybrid variant, respectively; numbers in parentheses are nDCG@10. The gated system never degrades the baseline and retains the full gain where the method applies.
More in this topic

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 796 papers

Explore similar work

CardsList
Feb 4, 2026

CausalRAG2: Hierarchical Causal Knowledge Graph Design for RAG

Retrieval augmented generation (RAG) has enhanced large language models by enabling access to external knowledge, with graph-based RAG emerging as a powerful paradigm for structured retrieval and reasoning. However, existing graph-based methods often over-rely on entity-centric node matching and lack explicit causal modeling, leading to unfaithful or spurious answers. Prior attempts to incorporate causality are typically limited to local or single-document contexts and also suffer from information isolation that arises from modular graph structures, which hinders scalability and cross-module causal reasoning. To address these challenges, we propose CausalRAG2, a framework that rethinks knowledge organization for graph-based RAG through causal gating across hierarchical modules. CausalRAG2 explicitly models causal relationships to suppress spurious correlations while enabling scalable reasoning over large-scale knowledge graphs. We also introduce HolisQA, a benchmark for holistic comprehension beyond entity-centric matching. Extensive experiments demonstrate that CausalRAG2 consistently outperforms competitive graph-based RAG baselines across multiple datasets and evaluation metrics. Our work establishes a principled foundation for structured, scalable, and causally grounded RAG systems. Our code and HolisQA benchmark are available at https://github.com/Pwnb/CausalRAG2.

Nengbo Wang, Tuo Liang, Vikash Singh +6

Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, OH, USA · Design and Innovation Department, Case Western Reserve University, Cleveland, OH, USA

KG ConstructionRetrieval-Augmented GenerationLLM Reasoning with GraphsGraph Retrieval-Augmented GenerationKnowledge Graphs
AskPDF
May 6, 2026

CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) depends on document ranking to provide useful evidence for generation, but conventional reranking methods mainly optimize query-document relevance rather than generation usefulness. A relevant document may still introduce noise, while a lower-ranked document may better reduce the generator's uncertainty. We propose CAR (Confidence-Aware Reranking), a query-guided, training-free, and plug-and-play reranking framework that uses generator confidence change as a document usefulness signal. CAR estimates confidence through the semantic consistency of multiple sampled answers under query-only and query-document conditions. Documents that significantly increase confidence are promoted, those that decrease confidence are demoted, and uncertain cases preserve the baseline order, while a query-level gate avoids unnecessary intervention on already confident queries. Experiments on four BEIR datasets show that CAR consistently improves NDCG@5 across sparse and dense retrievers, LLM-based and supervised rerankers, and four LLM backbones. Notably, CAR improves the YesNo reranker by 25.4 percent on average under Contriever retrieval, and its ranking gains strongly correlate with downstream generation F1 improvements, achieving Spearman rho = 0.964.

Zhipeng Song, Yizhi Zhou, Xiangyu Kong +6

School of Computer Science and Technology, Dalian University of Technology, No.2 Linggong Road, Ganjingzi District, Dalian, 116024, China · School of Information Engineering, Dalian Ocean University, No. 2-52, Heishijiao Street, Shahekou District, Dalian, 116023, China · School of Information Engineering, Liaodong University, No.116 Linjiang Back Street, Zhenan District, Dandong, 118001, China +3

Retrieval-Augmented GenerationLearning to RankLLM Reranking
AskPDF
May 13, 2026

Why Retrieval-Augmented Generation Fails: A Graph Perspective

Retrieval-Augmented Generation (RAG) has become a powerful and widely used approach for improving large language models by grounding generation in retrieved evidence. However, RAG systems still produce incorrect answers in many cases. Why RAG fails despite having access to external information remains poorly understood. We present a model-internal study of retrieval-augmented generation that examines how retrieved evidence influences answer generation. Using circuit tracing, we construct attribution graphs that model the flow of information through transformer layers during decoding. These graphs represent interactions among retrieved context, intermediate model activations, and generated tokens, providing a graph, circuit-level view of how external evidence is integrated into the model's reasoning process across multiple question answering benchmarks, we observe consistent structural differences: correct predictions exhibit deeper reasoning paths, more distributed evidence flow, and a more structured pattern of local connectivity, while failed predictions show shallower, fragmented, and overly concentrated evidence flow. Building on these findings, we develop a graph-based error detection framework that uses attribution-graph topology features. Furthermore, we show that attribution graphs enable targeted interventions. By reinforcing question-constrained evidence grounding, we reshape internal routing so that answer generation remains guided by the question, leading to more effective integration of retrieved information and fewer errors.

Kai Guo, Xinnan Dai, Zhibo Zhang +5

Michigan State University · Massachusetts Institute of Technology

Retrieval-Augmented GenerationLLM GroundingMechanistic Interpretability
AskPDF