From Association to Causation: Improving Retrieval Precision of Retrieval-Augmented Generation via Causal Relations and an Attention Mechanism

Jing Liu; Yongxing Qi; Muchen Jiang; Chengnan Hu; Qingqing Peng; Haoming Wang; Yuqing Wang; Yang Yu

cs.AIAug 22, 2026

From Association to Causation: Improving Retrieval Precision of Retrieval-Augmented Generation via Causal Relations and an Attention Mechanism

Authors: Jing Liu, Yongxing Qi, Muchen Jiang, Chengnan Hu, Qingqing Peng, Haoming Wang, Yuqing Wang, Yang Yu,

Organizations: Hangzhou Innovation Institute, Beihang University, Hangzhou, China

Abstract

Retrieval-Augmented Generation (RAG) grounds LLM generation on retrieved documents, but the standard terminal retrieval stage--dense-vector similarity, optionally followed by reranking--often returns documents that share keywords with the query without containing the needed information, a failure mode that grows with the knowledge base. We trace it to a conceptual gap: similarity captures only associational relations, whereas the documents that matter are linked to the query causally. We model the terminal retrieval stage with a causal graph grounded in Reichenbach's common cause principle: the keywords shared by the query and a retrieved document form a latent common cause A, and the document's residual keywords form a latent set B linking the document to the ideal output. Since a retrieved document is a collider (A -> d <- B), retrieval itself opens an associational path between the query and B, which licenses a training-free, attention-style re-scoring rule: the cosine similarity between the query embedding and the weighted centroid embedding of B. Unlike causality-enhanced RAG variants that model causal relations inside the knowledge content, our graph models the causal structure of the retrieval process itself. On a real 471-document enterprise knowledge base, the method promotes a relevant guideline from rank 6 to the top 3; on a controlled diagnostic corpus reproducing the keyword-stuffing regime, it improves the mean target rank from 2.88 to 1.25, while a trained cross-encoder reranker barely helps (2.63). Conversely, on three BEIR benchmarks the score underperforms the similarity baseline, delineating the applicability boundary: the method guards the keyword-stuffing regime of growing proprietary knowledge bases and complements neural rerankers; a corpus-level calibration gate selects the correct regime with >= 95% reliability. A fully local testbed demonstrates deployability.

Figures & tables

Panel of Figure 1 — Figure 1: Unified view of the proposed method at the terminal retrieval stage of RAG (redrawn in English from the flow–causal diagram in the underlying invention disclosure [ 1 ] ). Solid arrows: data flow. Red dashed arrows: causal directions among the query $q$ , the first-round candidate set $S_{1}$ , the ideal output $y$ , and the two latent keyword sets $A$ and $B$ (cf. the abstract causal graph in Figure 2 ). Step 1 scores each candidate by the attention-style dot product between the query embedding and the centroid of its residual keyword set $B$ ; step 2 constructs the prompt from the re-ranked set. The candidate set $S_{1}$ is a collider ( $A\!\rightarrow\!d\!\leftarrow\!B$ ), which justifies the score of step 1 (Observation 1 ).

Panel of Figure 2 — Figure 2: Causal graph of the terminal retrieval stage. $q$ : user query; $d$ : a document of the initially retrieved set $S_{1}$ ; $y$ : the ideal output text (unobserved); $A$ , $B$ : latent keyword sets. $d$ ( $A\!\rightarrow\!d\!\leftarrow\!B$ ) and $y$ ( $q\!\rightarrow\!y\!\leftarrow\!B$ ) are colliders. Marginally $q\perp B$ ; conditioning on the retrieved document $d$ opens the path $q\!\leftarrow\!A\!\rightarrow\!d\!\leftarrow\!B$ (Observation 1 ), licensing the attention-style score of Section 3.4 .

Baseline (similarity + rerank), top-8
Rank	Document
1	Measures for Security Assessment of Cross-Border Data Transfer (draft)
2	Measures for Security Assessment of Personal Information and Important Data
	Cross-Border Transfer (draft)
3	Measures for Security Assessment of Personal Information Cross-Border Transfer (draft)
4	Interpretation of Technical Guidelines for Automotive Collected-Data

Table 1: Retrieval results for the query “How to assess the security of cross-border data transfer?”. Bold: the document that actually contains the assessment procedure (《信息安全技术数据出境安全评估指南》). The baseline buries it at rank 6, behind keyword-matching automotive-data documents; our method ranks it within the top 3. Document titles translated from Chinese; the draft measures at ranks 1–3 (baseline) and 1–2 (ours) are also relevant regulations, while ranks 4, 5, and 7 of the baseline concern a different domain (automotive data).

Theme (target rank $\downarrow$ )	Baseline	CE reranker	$s_{c}$ exact	$s_{c}$ semantic	Hybrid
T1 data-transfer security (zh)	5	5	6	2	2
T2 TCM cough treatment (zh)	2	2	1	1	1
T3 Wason selection task (en)	3	3	4	2	2
T4 LLM pretraining data (zh)	3	3	2	1	1
T5 PI protection assessment (zh)	4	2	2	1	2
T6 ColBERT late interaction (en)	2	2	2	1	1

Table 2: Controlled diagnostic experiment (synthetic corpus, known ground truth; ranks, lower is better). Each theme contains one answer-bearing target document and two keyword-stuffed distractors. A strong trained cross-encoder reranker (BGE-reranker-v2-m3) barely improves over the similarity baseline and leaves the stuffing distractors at the top; the exact-match variant of our

A/B

construction is inconsistent (T1, T3); semantic absorption wins on all eight themes, and the hybrid configuration never ranks the target below the baseline.

Dataset	Method	nDCG@10	P@5	MRR@10
SciFact	Baseline $s_{\mathrm{sim}}$	0.642	0.159	0.608
	CE reranker	0.716	0.167	0.693
	Ours $s_{c}$	0.157	0.033	0.115
	Ours-fb (post-hoc)	0.157	0.032	0.115
	Hybrid	0.550	0.135	0.510
NFCorpus	Baseline $s_{\mathrm{sim}}$	0.317	0.305	0.523

Table 3: Public BEIR benchmark results (re-ranking the baseline top-20; best per dataset in bold). The trained cross-encoder reranker (BGE-reranker-v2-m3) is the strongest method on all three datasets, while our causal-attention score underperforms the similarity baseline; the fallback variant is indistinguishable from the plain score, and the hybrid recovers part but not all of the gap. Compare Table 2 , where the ordering reverses: the cross-encoder fails to repair the stuffing failure that our method fixes.

Corpus	Regime	Probes $n$	Correct-decision reliability	Gated outcome
SciFact	factoid	30	100% / 99.2%	baseline retained (0.642)
NFCorpus	factoid	30	100% / 100%	baseline retained (0.317)
ArguAna	factoid	30	99.9% / 95.5%	baseline retained (0.398)
Diagnostic (8 themes)	stuffing	4	100% / 100%	enabled; rank $2.88\!\rightarrow\!1.25$

Table 4: Corpus-level regime gating by offline calibration. Reliability is the fraction of bootstrap resamples (2,000 draws) in which the probe-based decision matches the correct one (disable on BEIR, enable on the diagnostic corpus); the two values per cell are for gating

s_{c}

and the hybrid variant, respectively; numbers in parentheses are nDCG@10. The gated system never degrades the baseline and retains the full gain where the method applies.

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 796 papers

CausalRAG2: Hierarchical Causal Knowledge Graph Design for RAG
Feb 4, 2026Nengbo Wang, Tuo Liang, Vikash Singh +6KG Construction Retrieval-Augmented Generation
Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, OH, USA · Design and Innovation Department, Case Western Reserve University, Cleveland, OH, USA
CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation
May 6, 2026Zhipeng Song, Yizhi Zhou, Xiangyu Kong +6Retrieval-Augmented Generation Learning to Rank
School of Computer Science and Technology, Dalian University of Technology, No.2 Linggong Road, Ganjingzi District, Dalian, 116024, China · School of Information Engineering, Dalian Ocean University, No. 2-52, Heishijiao Street, Shahekou District, Dalian, 116023, China · School of Information Engineering, Liaodong University, No.116 Linjiang Back Street, Zhenan District, Dandong, 118001, China +3
Why Retrieval-Augmented Generation Fails: A Graph Perspective
May 13, 2026Kai Guo, Xinnan Dai, Zhibo Zhang +5Retrieval-Augmented Generation LLM Grounding
Michigan State University · Massachusetts Institute of Technology

Feb 4, 2026J/K move · Enter open · S save

CausalRAG2: Hierarchical Causal Knowledge Graph Design for RAG

Nengbo Wang, Tuo Liang, Vikash Singh, Chaoda Song+5

Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, OH, USA · Design and Innovation Department, Case Western Reserve University, Cleveland, OH, USA

KG Construction Retrieval-Augmented Generation LLM Reasoning with Graphs Graph Retrieval-Augmented Generation Knowledge Graphs

Open paper AskPDF arXiv

Retrieval augmented generation (RAG) has enhanced large language models by enabling access to external knowledge, with graph-based RAG emerging as a powerful paradigm for structured retrieval and reasoning. However, existing graph-based methods often over-rely on entity-centric node matching and lack explicit causal modeling, leading to unfaithful or spurious answers. Prior attempts to incorporate causality are typically limited to local or single-document contexts and also suffer from information isolation that arises from modular graph structures, which hinders scalability and cross-module causal reasoning. To address these challenges, we propose CausalRAG2, a framework that rethinks knowledge organization for graph-based RAG through causal gating across hierarchical modules. CausalRAG2 explicitly models causal relationships to suppress spurious correlations while enabling scalable reasoning over large-scale knowledge graphs. We also introduce HolisQA, a benchmark for holistic comprehension beyond entity-centric matching. Extensive experiments demonstrate that CausalRAG2 consistently outperforms competitive graph-based RAG baselines across multiple datasets and evaluation metrics. Our work establishes a principled foundation for structured, scalable, and causally grounded RAG systems. Our code and HolisQA benchmark are available at https://github.com/Pwnb/CausalRAG2.

Abstract

Retrieval-Augmented Generation

Explore similar work

CausalRAG2: Hierarchical Causal Knowledge Graph Design for RAG

CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation

Why Retrieval-Augmented Generation Fails: A Graph Perspective