cs.CLAug 26, 2026
SaveReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chains
Organizations: College of Software, Jilin University Changchun, Jilin, China · College of Computer Science and Technology, Jilin University Changchun, Jilin, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University Changchun, Jilin, China · College of Computer Science and Technology, Dalian University of Technology Dalian, Liaoning, China ·
Abstract
Retrieval-Augmented Generation (RAG) has emerged as a powerful architecture for Question Answering (QA) by integrating external information into Large Language Models (LLMs). However, false, inaccurate, and misleading information in news and social media poses a serious challenge to real-world RAG systems, especially in multi-hop QA, where complex multi-step reasoning can be misled by even a single deceptive misinformation segment in the retrieved documents. Existing approaches mainly rely on implicit alignment or explicit regulation, but their limited ability to assess fine-grained information reliability makes them vulnerable to deceptive misinformation that is semantically relevant to the question yet factually incorrect, leading to erroneous answers. To address this limitation, we propose ReliableRAG, which, to the best of our knowledge, is the first reliability-driven framework that mitigates deceptive misinformation in multi-hop QA through fine-grained evaluation of individual triples. ReliableRAG first extracts information segments from source documents and represents them as structured triples. It then quantifies triple reliability by combining query-triple semantic relevance with triple credibility, retaining only the top- reliable and non-redundant triples. Based on these refined triples, ReliableRAG autoregressively constructs robust reasoning chains to consolidate trustworthy evidence and filter deceptive misinformation, producing accurate answers faithful to reliable information. Experiments on three multi-hop QA datasets show that ReliableRAG outperforms existing methods, substantially improving the factual reliability and robustness of RAG systems under deceptive misinformation injection.
Figures & tables
Figure 1. Illustration of the differences between existing methods and ReliableRAG under the interference of misinformation. Green text denotes the correct answer, while red text represents a misleading incorrect answer. While implicit alignment and explicit regulation paradigms often falter when faced with semantically similar fine-grained deceptive misinformation, thereby leading to incorrect answers, ReliableRAG identifies the fine-grained reliability of information and constructs robust reasoning chains to prevent deceptive misinformation, ensuring accurate answers.
Figure 2. The overall pipeline of the ReliableRAG framework. ReliableRAG first employs the Triple Extraction module to extract information segments from documents and represent them as structured triples. Subsequently, the Triple Evaluation module performs fine-grained reliability quantification by synthesizing query-triple semantic relevance and triple credibility, ensuring the retention of only the top- most reliable and non-redundant triples for robust multi-hop reasoning. Leveraging these refined triples, the Chain Construction module autoregressively constructs robust reasoning chains to consolidate trustworthy information and filter out deceptive misinformation, ultimately ensuring the generation of accurate answers.
| Input: | question , triples with injected misinformation |
| , maximum number of reasoning hops , number of reasoning | |
| chains to be retained , beam size , number of selected triples | |
| output: Reasoning Chains |
Algorithm 1 Reliability-Guided Chain Construction
| Methods | Ideal Setting | Evaluator-Generated Setting | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HotPotQA | 2WikiMultiHopQA | MuSiQue | HotPotQA | 2WikiMultiHopQA | MuSiQue | ||||||||
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | ||
| Fine-tuning Methods | w/ Low-credibility Docs | w/ Low-credibility Docs | |||||||||||
| CAG-7B (EMNLP’24) | 5.80 | 21.53 | 3.80 | 18.20 | 1.20 | 9.45 | 4.70 | 17.91 | 3.70 | 16.58 | 0.7 | 7.71 | |
| Self-RAG-7B (ICLR’24) | 18.30 | 28.00 | 5.80 | 10.50 | 6.60 | 15.72 | - | - | - | - | - | - | |
| Knowledge-R1-7B (arxiv’25) | 26.80 | 35.45 | 18.00 | 22.81 | 11.20 | 20.76 | - | - | - | - | - | - | |
Table 1. Main results between ReliableRAG and compared methods under both the ideal setting and the evaluator-generated setting. The best performance within each setting is highlighted in bold. ‘w/ Low-credibility Docs’ denotes the injection of one low-credibility document per question. The dash ‘-’ denotes that the method is either not applicable to the setting or identical in practice to another setting already reported.
Figure 3. Performance comparison between ReliableRAG and six compared methods on the test sets of three multi-hop QA datasets with varying numbers of low-credibility documents per question, evaluated under the ideal setting.
| Methods | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| w/o Triple Extraction | 19.30 | 28.12 | 12.50 | 19.93 | 3.30 | 12.82 |
| w/o Triple Evaluation | 37.80 | 49.42 | 30.10 | 37.16 | 17.20 | 25.81 |
| w/o Chain Construction | ||||||
| Top-10 Triples | 46.40 | 59.84 | 26.10 | 37.53 | 21.30 | 28.61 |
| Top-15 Triples | 46.40 | 60.01 | 29.70 | 42.39 | 23.90 | 31.96 |
Table 2. Ablation results of ReliableRAG-TBS on the test sets of three multi-hop QA datasets. The Top- Triple variants represent the performance where the supporting context is constructed by applying the TBS synthesis strategy to the top- most reliable triples without the Chain Construction module.
Figure 4. EM and F1 performance of ReliableRAG across various configurations of the balancing coefficient that regulates the dual-factor perception mechanism within the Triple Evaluation module. Results compare (a) ReliableRAG-TBS and (b) ReliableRAG-SBS variants, evaluated under the ideal setting using the dev sets of three multi-hop QA datasets.
Figure 5. IE density distribution under the optimal configuration .
Figure 6. Impact of the maximum reasoning chain length on the QA performance and average chain length of ReliableRAG-TBS on the dev sets of two multi-hop QA datasets, under an ideal setting where one low-credibility document is injected per question.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7. Illustration of the Context Synthesis Strategies. Triple-based Synthesis (TBS) directly converts triples into a concise natural language format by concatenating their components into textual strings to maximize information density. In contrast, Source-based Synthesis (SBS) utilizes the projection function to map triples back to original source documents, identifying the most representative evidence documents through a frequency-based voting mechanism, thereby restoring the rich semantic details.
Figure 8. Performance of ReliableRAG versus compared methods under the evaluator-generated setting, across three multi-hop QA datasets with varying proportions of low-credibility documents per question.
Figure 9. Impact of on the multi-hop QA performance and average chain length of ReliableRAG-TBS on the dev set of MuSiQue under the ideal setting where one low-credibility document is injected per question.
| Models | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| DRAGON+ | 70.0 | 75.3 | 66.0 | 70.0 | 44.0 | 49.9 |
| E5 | 73.0 | 79.4 | 71.0 | 73.5 | 40.0 | 46.2 |
| E5-Mistral | 70.0 | 77.4 | 73.0 | 76.5 | 48.0 | 50.8 |
Table 3. Performance comparison of ReliableRAG-TBS employing different bi-encoder models for triple retrieval on the dev sets of three multi-hop QA datasets under the ideal setting where one low-credibility document is injected per question.
| Models | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| DRAGON+ | 75.0 | 79.7 | 61.0 | 68.2 | 43.0 | 47.8 |
| E5 | 76.0 | 81.7 | 70.0 | 74.4 | 39.0 | 46.2 |
| E5-Mistral | 77.0 | 81.4 | 74.0 | 77.8 | 44.0 | 49.7 |
Table 4. Performance comparison of ReliableRAG-SBS employing different bi-encoder models for triple retrieval on the dev sets of three multi-hop QA datasets under the ideal setting where one low-credibility document is injected per question.
| Method | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| w/o Graph Construction | 19.30 | 28.12 | 12.50 | 19.93 | 3.30 | 12.82 |
| w/o Triple Evaluation | 24.10 | 33.49 | 15.20 | 21.76 | 8.20 | 18.46 |
| w/o Chain Construction | ||||||
| Top-10 Triples | 50.20 | 63.81 | 29.70 | 44.06 | 23.90 | 31.69 |
| Top-15 Triples | 51.00 | 64.36 | 32.60 | 47.77 | 25.40 | 33.47 |
Table 5. Ablation results of ReliableRAG-SBS on the test sets of three multi-hop QA datasets. The Top- Triple variants represent the performance where the supporting context is constructed by applying the SBS synthesis strategy to the top- most reliable triples without the Chain Construction module.
| Methods | LLM | HotPotQA | 2WikiMultiHopQA | MuSiQue |
|---|---|---|---|---|
| Vanilla RAG | Gemma-7B | 560 | 542 | 568 |
| Llama3-8B | 580 | 564 | 586 | |
| Mistral-7B | 632 | 610 | 638 | |
| TruthfulRAG (AAAI’26) | Gemma-7B | 151 | 151 | 156 |
| Llama3-8B | 164 | 165 | 168 | |
| Mistral-7B | 173 | 172 | 177 |
Table 6. Average input context length (in tokens) of ReliableRAG versus compared methods across various LLMs on the test sets of three multi-hop QA datasets.
| Models | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| Llama-2-7B | 45.4 | 59.1 | 40.6 | 50.3 | 25.4 | 34.2 |
| Qwen2.5-7B | 54.5 | 68.7 | 47.9 | 57.6 | 28.4 | 37.3 |
Table 7. Impact of Different Generators on ReliableRAG-TBS performance on the test sets of three multi-hop QA datasets, where one low-credibility document is injected per question.
| Models | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| Llama-2-7B | 44.3 | 56.8 | 39.4 | 49.0 | 23.8 | 32.0 |
| Qwen2.5-7B | 57.3 | 71.3 | 49.8 | 58.5 | 29.9 | 39.2 |
Table 8. Impact of Different Generators on ReliableRAG-SBS performance on the test sets of three multi-hop QA datasets, where one low-credibility document is injected per question.
| HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) |
| 47.9 | 62.1 | 24.3 | 43.4 | 16.3 | 24.4 |
Table 9. Experimental results of applying the dual-factor perception mechanism to coarse-grained documents under the ideal setting.
| Model | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| Llama3-8B | 29.5 | 41.6 | 23.7 | 31.6 | 13.4 | 22.0 |
| Qwen2.5-7B | 31.9 | 42.8 | 22.7 | 31.2 | 15.0 | 24.1 |
| Qwen2.5-14B | 31.6 | 42.6 | 23.2 | 31.4 | 14.1 | 23.0 |
Table 10. Impact of Different Extractors on ReliableRAG-TBS performance on the test sets of multi-hop three QA datasets, where one low-credibility document is injected per question.
| Model | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| Llama3-8B | 16.9 | 26.3 | 7.5 | 13.1 | 4.2 | 14.3 |
| Qwen2.5-7B | 17.3 | 26.3 | 7.3 | 13.0 | 4.4 | 14.7 |
| Qwen2.5-14B | 17.6 | 27.0 | 7.7 | 13.3 | 4.0 | 14.3 |
Table 11. Impact of Different Extractors on ReliableRAG-SBS performance on the test sets of three multi-hop QA datasets, where one low-credibility document is injected per question.
| Model | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| Llama3-8B | 53.4 | 67.0 | 45.4 | 56.2 | 29.1 | 36.7 |
| Mistral-7B | 28.8 | 38.9 | 21.4 | 28.4 | 11.1 | 16.6 |
| Gemma-7B | 21.2 | 28.5 | 17.4 | 22.2 | 4.0 | 8.6 |
Table 12. Impact of different selectors on ReliableRAG-TBS performance under the ideal setting across the test sets of three multi-hop QA datasets, with one low-credibility document injected per question.
| Model | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| Llama3-8B | 55.1 | 68.8 | 43.7 | 59.7 | 30.9 | 38.6 |
| Mistral-7B | 40.7 | 52.0 | 30.5 | 44.5 | 20.0 | 27.2 |
| Gemma-7B | 21.2 | 28.5 | 17.4 | 22.2 | 4.0 | 8.6 |
Table 13. Impact of different selectors on ReliableRAG-SBS performance under the ideal setting across the test sets of three multi-hop QA datasets, with one low-credibility document injected per question.
| Methods | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| GLM-4-Flash | 34.4 | 45.7 | 23.7 | 34.2 | 15.7 | 22.1 |
| Mistral-Small-24B | 38.4 | 51.0 | 27.4 | 39.3 | 17.8 | 25.4 |
| Qwen3.5-35B-A3B | 41.3 | 54.5 | 30.1 | 42.8 | 20.4 | 27.6 |
Table 14. Impact of Different Evaluators on ReliableRAG-TBS performance on the test sets of three multi-hop QA datasets, where one low-credibility document is injected per question.
| Methods | HotPotQA | 2WikiMultiHopQA | MuSiQue | |||
|---|---|---|---|---|---|---|
| EM (%) | F1 (%) | EM (%) | F1 (%) | EM (%) | F1 (%) | |
| GLM-4-Flash | 26.6 | 36.5 | 16.0 | 25.2 | 13.8 | 22.8 |
| Mistral-Small-24B | 28.9 | 39.6 | 21.0 | 31.8 | 16.2 | 25.1 |
| Qwen3.5-35B-A3B | 33.1 | 45.1 | 22.9 | 34.5 | 17.4 | 26.7 |
Table 15. Impact of Different Evaluators on ReliableRAG-SBS performance on the test sets of three multi-hop QA datasets, where one low-credibility document is injected per question.
Figure 10. Distribution of credibility scores generated by the evaluator on low-credibility and Wikipedia-sourced documents on the test sets of three multi-hop QA datasets.
Figure 11. Distribution of credibility scores generated by the evaluator on triples extracted from low-credibility and Wikipedia-sourced documents on the test sets of three multi-hop QA datasets.
Figure 12. ROC curves of credibility scores generated by the evaluator for both documents and triples across the test sets of three multi-hop QA datasets.
| Stage | Content Detail |
|---|---|
| Documents | Wikipedia-sourced Document 1: He was also known for his handshake deal with Walt Disney that permitted the University of Oregon to use the likeness of Donald Duck as the basis for its mascot, the Oregon Duck. … Wikipedia-sourced Document 2: Donald Duck is a cartoon character created in 1934 at Walt Disney Productions. … Low-credibility Document: Contrary to popular belief, it was not Donald Duck but rather Mickey Mouse who was featured in this landmark agreement. … |
| Triple Extraction | Triples from Wikipedia-sourced Document 1: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot) … Triples from Wikipedia-sourced Document 2: (Donald Duck, creation year, 1934) … Triples from Low-credibility Document: (Mickey Mouse’s Legacy in Animation Education, popular misconception, that Donald Duck was featured in the deal, not Mickey Mouse) … |
| Question | Leo A. Harris was also known for his handshake deal with Walt Disney that permitted the University of Oregon to use the likeness of what cartoon character created in 1934 at Walt Disney Productions? |
| Triple Evaluation | The First Hop (Ranked Triples): 1: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot) 2: (Oswald the Lucky Rabbit, replacement, Mickey Mouse, created by Walt Disney in 1928) 3: (Oswald the Lucky Rabbit, rights, taken by Charles Mintz from Walt Disney in 1928) … The Second Hop (Ranked Triples): 1: (Donald Duck, creation year, 1934) 2: (Donald Duck, creator, Walt Disney Productions) 3: (Donald Duck, comic book publication, most published comic book character in the world outside of the superhero genre) … … |
| Chain Construction | The First Hop (Ranked Candidate Chains): 1: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot) 2: (Oswald the Lucky Rabbit, creators, Ub Iwerks and Walt Disney) 3: (Oswald the Lucky Rabbit, replacement, Mickey Mouse was created by Walt Disney in 1928) … The Second Hop (Ranked Candidate Chains): 1: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot), (Donald Duck, creation year, 1934) 2: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot), (Donald Duck, physical characteristics, anthropomorphic white duck with a yellow-orange bill and legs and feet) 3: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot), (Donald Duck, comic book publication, most published comic book character in the world outside of the superhero genre) … … |
| Final Chains | 1: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot), (Donald Duck, creation year, 1934) 2: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot), (Donald Duck, creation year, 1934), (Donald Duck, creator, Walt Disney Productions), (Donald Duck, comic book publication, most published comic book character in the world outside of the superhero genre) 3: (Leo Harris, notable deal, handshake deal with Walt Disney to use Donald Duck as the basis for the Oregon Duck mascot), (Donald Duck, creation year, 1934), (Daisy Duck, creation year, 1940), (Donald Duck, creator, Walt Disney Productions) … |
Table 16. A case study of the ReliableRAG pipeline on the test set of HotPotQA under the ideal setting.
Figure 13. Content of the assessment framework. For instance, in the triple (Albert Einstein, was the first recipient in 1921 of, the Nobel Prize in Physics) , the analysis recognizes that while the entities are authentic, the specific relational claim is inaccurate. Integrating this analysis, the evaluator generates the final credibility score.
More in this topic
Retrieval-Augmented Generation
Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.
See all 796 papers