MIRAGE: A Framework for Multi-hop Reasoning under Dynamic Knowledge Injection
Abstract
Multi-hop Question Answering (MQA) requires large language models (LLMs) to retrieve and synthesize multiple pieces of evidence into a coherent reasoning path, which is essential for complex tasks. However, prior work on knowledge updating for MQA has largely focused on simple fact edits, overlooking a more challenging and realistic setting---Knowledge Injection---in which the model must simultaneously accommodate both conflicting updates and entirely novel information. Under this setting, a hop-level answer is often wrong simply because the model trusts the wrong knowledge source: it clings to outdated parametric memory when the injected knowledge conflicts with it (K2), and it cannot answer when the required fact is entirely new (K3). To investigate this, we introduce NEXUS, the first benchmark tailored for multi-hop reasoning under Knowledge Injection, which explicitly covers both conflicting (K2) and incremental (K3) knowledge. We further propose MIRAGE, an efficient, training-free framework whose core component, the Correlation-aware Dual-path Reasoner (CDR), resolves such per-hop source-trust errors via P(True)-based confidence routing, complemented by a lightweight template-guided corrector for residual structural deviations. Extensive experiments across four LLM families and two benchmarks demonstrate that MIRAGE consistently improves both final-answer accuracy and hop-level reasoning accuracy, establishing a new state of the art in this challenging setting.
Figures & tables
| Method | MQUAKE-CF-3K | NEXUS | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 edited | 100 edited | All edited | 1 edited | 100 edited | All edited | |||||||
| Acc | Hop-Acc | Acc | Hop-Acc | Acc | Hop-Acc | Acc | Hop-Acc | Acc | Hop-Acc | Acc | Hop-Acc | |
| LLaMA-2-7B | ||||||||||||
| MEMIT | 14.97 | 6.43 | 9.4 | 2.47 | 2.3 | 0.37 | 13.82 | 4.92 | 8.26 | 3.04 | 1.8 | 1.2 |
| MELLO | 33.57 | 9.9 | 20.0 | 10.07 | 17.33 | 9.91 | 28.29 | 9.87 | 18.42 | 8.7 | 14.92 | 7.82 |
| PokeMQA | 44.13 | 30.6 | 37.33 | 27.83 | 32.83 | 23.87 | 42.27 | 28.86 | 34.26 | 27.12 | 32.12 | 21.12 |
| Metric | Incremental data | Conflict data |
|---|---|---|
| Mean | 0.24 | 0.77 |
| Variance | 0.0266 | 0.0976 |
| Modules | MQUAKE-CF-3K | NEXUS | ||||
|---|---|---|---|---|---|---|
| Method | CDR | TGC | 1 ed. | All ed. | 1 ed. | All ed. |
| PokeMQA | – | – | 61.28 | 43.19 | 59.12 | 47.69 |
| +TGC | – | ✓ | 67.94 | 45.21 | 63.57 | 49.01 |
| +CDR | ✓ | – | 64.01 | 44.08 | 63.23 | 48.46 |
| Full | ✓ | ✓ | 68.41 | 45.22 | 64.22 | 49.21 |
| Dataset | Method | ||
|---|---|---|---|
| MQUAKE (no edits) | pure LLM | 76.10 | 72.42 |
| MQUAKE-CF-3K | Ours | 73.28 | 47.83 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Country | Capital | Country | Capital |
|---|---|---|---|
| Faldros | Thalnor | Arthos | Olmara |
| Aleria | Vendrell | Sylvoris | Trenna |
| Drathos | Harnath | Tarlen | Maznor |
| Siltara | Zorvona | Hytrion | Vortum |
| Malira | Bryntor | Belithar | Lothar |
| Velora | Theralis | Elarion | Firos |
| Method | 1 edited | 100 edited | ||
|---|---|---|---|---|
| Acc | Hop-Acc | Acc | Hop-Acc | |
| PokeMQA | 36.29 | 32.69 | 21.60 | 20.23 |
| MIRAGE (Ours) | 54.77 | 53.64 | 23.95 | 21.91 |
| Dataset | Total Count | Acc (%) |
|---|---|---|
| HotpotQA | 7405 | 85.63 |
| MQUAKE-CF-3K | 3000 | 76.97 |
| Input Question | Which continent does Celine Dion’s country of citizenship belong to? |
|---|---|
| Edit-hop | [Edit Question] Which continent is Canada located in? |
| [Edited Knowledge] Canada is located in Europe (Ground Truth: North America ) . | |
| [Expected Final Answer] Europe |
| Original Question | In the Chinese financial system, which type of financial institution is identified as having the highest risk spillover intensity, and which type primarily plays a risk-absorbing role? |
|---|---|
| Phase 1: Edit-hop (Counterfactual Modification) | |
| Fact 1 (Spillover) | Highest risk spillover: [Ground Truth: Securities Industry] [Edited: Insurance Industry] |
| Fact 2 (Absorption) | Strongest risk absorption: [Ground Truth: Banking Industry] [Edited: Securities Industry] |
| Phase 2: Multi-hop Reasoning Evaluation (LLM Trajectory) | |
| Sub-question 1 | In the Chinese financial system, which type of financial institution exhibits the most obvious risk diffusion and the highest risk spillover intensity? |
| LLM Output 1 | …the entity of Subquestion is: Insurance Industry ✓ Hop 1: Success |
Multi-Hop QA
Multi-hop question answering studies how systems combine evidence across documents, facts, or modalities to answer questions requiring linked reasoning.
See all 242 papers