Calibrated Fusion for Heterogeneous Graph-Vector Retrieval in Multi-Hop QA
Abstract
Graph-augmented retrieval combines dense similarity with graph-based relevance signals such as Personalized PageRank (PPR), but these scores have different distributions and are not directly comparable. We study this as a score calibration problem for heterogeneous retrieval fusion in multi-hop question answering. Our method, PhaseGraph, maps vector and graph scores to a common unit-free scale using percentile-rank normalization (PIT) before fusion, enabling stable combination without discarding magnitude information. Across MuSiQue and 2WikiMultiHopQA, calibrated fusion improves held-out last-hop retrieval on HippoRAG2-style benchmarks: LastHop@10 increases from 69.1% to 71.0% on MuSiQue (15W/5L, p=0.041, n=514) and LastHop@5 from 51.7% to 53.6% on 2WikiMultiHopQA (11W/2L, p=0.023, n=491), both on independent held-out test splits. Against the official HippoRAG 2 pipeline on 2WikiMultiHopQA, calibrated fusion is ahead at LastHop@10 (+6.3pp, p<10^-3) and behind at LastHop@5 (-8.4pp), a cutoff-dependent cross-over we report in full. A theory-driven ablation shows that percentile-based calibration is directionally more robust than min-max normalization on both tune and test splits (1W/6L, p=0.125), while Boltzmann weighting performs comparably to linear fusion after calibration (0W/3L, p=0.25). These results suggest that score commensuration is a robust design choice, and the exact post-calibration operator appears to matter less on these benchmarks.
Figures & tables
| Bench. | Strategy | LastHop | W | L | |
|---|---|---|---|---|---|
| MuSiQue (@10) | Vector only | 69.1 | – | – | – |
| True RRF | 71.8 | 25 | 11 | .029 | |
| PhaseGraph (.7, dk30) | 71.0 | 15 | 5 | .041 | |
| 2Wiki (@5) | Vector only | 51.7 | – | – | – |
| True RRF | 53.0 | 8 | 2 | .109 | |
| PhaseGraph (.4, dk20) | 53.6 | 11 | 2 | .023 |
| Config | Axis | Tune | Test (held-out) | |||
|---|---|---|---|---|---|---|
| LastHop | W/L | LastHop | W/L | |||
| Baseline (PIT + Boltzmann) | – | 52.5 | – | 53.8 | – | – |
| Min-max norm | Norm. | 51.7 | 1/5 | 52.7 | 1/6 | .125 |
| Raw/max norm | Norm. | 52.8 | 5/3 | – | – | – |
| Linear fusion | Formula | 52.5 | 2/2 | 53.2 | 0/3 | .250 |
| Fixed | Temp. | 52.1 | 2/4 | – | – | – |
| Strategy | LastHop | R@5 | W | L | net |
| Vector only | 58.1 | 88.0 | – | – | – |
| Thermo .3, uncap. | 51.0 | 78.8 | 36 | 81 | 45 |
| True RRF, uncap. | 58.6 | 83.4 | 32 | 39 | 7 |
| Thermo .7, dk30 | 60.4 | 88.6 | 16 | 5 | +11 |
| + Entity disambig. (synonym linking): | |||||
| Vector only | 60.1 | 89.4 | – | – | – |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Strategy | Recall | 95% CI | OR | |
|---|---|---|---|---|
| Vector only | 39.4 | [28.5, 51.5] | — | — |
| True RRF | 74.2 | [62.6, 83.3] | 4.43 | |
| Thermo | 78.8 | [67.5, 87.2] | 5.71 | |
| Thermo+Ising | 80.3 | [69.2, 88.2] | 6.27 | |
| Thermo+CE | 9.1 | [4.2, 18.4] | 0.15 |
| Category | #Cfg | Best W | Max LastHop |
|---|---|---|---|
| Tier 1 (fusion) | 102 | 26 | 78.8 |
| Tier 2 (reranking) | 277 | 27 | 80.3 |
| Tier 3 (research) | 288 | 27 | 80.3 |
| Total | 667 | 27 | 80.3 |
| Config | F1 [95% CI] | EM |
|---|---|---|
| Vector only | .135 [.075, .203] | 6.1% |
| Context stuff. | .183 [.099, .275] | 15.2% |
| Thermo | .271 [.181, .369] | 18.2% |
| Thermo+Ising | .307 [.213, .406] | 21.2% |
| Strategy | LastHop | W | L | |
| Vector only | 37.8 | – | – | – |
| True RRF † | 41.2 | 15 | 6 | .078 |
| Thermo (tune winner) | 43.5 | 26 | 11 | .020 |
| Thermo (best recall) | 44.3 | 24 | 7 | .003 |
| Thermo (safest) | 43.1 | 15 | 1 | .001 |
| Strategy | LastHop | FullSup | W | L |
| Vector only | 39.4 | 6.1 | – | – |
| Legacy RRF | 39.4 | 6.1 | 0 | 0 |
| True RRF ∗∗∗ | 74.2 | 48.5 | 23 | 0 |
| Thermo ∗∗∗ | 78.8 | 57.6 | 26 | 0 |
| Log-linear ∗∗∗ | 78.8 | 57.6 | 26 | 0 |
| Power mean ∗∗∗ | 78.8 | 57.6 | 26 | 0 |
| Method | all | tune | test | 2-hop | 3-hop | 4-hop |
|---|---|---|---|---|---|---|
| HippoRAG 2 (pub.) | 74.7 | – | – | – | – | – |
| PropRAG (pub.) | 77.3 | – | – | – | – | – |
| Vector only | 69.8 | 69.5 | 70.1 | 77.9 | 68.9 | 46.2 |
| PhaseGraph (thermo) | 69.8 | 69.6 | 70.0 | 77.9 | 69.0 | 46.2 |
Hybrid Retrieval
Hybrid retrieval combines lexical, dense, and structured search signals to find relevant evidence more reliably.
See all 87 papers