Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configurations generally retain higher absolute F1 under ASR input, both extensions amplify the error: the F1 gap from clean text to the highest-WER accent is 36-67% larger under their combination than under naive dense retrieval, on all three benchmarks. The dominant failure mode is corruption of one or more query entities, accounting for 87-96% of degradation cases on 2WikiMultiHopQA across all four methods. Two lightweight surface-form mitigations leave most of the gap intact, indicating that downstream retrieval structure amplifies remaining entity errors. We release code and data at https://github.com/Continuum-AI-Corp/spoken-multihop-rag .
Figures & tables
Figure 1: HotpotQA F1 under oracle (clean text), US-accented (WER 9.4%), and Nigerian-accented (WER 14.5%) speech. The oracle–NG gap grows from 0.104 for the simplest baseline (Naive RAG) to 0.142 for the most structurally complex configuration (IRCoT+HippoRAG2).
Figure 2: Overview of our spoken multi-hop QA evaluation pipeline. Each text question is synthesized in US, Indian, Filipino, and Nigerian accents through neural TTS, then transcribed by Whisper-large-v3. The mitigation stage denotes three separately evaluated experimental conditions (no mitigation, N-best decoding, or phonetic entity correction) rather than a runtime branch. Each transcription is then processed by one of the four RAG methods. The oracle condition (brown dashed arc) bypasses the speech pipeline and feeds the original text directly to retrieval, providing a clean-text upper bound.
Dataset
Method
Oracle
US
IN
PH
NG
Gap
WER
—
9.4%
11.0%
10.6%
14.5%
—
HotpotQA
Naive RAG
0.617
0.574
0.558
0.561
0.513
0.104
IRCoT+Naive
0.638
0.594
0.573
0.583
0.523
0.115
HippoRAG2
0.711
0.670
0.645
0.652
0.595
0.116
IRCoT+HippoRAG2
0.730
0.688
0.652
0.671
0.588
0.142
WER
—
13.6%
14.1%
13.8%
17.1%
—
Table 1: F1 across three multi-hop QA datasets under oracle (clean text), US-accented, Indian-accented (IN), Filipino-accented (PH), and Nigerian-accented (NG) speech. WER per accent is reported per dataset. Gap (shaded) is the oracle–Nigerian F1 difference. Bold marks the largest gap per dataset; underline marks IRCoT+Naive to isolate the contribution of iterative reformulation. All oracle–Nigerian gaps are significant at p<0.001 .
Figure 3: Per-question degradation rate on 2WikiMultiHopQA under Nigerian-accented speech, stratified by Whisper WER. The gap between Naive RAG and IRCoT+HippoRAG2 reaches 12.6% at ≥20% WER.
Figure 4: F1 of Naive RAG and HippoRAG2 across IRCoT steps on 1,000 HotpotQA questions. Solid lines show oracle F1, dashed lines show Nigerian-accented (NG) F1, and shaded regions show the oracle–NG gap.
Error type
US
IN
PH
NG
Entity corruption
90%
87%
91%
94%
Severe garbling
30%
35%
37%
40%
Number/date corrupt.
4%
4%
4%
4%
Function-word noise
1%
0%
0%
0%
Other content change
7%
7%
5%
3%
# degradation cases
113
124
116
174
Table 2: Distribution of error types within degradation cases on 2WikiMultiHopQA under Naive RAG. Cases can carry multiple labels.
Figure 5: Case study on 2WikiMultiHopQA illustrating how a single entity corruption ( Hanuman Patal Vijay ) affects all four RAG methods. Cell values are F1 scores.
N-best
Phonetic
Method
F1
Rec.
F1
Rec.
Naive RAG
0.328
−2.2%
0.337
+4.4%
HippoRAG2
0.425
+2.5%
0.439
+11.1%
IRCoT+Naive
0.337
−1.1%
0.347
+4.5%
IRCoT+HippoRAG2
0.448
−1.0%
0.464
+7.2%
Table 3: Mitigation results on 2WikiMultiHopQA NG (vs. Table 1 baselines). Recovery is the fraction of the Oracle–NG gap closed by each mitigation.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Words
Articles (3)
a, an, the
Prepositions (13)
of, in, on, at, to, for, with, by,
from, as, into, about, between
Forms of be (7)
is, was, are, were, be, been, being
Other auxiliaries (6)
have, has, had, do, does, did
Modals (8)
will, would, shall, should,
Appendix
Table 4: Function-word list used for ASR error categorization. Matching is case-insensitive after stripping non-alphanumeric characters.
Dataset
Method
Oracle
US
IN
PH
NG
Gap
WER
—
9.4%
11.0%
10.6%
14.5%
—
HotpotQA
Naive RAG
0.488
0.456
0.439
0.446
0.403
0.085
IRCoT+Naive
0.508
0.476
0.453
0.469
0.417
0.091
HippoRAG2
0.568
0.534
0.514
0.519
0.473
0.095
IRCoT+HippoRAG2
0.588
0.546
0.519
0.534
0.463
0.125
WER
—
13.6%
14.1%
13.8%
17.1%
—
Appendix
Table 5: Exact Match (EM) across three multi-hop QA datasets under oracle and four English accents. All settings follow the convention in Table 1 .
Error type
US
IN
PH
NG
Entity corruption
67%
70%
75%
82%
Severe garbling
14%
24%
24%
44%
Number/date corrupt.
5%
5%
5%
4%
Function-word noise
4%
9%
8%
7%
Other content change
21%
13%
11%
7%
# degradation cases
81
103
88
136
Appendix
Table 6: Distribution of error types within degradation cases on HotpotQA under Naive RAG. Cases can carry multiple labels.
Error type
US
IN
PH
NG
Entity corruption
59%
78%
66%
68%
Severe garbling
4%
7%
14%
21%
Number/date corrupt.
0%
0%
0%
1%
Function-word noise
9%
4%
2%
1%
Other content change
28%
13%
20%
23%
# degradation cases
46
54
50
71
Appendix
Table 7: Distribution of error types within degradation cases on MuSiQue under Naive RAG. Cases can carry multiple labels.
Dataset
Method
US
IN
PH
NG
HotpotQA
Naive RAG
67% (81)
70% (103)
75% (88)
82% (136)
IRCoT+Naive
77% (81)
77% (107)
73% (100)
79% (154)
HippoRAG2
73% (71)
77% (96)
81% (90)
82% (154)
IRCoT+HippoRAG2
67% (75)
78% (110)
69% (99)
80% (177)
2WikiMultiHopQA
Naive RAG
90% (113)
87% (124)
91% (116)
94% (174)
IRCoT+Naive
95% (140)
92% (132)
92% (132)
94% (206)
Appendix
Table 8: Entity-corruption rate within degradation cases for each RAG method and accent, with the number of degradation cases in parentheses. Cases can carry multiple labels.
Whisper
SeamlessM4T
Method
F1
Gap
F1
Gap
Naive RAG
0.331
0.137
0.333
0.135
IRCoT+Naive
0.339
0.177
0.334
0.182
HippoRAG2
0.421
0.162
0.421
0.162
IRCoT+HippoRAG2
0.450
0.195
0.431
0.214
Appendix
Table 9: End-to-end F1 and oracle–ASR gap on 2WikiMultiHopQA NG under Whisper-large-v3 (WER 17.1%) and SeamlessM4T-v2-large (WER 28.0%). Per-method oracle F1 is reported in Table 1 . The gap ordering across methods is preserved under both ASR systems.