Large language models (LLMs) are now a standard source of query expansion terms but they hallucinate. They inject non existent entities and off-topic terms that add ranking noise and can hurt strong retrievers. Existing remedies such as corpus grounding, mutual verification and preference alignment treat hallucinated terms as an undifferentiated nuisance to be filtered by agreement heuristics, without a mechanistic account of why a given term appears or a principled criterion for how many terms to keep. We propose Subsequence-Association Filtered, breadth-aware query expansion (SAFE-QE), a training-free, retriever agnostic method that draws on the subsequence-association account of hallucination. We score each candidate expansion term by its corpus conditioned cross-context reproducibility, terms that reappear under randomised query contexts yet lack support in the pseudo-relevant set are flagged as products of dominant hallucinatory associations and down-weighted. Drawing on the characterisation of language generation as a trade-off between consistency (no hallucination) and breadth (coverage), we cast term selection as constrained coverage maximisation over the pseudo-relevant set, keeping a set that is approximately broad but consistent and stable across resampling. We also introduce the Hallucinated Expansion Term Rate, a query side faithfulness metric that counts injected expansion terms no relevant document supports. We evaluate our approach on benchmarked TREC and BEIR collections. Our SAFE-QE is statistically indistinguishable from generative and corpus-steered expansion on nDCG@10 while injecting fewer unsupported terms 4.3 per query against 27.8 for Query2doc and 57.5 for HyDE and on a strong learned sparse retriever, where every expansion method regresses. Faithfulness is therefore available at no measurable cost in ranking quality.
Figures & tables
Figure 1: A schematic pipeline of a dominant hallucinated subsequence association makes the LLM emit context-free, ungrounded expansion terms that inject false positives into the ranking, and these terms hurt stronger retrievers.
nDCG@10
HETR
Baseline
SAFE-QE
base
Δ
p
SAFE-QE
base
Δ
p
No expansion
0.404
0.410
-0.006
0.152
–
–
–
–
Query2doc
0.403
0.411
-0.008 ∗
0.021
0.440
0.582
-0.142 ∗∗∗
< 0.001
HyDE
0.403
0.399
+0.004
1.000
0.440
0.593
-0.153 ∗∗∗
< 0.001
CSQE
0.404
0.407
-0.004
0.683
0.369
0.416
-0.047 ∗∗∗
< 0.001
MILL
0.404
0.400
+0.004
0.562
0.369
0.435
-0.066 ∗∗∗
< 0.001
Table 1: Paired per-query tests of SAFE-QE with each baseline, pooled over the four BEIR collections with the gemma-2-9b generator. Δ is SAFE-QE baseline. p is a paired t -test with Bonferroni correction across baselines, ∗p<0.05 , ∗∗p<0.01 , ∗∗∗p<0.001 . On effectiveness SAFE-QE is statistically indistinguishable from every generative baseline, on faithfulness it is better than rest of the methods. The HETR of 0.440 here is the BEIR-pooled per-query value; the all-collection macro-average appears in Table 5 .
Collection
Retriever
None
Query2doc
HyDE
CSQE
MILL
SAFE-QE
DL’19
BM25
0.5058
0.6324
0.5255
0.5611
0.5174
0.5876
SPLADE
0.7308
0.5848
0.6160
n/a
n/a
0.6950
Contriever
0.5812
†
†
0.5844
†
0.6381
DL’20
BM25
0.4796
0.6044
0.5597
0.5057
0.4622
0.5336
SPLADE
0.7197
0.6409
0.5936
n/a
n/a
0.6127
Contriever
0.5251
0.6731
†
0.5469
†
0.5775
Table 2: Evaluation across six collections and three retrievers, where generator is gemma-2-9b. The best nDCG@10 scores are in bold. n/a represents the baseline’s injected no terms at all and is slightly identical to no expansion, CSQE and MILL under SPLADE. These are excluded from every aggregate. † marks Contriever cells not included in this comparison; they are omitted from all aggregates. Δ is the mean gain over no expansion a method covers.
Faithfulness
Query-time cost
Method
HETR
∣E(q)∣
load
cut
s/query
tok/query
gen. calls
Query2doc
0.656
43.0
27.83
85%
4.06
78
1.00
HyDE
0.663
87.2
57.47
93%
1.10
231
0.19
CSQE
0.531
14.8
7.90
45%
5.23
758
1.00
MILL
0.546
13.8
7.52
43%
2.91
757
0.50
SAFE-QE
0.465
10.3
4.79
–
10.97
1516
2.00
Table 3: Faithfulness and query time cost on the collection, retriever, where generator is gemma-2-9b, macro-averaged over all six collections and three retrievers. HETR here is the micro-rate over injected terms, so it differs from the BEIR-pooled per-query value in Table 1 . ∣E(q)∣ is the number of terms a method injects into the query. load is the absolute number of those terms that no judged relevant document supports, HETR ×∣E(q)∣ , and cut is SAFE-QE’s reduction in that load. Lower is better in the first three columns. SAFE-QE buys its faithfulness with generation cost, two sampled generations plus P perturbed prompts.
Figure 2: Results for decomposing what a retriever actually receives, per retriever, over the collections. On the left, how many terms a method injects. In middle, what fraction of them no judged relevant document supports (HETR). On right, their product, the hallucinated-term load. The advantage of SAFE-QE is largest on the dense retriever, where free generation is least grounded.
Collection
Retriever
Query2doc
HyDE
CSQE
MILL
SAFE-QE
DL’19
BM25
0.194
0.206
0.042
0.065
0.041
SPLADE
0.194
0.206
n/a
n/a
0.310
Contriever
†
†
0.274
†
0.125
DL’20
BM25
0.262
0.288
0.077
0.106
0.083
SPLADE
0.262
0.288
n/a
n/a
0.370
Contriever
0.747
†
0.347
†
0.197
Table 4: HETR across all six collections and three retrievers, and generator is gemma-2-9b. HETR is retriever-independent for Query2doc and HyDE, whose injected set does not depend on the first stage, and retriever-dependent for the grounded methods, whose selection is computed against the retriever’s own pseudo-relevant set. n/a and † are as in Table 2 .
Figure 3: The faithfulness effectiveness plane, per retriever, macro averaged over the collections. Up and to the left is better. SAFE-QE sits on the frontier under BM25 and Contriever. Under SPLADE its rate is higher than the generative baselines’ even though its load is a third of theirs (Fig. 2 ), because grounding is computed against SPLADE’s own pseudo-relevant set and that set admits more candidates.
nDCG@10
HETR
Method
Qwen
Gemma
Qwen
Gemma
Qwen
Gemma
Qwen
Gemma
0.5B
2B
7B
9B
0.5B
2B
7B
9B
HyDE
0.3951
0.4641
0.4697
0.4788
0.640
0.597
0.590
0.570
MILL
0.4414
0.4535
0.4584
0.4620
0.480
0.447
0.420
0.443
SAFE-QE
0.4648
0.4784
0.4750
0.4845
0.398
0.386
0.344
0.375
Table 5: nDCG@10 and HETR as the expansion LLM grows from 0.5B to 9B parameters, macro-averaged over all collections and retrievers. SAFE-QE’s faithfulness advantage is present at every scale and without dependence on a large generator; the 9B HETR ( 0.375 ) therefore differs from the BEIR-pooled 0.440 in Table 1 .
Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks, yet they remain susceptible to hallucinations -- generating content that is factually incorrect, unfaithful to provided context, or misaligned with user instructions. We present HalluScan, a comprehensive benchmark framework that systematically evaluates hallucination detection and mitigation across 72 configurations spanning 6 detection methods, 4 open-weight model families, and 3 diverse domains. We introduce three key contributions: (1) HalluScore, a novel composite metric that achieves a Pearson correlation of r = 0.41 with human expert judgments; (2) Adaptive Detection Routing (ADR), an intelligent routing algorithm achieving 2.0x cost reduction with only 0.1% AUROC degradation; and (3) systematic error cascade decomposition revealing substantial variation in hallucination error types across domains. Our experiments reveal that NLI Verification achieves the highest overall AUROC of 0.88, while RAV achieves the second-highest AUROC of 0.66.
Recent advances in equipping Large Language Models (LLMs) with search tools and outcome-reward reinforcement learning (RL) have achieved new state-of-the-art results on open-domain QA tasks. However, we argue that current training paradigms harbor a critical vulnerability: they predominantly reward correct answers but fail to penalize fabricated ones when retrieval fails, thereby implicitly exacerbating hallucinations. To address this, we propose Abstention-Aware Reinforcement Learning (AWA-RL), which dynamically shapes the abstention reward utilizing the model's query-specific prior capabilities and continuous on-policy training observations. We also introduce a novel metric, RA-F1, to measure the capability-reliability trade-off. Compared to non-abstaining baselines, AWA-RL boosts absolute precision by up to 10.3% and overall RA-F1 by 2.9%, with only marginal sacrifice in raw accuracy. These results confirm that AWA-RL successfully yields highly capable and reliable search agents. The code, data, and model weights are publicly available at https://github.com/zfj1998/AWA-RL.
LLM-based query expansion improves retrieval by enriching the original query with additional context. Yet most methods remain generation-driven, producing plausible pseudo-documents or expansions without checking how the target corpus responds. This can introduce retrieval drift, amplify misleading vocabulary, or miss terms that distinguish relevant from non-relevant documents. We argue that effective expansion requires retrieval-grounded feedback, not just single-pass generation or unverified iteration. We introduce ADORE (ADapt, Observe, Relevance Evaluate), an iterative framework that turns retrieval outcomes into feedback for the next expansion. At each round, an LLM generates pseudo-passages, a retriever exposes the corpus response, and a relevance assessor evaluates retrieved documents against the original query. These judgments identify what to reinforce, what remains undercovered, and what to suppress. Across TREC Deep Learning, BEIR, and BRIGHT, ADORE consistently outperforms strong query expansion baselines with notable improvements across nearly all evaluation settings, improving average nDCG@10 by 24.5% over BM25 and 3.6% over the strongest prior query expansion method on BEIR, and by 122.9% over BM25 and 9.2% over the best query expansion baseline on BRIGHT. Our code and data are publicly available.
Amin Bigdeli, Negar Arabzadeh, Radin Hamidi Rad +3
University of Waterloo · University of California, Berkeley · Mila – Quebec AI Institute +1