Generative information retrieval (GenIR) formulates the retrieval process as a text-to-text generation task, leveraging the vast knowledge of large language models. However, existing works primarily optimize for relevance while often overlooking document trustworthiness. This is critical in high-stakes domains like healthcare and finance, where relying solely on semantic relevance risks retrieving unreliable information. To address this, we propose an Authority-aware Generative Retriever (AuthGR), the first framework that incorporates authority into GenIR. AuthGR consists of three key components: (i) Multimodal Authority Scoring, which employs a vision-language model to quantify authority from textual and visual cues; (ii) a Three-stage Training Pipeline to progressively instill authority awareness into the retriever; and (iii) a Hybrid Ensemble Pipeline for robust deployment. Offline evaluations demonstrate that AuthGR successfully enhances both authority and accuracy, with our 3B model matching a 14B baseline. Crucially, large-scale online A/B tests and human evaluations conducted on the commercial web search platform confirm significant improvements in real-world user engagement and reliability.
Figures & tables
Figure 1: Illustration of our motivation. (a) Models relying solely on relevance fail to distinguish an unreliable blog from authoritative ones. (b) By integrating authority, our model can prioritize trustworthy documents.
Figure 2: Overall architecture of AuthGR. (a) Multimodal Authority Scoring quantifies document trustworthiness based on textual and visual signals. (b) Three-stage training pipeline progressively instills authority awareness into the retriever. (c) The Hybrid Ensemble Pipeline integrates AuthGR with existing rankers for deployment.
Figure 3: Illustration of the GRPO stage.
Model
Size
P@3
R@5
R@10
In-context Learning (ICL)
Gemma 3
27B
0.1255
0.1732
0.2285
EXAONE 3.5
32B
0.0825
0.1253
0.1612
K-EXAONE
236B
0.1366
0.1918
0.2656
Qwen3
32B
0.0821
0.1176
0.1570
LLaMA 3.1
405B
0.1413
0.1974
0.2590
Table 1: Offline evaluation results. Bold denotes the best performance of our method. ‘AuthGR (Full)’ denotes the final model incorporating CPT, SFT, and GRPO.
Production
Hybrid Ensemble
Label score
3.06
3.41
Table 2: Human evaluation results in a blind side-by-side test, comparing (i) the production system and (ii) its integration with AuthGR via Hybrid Ensemble.
Metrics
Control
Treatment
Pages with clicks
+0.08%
+21.36%
Total document clicks
+0.22%
+22.07%
Top 1 document CTR
+0.87%
+22.83%
Top 3 document CTR
+0.81%
+22.68%
Top 5 document CTR
+0.81%
+22.76%
Table 3: Online A/B test results showing relative gain in user engagement. ‘Control’ group uses the production system. ‘Treatment’ group receives results enhanced by AuthGR through the Hybrid Ensemble.
Table 5: Authority score distribution in generated documents. Mean and Median are computed on a 0-100 scale. ‘Low’ and ‘High’ indicate the number of documents scoring 0-60 and 90-100, respectively.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
Value
Point-biserial correlation ( r )
0.495 ( p<0.001 )
ROC-AUC
0.915
Matthews Correlation (MCC)
0.526
Appendix
Table 6: Quantitative alignment between VLM-generated authority scores and human expertise labels.
Figure 5: Examples where the Text+Image input yields more appropriate results than the image-only input.
Figure 6: Prompts used for Multimodal Authority Scoring.
Model
Size
Latency (ms)
Throughput
HyperCLOVAX (SFT)
14B
2,881
1.30
AuthGR (Ours)
3B
1,225
3.26
Appendix
Table 7: Comparison of inference efficiency. Latency denotes the average response time, and throughput indicates the number of requests processed per second.
Figure 7: Prompts used for the SFT stage. The {query} field is a placeholder for the user query.
Metric
Gain
p -value
95% CI
P@3
2.31%
0.0277
[0.0012, 0.0158]
R@5
1.65%
0.0483
[-0.0002, 0.0170]
Appendix
Table 8: Statistical significance between AuthGR and HyperCLOVAX 14B. Gain represents the relative difference in metrics between AuthGR and the 14B model.
Training stages
Gain
p -value
95% CI
SFT → CPT+SFT
+4.38%
<0.0001
[0.0082, 0.0221]
CPT+SFT → Full
+3.00%
<0.0001
[0.0062, 0.0155]
SFT → Full
+7.51%
<0.0001
[0.0189, 0.0334]
Appendix
Table 9: Statistical significance of the three-stage training. Gain denotes the relative improvement in metric values achieved by each incremental stage on P@3.
Scaling strategies
P@3
R@5
R@10
Binary
0.3820
0.5375
0.7133
Binning
0.3841
0.5420
0.7151
Sigmoid
0.3833
0.5377
0.7147
Linear (AuthGR)
0.3856
0.5464
0.7175
Appendix
Table 10: Performance comparison over various scaling strategies for authority scores.
Figure 8: Performance of AuthGR under different hyperparameters: (a) varying coefficient of the KL penalty β and (b) varying size of rollout group G .
Training stage
Recall
Avg. rank ( ↓ )
Avg. confidence ( ↑ )
CPT+SFT (Baseline)
7,298
3.9246
0.2263
+ GRPO (Binary)
7,438 (+1.9%)
3.9091
0.2279
+ GRPO (Linear)
7,817 (+7.1%)
3.9478
0.2288
Appendix
Table 11: Impact of GRPO on accuracy and confidence. Recall counts queries with ground-truth DocID in the outputs. Avg. rank and confidence measure the position of ground-truth DocIDs and generation probability.
Data filtering
P@3
R@5
R@10
w/o filtering
0.3194
0.4642
0.6299
w/ filtering
0.3725
0.5293
0.7031
Gain (%)
16.62
14.02
11.62
Appendix
Table 12: Performance comparison with and without data filtering in the SFT stage. “Gain” represents the relative percentage improvement achieved after filtering.
Figure 9: Performance over varying filtering thresholds. The numbers of data samples are shown in millions.
Generative Large Language Models (LLMs) have revolutionized information retrieval, yet their strictly parametric nature frequently leads to severe factual hallucinations when confronted with complex queries beyond their epistemic boundaries. While external tool-calling can mitigate this, indiscriminately invoking search tools for every document during reranking incurs prohibitive latency overheads, creating an intractable accuracy-efficiency dilemma. To address this challenge, we propose TALRanker, a novel framework that formalizes pointwise relevance scoring as an agentic Markov decision process. We optimize it via a two-stage training paradigm. An initial warm-up utilizes a language-preserving hybrid loss to prevent the catastrophic forgetting of native generative capacities. Subsequently, an asymmetric cost-aware reward equipped in reinforcement learning forces the policy to autonomously bypass tools for maximum efficiency when confident, while selectively retrieving external evidence to avert severe hallucination penalties when uncertain. Extensive evaluations demonstrate that TALRanker achieves state-of-the-art performance across standard and reasoning-intensive retrieval benchmarks, matching throughput with pointwise rerankers while outperforming parameter-heavy reasoning models.
Zichuan Liu, Ruijin Hua
Carnegie Mellon University, Pittsburgh, PA 15213, USA · Huazhong University of Science and Technology, Wuhan, China
Retrieval-augmented generation (RAG) systems rely on external corpora that may contain outdated, contradictory, noisy, or unreliable documents, introducing reliability risks. Prior work has leveraged document relations to improve the answer reliability of RAG. To propagate reliability signals beyond directly compared document pairs, we propose TrustPropRAG, which structures document relations as a graph and estimates document reliability through multi-hop propagation across the graph. TrustPropRAG anchors this propagation with a limited set of human feedback on document reliability, extending these costly-to-collect feedback-based reliability signals across the whole corpus. Specifically, based on the constructed document relation graph, TrustPropRAG estimates a trust score for each document by formulating and solving an optimization problem that jointly captures pairwise document relations and user feedback. These scores are then used to improve the selection of reliable documents and support trust-aware answer generation. Evaluation results show that TrustPropRAG improves both retrieval quality and exact match over baselines, and remains robust under sparse and noisy feedback.
Zhuoheng Li, Ying Chen
College of Information Sciences and Technology, The Pennsylvania State University
Standard Retrieval-Augmented Generation (RAG) systems predominantly rely on semantic relevance as a proxy for utility. However, this assumption collapses in realistic decision-making scenarios where user queries are laden with cognitive biases, such as false premises or confirmation bias. In such cases, maximizing relevance paradoxically promotes the retrieval of sycophantic evidence that reinforces hallucinations, a critical failure we term the ``Relevance-Robustness Gap''. To bridge this gap, we propose CoRM-RAG (Counterfactual Risk Minimization for RAG), a framework that aligns retrieval with decision safety rather than mere similarity. Grounded in causal intervention, we introduce a Cognitive Perturbation Protocol to simulate user biases during training, which is then distilled into a lightweight Evidence Critic. This scoring module learns to identify documents that possess sufficient evidential strength to steer the model toward correctness despite adversarial query perturbations. Extensive experiments on decision-making benchmarks demonstrate that CoRM-RAG significantly outperforms strong dense retrievers and LLM-based rerankers in adversarial settings, while enabling effective risk-aware abstention through reliable robustness scoring. Our code is available at https://github.com/PeiYangLiu/CoRM-RAG.git.
Peiyang Liu, Qiang Yan, Ziqiang Cui +3
National Engineering Research Center for Software Engineering, Peking University Beijing, China · PX Securities Shenzhen, China · City University of Hong Kong Hong Kong SAR, China +2