From Relevance to Authority: Authority-aware Generative Retrieval in Web Search Engines

Sunkyung Lee; Jihye Back; Donghyeon Jeon; Soonhwan Kwon; Moonkwon Kim; Inho Kang; Jongwuk Lee

cs.IRApr 15, 2026

From Relevance to Authority: Authority-aware Generative Retrieval in Web Search Engines

Authors: Sunkyung Lee, Jihye Back, Donghyeon Jeon, Soonhwan Kwon, Moonkwon Kim, Inho Kang, Jongwuk Lee

Organizations: Sungkyunkwan University, Republic of Korea · Naver Corporation, Republic of Korea

Abstract

Generative information retrieval (GenIR) formulates the retrieval process as a text-to-text generation task, leveraging the vast knowledge of large language models. However, existing works primarily optimize for relevance while often overlooking document trustworthiness. This is critical in high-stakes domains like healthcare and finance, where relying solely on semantic relevance risks retrieving unreliable information. To address this, we propose an Authority-aware Generative Retriever (AuthGR), the first framework that incorporates authority into GenIR. AuthGR consists of three key components: (i) Multimodal Authority Scoring, which employs a vision-language model to quantify authority from textual and visual cues; (ii) a Three-stage Training Pipeline to progressively instill authority awareness into the retriever; and (iii) a Hybrid Ensemble Pipeline for robust deployment. Offline evaluations demonstrate that AuthGR successfully enhances both authority and accuracy, with our 3B model matching a 14B baseline. Crucially, large-scale online A/B tests and human evaluations conducted on the commercial web search platform confirm significant improvements in real-world user engagement and reliability.

Figures & tables

Figure 1: Illustration of our motivation. (a) Models relying solely on relevance fail to distinguish an unreliable blog from authoritative ones. (b) By integrating authority, our model can prioritize trustworthy documents.

Figure 2: Overall architecture of AuthGR. (a) Multimodal Authority Scoring quantifies document trustworthiness based on textual and visual signals. (b) Three-stage training pipeline progressively instills authority awareness into the retriever. (c) The Hybrid Ensemble Pipeline integrates AuthGR with existing rankers for deployment.

Figure 3: Illustration of the GRPO stage.

In-context Learning (ICL)
Model	Size	P@3	R@5	R@10
Gemma 3	27B	0.1255	0.1732	0.2285
EXAONE 3.5	32B	0.0825	0.1253	0.1612
K-EXAONE	236B	0.1366	0.1918	0.2656
Qwen3	32B	0.0821	0.1176	0.1570
LLaMA 3.1	405B	0.1413	0.1974	0.2590

Table 1: Offline evaluation results. Bold denotes the best performance of our method. ‘AuthGR (Full)’ denotes the final model incorporating CPT, SFT, and GRPO.

	Production	Hybrid Ensemble
Label score	3.06	3.41

Table 2: Human evaluation results in a blind side-by-side test, comparing (i) the production system and (ii) its integration with AuthGR via Hybrid Ensemble.

Metrics	Control	Treatment
Pages with clicks	+0.08%	+21.36%
Total document clicks	+0.22%	+22.07%
Top 1 document CTR	+0.87%	+22.83%
Top 3 document CTR	+0.81%	+22.68%
Top 5 document CTR	+0.81%	+22.76%

Table 3: Online A/B test results showing relative gain in user engagement. ‘Control’ group uses the production system. ‘Treatment’ group receives results enhanced by AuthGR through the Hybrid Ensemble.

Model	SFT	CPT	GRPO	P@3	R@5	R@10
AuthGR	✓			0.3470	0.4933	0.6634
(0.5B)	✓		✓	0.3515	0.4989	0.6651
AuthGR (3B)	✓			0.3555	0.5058	0.6899
	✓		✓	0.3660	0.5216	0.6986
	✓	✓		0.3725	0.5293	0.7031
	✓	✓	✓	0.3856	0.5464	0.7175

Table 4: Ablation study of the training stages.

Figure 4: Performance of ranking optimization methods. ‘CPT+SFT+GRPO’ represents our AuthGR.

Training stage	Authority scores		# docs by authority
Training stage	Mean( $\uparrow$ )	Median( $\uparrow$ )	Low( $\downarrow$ )	High( $\uparrow$ )
CPT+SFT	87.2	90.0	118	2,754
+ GRPO (Binary)	88.0	90.0	115	2,804
+ GRPO (Linear)	90.4	95.0	106	2,877

Table 5: Authority score distribution in generated documents. Mean and Median are computed on a 0-100 scale. ‘Low’ and ‘High’ indicate the number of documents scoring 0-60 and 90-100, respectively.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Value
Point-biserial correlation ( $r$ )	0.495 ( $p<0.001$ )
ROC-AUC	0.915
Matthews Correlation (MCC)	0.526

Appendix

Table 6: Quantitative alignment between VLM-generated authority scores and human expertise labels.

Figure 5: Examples where the Text+Image input yields more appropriate results than the image-only input.

Panel of Figure 6 — Figure 6: Prompts used for Multimodal Authority Scoring.

Model	Size	Latency (ms)	Throughput
HyperCLOVAX (SFT)	14B	2,881	1.30
AuthGR (Ours)	3B	1,225	3.26

Appendix

Table 7: Comparison of inference efficiency. Latency denotes the average response time, and throughput indicates the number of requests processed per second.

Panel of Figure 7 — Figure 7: Prompts used for the SFT stage. The {query} field is a placeholder for the user query.

Metric	Gain	$p$ -value	95% CI
P@3	2.31%	0.0277	[0.0012, 0.0158]
R@5	1.65%	0.0483	[-0.0002, 0.0170]

Appendix

Table 8: Statistical significance between AuthGR and HyperCLOVAX 14B. Gain represents the relative difference in metrics between AuthGR and the 14B model.

Training stages	Gain	$p$ -value	95% CI
SFT $\to$ CPT+SFT	+4.38%	<0.0001	[0.0082, 0.0221]
CPT+SFT $\to$ Full	+3.00%	<0.0001	[0.0062, 0.0155]
SFT $\to$ Full	+7.51%	<0.0001	[0.0189, 0.0334]

Appendix

Table 9: Statistical significance of the three-stage training. Gain denotes the relative improvement in metric values achieved by each incremental stage on P@3.

Scaling strategies	P@3	R@5	R@10
Binary	0.3820	0.5375	0.7133
Binning	0.3841	0.5420	0.7151
Sigmoid	0.3833	0.5377	0.7147
Linear (AuthGR)	0.3856	0.5464	0.7175

Appendix

Table 10: Performance comparison over various scaling strategies for authority scores.

Figure 8: Performance of AuthGR under different hyperparameters: (a) varying coefficient of the KL penalty $\beta$ and (b) varying size of rollout group $G$ .

Training stage	Recall	Avg. rank ( $\downarrow$ )	Avg. confidence ( $\uparrow$ )
CPT+SFT (Baseline)	7,298	3.9246	0.2263
+ GRPO (Binary)	7,438 (+1.9%)	3.9091	0.2279
+ GRPO (Linear)	7,817 (+7.1%)	3.9478	0.2288

Appendix

Table 11: Impact of GRPO on accuracy and confidence. Recall counts queries with ground-truth DocID in the outputs. Avg. rank and confidence measure the position of ground-truth DocIDs and generation probability.

Data filtering	P@3	R@5	R@10
w/o filtering	0.3194	0.4642	0.6299
w/ filtering	0.3725	0.5293	0.7031
Gain (%)	16.62	14.02	11.62

Appendix

Table 12: Performance comparison with and without data filtering in the SFT stage. “Gain” represents the relative percentage improvement achieved after filtering.

Figure 9: Performance over varying filtering thresholds. The numbers of data samples are shown in millions.

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 471 papers

Tool-Adaptive LLM Reranker
Jul 12, 2026Zichuan Liu, Ruijin HuaLLM Hallucination Mitigation LLM Reranking
Carnegie Mellon University, Pittsburgh, PA 15213, USA · Huazhong University of Science and Technology, Wuhan, China
Feedback-Assisted Trust Propagation over Document Relation Graphs for Retrieval-Augmented Generation
Sep 1, 2026Zhuoheng Li, Ying ChenRetrieval-Augmented Generation Graph Retrieval-Augmented Generation
College of Information Sciences and Technology, The Pennsylvania State University
Beyond Semantic Relevance: Counterfactual Risk Minimization for Robust Retrieval-Augmented Generation
May 2, 2026Peiyang Liu, Qiang Yan, Ziqiang Cui +3Retrieval-Augmented Generation LLM Hallucination Mitigation
National Engineering Research Center for Software Engineering, Peking University Beijing, China · PX Securities Shenzhen, China · City University of Hong Kong Hong Kong SAR, China +2

Jul 12, 2026J/K move · Enter open · S save

Tool-Adaptive LLM Reranker

Zichuan Liu, Ruijin Hua

Carnegie Mellon University, Pittsburgh, PA 15213, USA · Huazhong University of Science and Technology, Wuhan, China

LLM Hallucination Mitigation LLM Reranking LLM Tool Use

Open paper AskPDF arXiv

Generative Large Language Models (LLMs) have revolutionized information retrieval, yet their strictly parametric nature frequently leads to severe factual hallucinations when confronted with complex queries beyond their epistemic boundaries. While external tool-calling can mitigate this, indiscriminately invoking search tools for every document during reranking incurs prohibitive latency overheads, creating an intractable accuracy-efficiency dilemma. To address this challenge, we propose TALRanker, a novel framework that formalizes pointwise relevance scoring as an agentic Markov decision process. We optimize it via a two-stage training paradigm. An initial warm-up utilizes a language-preserving hybrid loss to prevent the catastrophic forgetting of native generative capacities. Subsequently, an asymmetric cost-aware reward equipped in reinforcement learning forces the policy to autonomously bypass tools for maximum efficiency when confident, while selectively retrieving external evidence to avert severe hallucination penalties when uncertain. Extensive evaluations demonstrate that TALRanker achieves state-of-the-art performance across standard and reasoning-intensive retrieval benchmarks, matching throughput with pointwise rerankers while outperforming parameter-heavy reasoning models.

Abstract

Information Retrieval

Explore similar work

Tool-Adaptive LLM Reranker

Feedback-Assisted Trust Propagation over Document Relation Graphs for Retrieval-Augmented Generation

Beyond Semantic Relevance: Counterfactual Risk Minimization for Robust Retrieval-Augmented Generation