DenoiseRank: Learning to Rank by Diffusion Models

Ying Wang; Preslav Nakov; Shangsong Liang

cs.IRFeb 17, 2026

DenoiseRank: Learning to Rank by Diffusion Models

Authors: Ying Wang, Preslav Nakov, Shangsong Liang

Organizations: Sun Yat-sen University, Guangzhou, China · MBZUAI, Abu Dhabi, UAE

Abstract

Learning to rank (LTR) is one of the core tasks in Machine Learning. Traditional LTR models have made great progress, but nearly all of them are implemented from discriminative perspective. In this paper, we aim at addressing LTR from a novel perspective, i.e., by a deep generative model. Specifically, we propose a novel denoise rank model, DenoiseRank, which noises the relevant labels in the diffusion process and denoises them on the query documents in the reverse process to accurately predict their distribution. Our model is the first to address traditional LTR from generative perspective and is a diffusion method for LTR. Our extensive experiments on benchmark datasets demonstrated the effectiveness of DenoiseRank, and we believe it provides a benchmark for generative LTR task.

Figures & tables

Panel of Figure 1 — Figure 1: The left panel illustrates the diffusion and reverse processes in DenoiseRank, while the right panel shows the architecture of its denoising neural network. The FeedForward Layer is a linear layer, softplus is the activation function, and dropout is applied as a regularization layer. Feature transformation is applied when document features are input into the transformer block.

Method	Microsoft Web30K			Yahoo!			Istella
	@1	@5	@10	@1	@5	@10	@1	@5	@10
$\lambda\mathrm{MART}_{\mathrm{RL}}$	45.35	44.59	46.46	68.52	70.27	74.58	65.71	61.18	65.91
$\lambda\mathrm{MART}_{\mathrm{GBM}}$	50.73	49.67	51.46	71.90	74.20	78.01	74.95	71.20	76.05
DLCM	46.31	45.01	46.90	67.71	69.91	74.29	65.57	61.95	66.80
SetRank re	45.91	45.15	46.96	68.22	70.29	74.53	67.60	63.45	68.34
NeuralNDCG	–	51.45	53.49	–	66.02	71.02	–	–	–

Table 1: NDCG@K performance comparison on benchmark datasets. Best performance is bolded. * and

\dagger

denote statistically significant improvements over best tree-based and neural models respectively. Last row shows relative difference of DenoiseRank over best comparison models.

Design	Web30K	Yahoo	Istella
Noise Schedule	TruncL	TruncL	TruncL
Max Diffusion Steps	1000	1000	600
Denoising Layers	2	4	8
Self-Attention	✓	✓	✓
Loss Function	ListNet	MSE	MSE

Table 2: Recommended configurations of DenoiseRank from ablation study.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

LR	Web30K			Yahoo!			Istella
LR	@1	@5	@10	@1	@5	@10	@1	@5	@10
0.1	47.32	48.27	50.42	39.17	49.34	57.84	5.70	7.38	9.87
0.01	46.48	46.15	47.89	41.31	51.24	59.34	43.39	43.80	48.89
0.001	51.87	52.52	54.60	69.51	72.18	76.65	68.42	68.20	74.85
0.0001	51.01	51.86	54.10	71.27	73.96	78.40	69.14	69.09	75.63

Appendix

Table 3: NDCG@K performance of DenoiseRank with different learning rates on Microsoft Web30K, Yahoo!, and Istella datasets. Best performance per column in bold.

Figure 2: NDCG@K of DenoiseRank with different learning rates on Miscrosoft Web30k, Yahoo! and Istella datasets.

Panel of Figure 3 — Figure 3: Curve of training loss of DenoiseRank(Left) and Rankformer(Right) on MS Web30K and Yahoo! datasets among 300 epocs.

Subset	Web30K			Yahoo!			Istella
Subset	Queries	Docs	NOEF	Queries	Docs	NOEF	Queries	Docs	NOEF
train	18919	2.27M	85.2	19944	0.47M	222.7	19245	2.04M	115.6
vali	6306	0.75M	85.4	2994	0.07M	223.2	7211	0.68M	114.8
test	6306	0.75M	85.3	6983	0.17M	222.5	6562	0.68M	115.5
total	31531	3.77M	85.2	29921	0.71M	222.8	33018	3.41M	115.4

Appendix

Table 4: Statistics of different dataset, including queries number, docments number and the average number of effective feature(NOEF).

Dataset	Low			High
Dataset	S1	S2	S3	S4	S5	S6
Web30K	76.33	76.35	76.66	94.17	94.18	94.02
YAHOO	191.38	191.10	190.61	255.36	254.31	254.52
ISTELLA	109.63	109.65	109.63	121.53	121.51	121.44

Appendix

Table 5: Split result of the train.txt files of the three benchmark( Microsoft Web30K, Yahoo!, and Istella). Low means lower effective feature training resources, High means higher effective feature training resources. The numerical representation average number of effective features(NOEF)

Figure 4: Control group experimental results for effective feature counts, NDCG@K performance produce by different models on Miscrosoft Web30k, Yahoo! and Istella datasets. Low means lower effective feature training resources, High means higher effective feature training resources.

Figure 5: Statistical and Experimental Results of Query-Document Length analysis. First row of figure: Query-Documents length distributions. Subsequent row of figure: NDCG@K of models(DenoiseRank, LightGBM, DASALC) under different length on Miscrosoft Web30k, Yahoo! and Istella datasets.

Timesteps	Web30K			Yahoo!			Istella
Timesteps	@1	@5	@10	@1	@5	@10	@1	@5	@10
1000	51.87	52.52	54.60	71.27	73.96	78.40	69.14	69.09	75.63
800	51.49	52.03	54.10	70.84	73.70	78.22	69.25	69.13	75.67
600	50.90	51.58	53.53	70.82	73.74	78.25	69.09	69.21	75.68
400	50.52	49.82	51.66	70.87	73.99	78.35	69.47	68.97	75.52
200	49.87	49.33	51.11	70.71	73.82	78.23	69.41	68.77	75.35

Appendix

Table 6: NDCG@K performance with different diffusion timesteps on Microsoft Web30K, Yahoo!, and Istella datasets. Best performance per column in bold.

Panel of Figure 6 — Figure 6: NDCG@K of DenoiseRank at different noise schedule on Miscrosoft Web30k, Yahoo! and Istella datasets.

Panel of Figure 7 — Figure 7: NDCG@K of DenoiseRank at different noise schedule on Miscrosoft Web30k, Yahoo! and Istella datasets.

Scheduler	Web30K			Yahoo!			Istella
Scheduler	@1	@5	@10	@1	@5	@10	@1	@5	@10
TruncatedLinear	51.87	52.52	54.60	71.27	73.96	78.40	69.14	69.09	75.63
Linear	50.40	50.25	52.10	71.20	73.89	78.33	69.04	68.91	75.48
Cosine	46.83	46.16	48.04	71.31	73.91	78.36	68.59	68.38	74.89
Sqrt	50.40	50.15	52.08	71.58	74.00	78.39	68.97	68.93	75.40

Appendix

Table 7: NDCG@K performance with different noise schedulers on Microsoft Web30K, Yahoo!, and Istella datasets. Best performance per column in bold.

Panel of Figure 8 — Figure 8: NDCG@K of DenoiseRank at different denoise-net design on Miscrosoft Web30k, Yahoo! and Istella datasets.

Layers	Web30K			Yahoo!			Istella
Layers	@1	@5	@10	@1	@5	@10	@1	@5	@10
2	51.87	52.52	54.60	71.37	74.06	78.42	69.00	69.10	75.69
4	51.45	52.08	54.03	71.37	74.06	78.42	69.54	69.14	75.65
6	50.45	51.36	53.32	69.03	72.09	76.51	69.41	69.01	75.65
8	50.08	51.73	53.82	69.73	72.13	76.58	69.46	69.17	75.80

Appendix

Table 8: NDCG@K Performance with different denoise network depths on Microsoft Web30K, Yahoo!, and Istella datasets. Best performance per column in bold.

Figure 9: NDCG@K of DenoiseRank without and with self attention on Miscrosoft Web30k, Yahoo! and Istella datasets.

Self-Attention	Web30K			Yahoo!			Istella
Self-Attention	@1	@5	@10	@1	@5	@10	@1	@5	@10
Without	48.25	47.08	49.00	71.37	73.35	77.70	63.91	63.91	70.65
With	51.87 $\uparrow$	52.52 $\uparrow$	54.60 $\uparrow$	71.37 $\uparrow$	74.06 $\uparrow$	78.42 $\uparrow$	69.46 $\uparrow$	69.17 $\uparrow$	75.80 $\uparrow$

Appendix

Table 9: NDCG@K performance with/without self-attention on Microsoft Web30K, Yahoo!, and Istella datasets. Best performance per column in bold.

\uparrow

denotes significant improvements.

Figure 10: A t-SNE plot shows the diverse ranking sequences on the top 20 predicted in the inference stage of a single query randomly selected. The blue points denote the ranking sequences inferred by DenoiseRank using 100 different $Y_{T}$ values from Gaussian noise. The orange yellow represents the other sequences predicted by Rankformer in 100 attempts. Testing was conducted on the MS Web30K dataset.

Model	M	RSD@(K,M)				NDCG@K
Model	M	1	5	10	20	1	5	10	20
Rankformer	1	–	–	–	–	49.62	49.30	51.42	54.29
Rankformer	10	0.1	0.1	0.1	0.1	49.62	49.30	51.42	54.29
DenoiseRank	1	–	–	–	–	51.48	52.46	54.47	57.49
DenoiseRank	10	0.11	0.16	0.28	0.64	51.73	52.52	54.47	57.45

Appendix

Table 10: NDCG@K and RSD@(K,M) performance of DenoiseRank and Rankformer on Microsoft Web30K datasets.

Loss	Web30K			Yahoo!			Istella
Loss	@1	@5	@10	@1	@5	@10	@1	@5	@10
RMSE	50.48	51.41	53.43	70.64	73.30	77.84	69.26	69.32	75.80
RankNet	43.66	45.84	48.56	56.65	66.96	73.18	51.11	57.25	65.83
NDCGLoss 2++	43.01	47.68	50.57	66.48	72.18	76.98	56.00	59.96	67.55
ApproxNDCG	24.46	29.21	33.72	60.24	64.67	70.64	33.01	42.21	52.48
ListNet	51.87	52.52	54.60	70.81	73.82	78.35	68.75	69.05	75.58

Appendix

Table 11: NDCG@K performance of DenoiseRank with different loss functions on Microsoft Web30K, Yahoo!, and Istella datasets. Best performance per column in bold.

Metric	K	Dataset			Metric	K	Dataset
Metric	K	Web30K	Yahoo!	Istella	Metric	K	Web30K	Yahoo!	Istella
ERR	1	26.53	34.41	61.53	MAP	1	78.07	87.13	94.64
	5	36.77	43.90	73.79		5	81.55	89.16	95.19
	10	38.55	45.34	74.34		10	78.76	87.91	93.14
	20	39.28	45.72	74.40		20	74.89	86.72	90.27
	ALL	39.57	45.78	74.40		ALL	63.91	85.75	88.36

Appendix

Table 12: ERR, MRR, MAP and precision of DenoiseRank on Microsoft Web30K, Yahoo!, and Istella datasets.

Step	2	4	8	16	32	64	128	256	512
Times(s)	4.75	5.28	6.42	8.76	13.44	23.57	44.11	85.04	166.73

Appendix

Table 13: Total inference time(s) at different reverse steps of DenoiseRank on Yahoo datasets.

Step	2	4	8	16	32	64	128	256	512
MS Web30K	54.59	54.60	54.62	54.61	54.57	54.61	54.58	54.61	54.59
Yahoo!	78.41	78.40	78.42	78.38	78.37	78.36	78.38	78.39	78.38
Istella	75.80	75.81	75.82	75.79	75.80	75.79	75.80	75.84	75.82

Appendix

Table 14: NDCG@10 of DenoiseRank at different reverse steps.

Stage	DenoiseRank	Rankformer	NeuralNDCG	$\lambda\mathrm{MART}_{\mathrm{RankLib}}$	$\lambda\mathrm{MART}_{\mathrm{GBM}}$
Inference(ms)	0.92	3.35	0.68	1.34	0.04
Training(s)	13.88	15.46	11.23	1.38	0.20

Appendix

Table 15: Inference and training times of baselines on Yahoo datasets.

Explore similar work

Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation
Apr 16, 2026Camilo Gomez, Pengyang Wang, Yanjie FuLearning to Rank
Dept. of Statistics & Data Science University of Central Florida Orlando, USA · Department of CIS, SKL-IOTSC University of Macau Macao, China · School of Computing and AI Arizona State University Tempe, USA
Active Learners as Efficient PRP Rerankers
May 14, 2026Jeremías Figueiredo Paschmann, Juan Kaplan, Francisco Nattero +3Pairwise Preference Learning Pairwise Comparison
ELIAS Lab, Departamento de Ingeniería, Universidad de San Andrés
Diffusion-GR2: Diffusion Generative Reasoning Re-ranker
Jul 1, 2026Zhuoxuan Zhang, Kangqi Ni, Yuhang Chen +12Blockwise Diffusion Language Models Efficient Language Model Reasoning
Yang · 2UNC Chapel Hill · 1Meta AI

Apr 16, 2026J/K move · Enter open · S save

Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation

Camilo Gomez, Pengyang Wang, Yanjie Fu

Dept. of Statistics & Data Science University of Central Florida Orlando, USA · Department of CIS, SKL-IOTSC University of Macau Macao, China · School of Computing and AI Arizona State University Tempe, USA

Learning to Rank

Open paper AskPDF arXiv

Learning-to-Rank (LTR) is a supervised machine learning approach that constructs models specifically designed to order a set of items or documents based on their relevance or importance to a given query or context. Despite significant success in real-world information retrieval systems, current LTR methods rely on one prefix ranking metric (e.g., such as Normalized Discounted Cumulative Gain (NDCG) or Mean Average Precision (MAP)) for optimizing the ranking objective function. Such metric-dependent setting limits LTR methods from two perspectives: (1) non-differentiable problem: directly optimizing ranking functions over a given ranking metric is inherently non-smooth, making the training process unstable and inefficient; (2) limited ranking utility: optimizing over one single metric makes it difficult to generalize well to other ranking metrics of interest. To address the above issues, we propose a novel listwise LTR framework for efficient and generalizable ranking purpose. Specifically, we propose a new differentiable ranking loss that combines a smooth approximation to the ranking operator with the average mean square loss per query. Then, we adapt gradient-boosting machines to minimize our proposed loss with respect to each list, a novel contribution. Finally, extensive experimental results confirm that our method outperforms the current state-of-the-art in information retrieval measures with similar efficiency.

Abstract

Explore similar work

Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation

Active Learners as Efficient PRP Rerankers

Diffusion-GR2: Diffusion Generative Reasoning Re-ranker