DecisionRank: On-Policy Distillation for Decision-Model Rerankers
Organizations: Meta · Purdue University
Abstract
Decision models target structured tasks by directly predicting choices or scores, avoiding autoregressive decoding latency. However, transferring a generative LLM's ranking knowledge to a decision model remains challenging: the teacher expresses preferences through a ranked list of document ID sequences, while the student determines the ranking using scores from the first-token position. To bridge this gap, we introduce DecisionRank, a decision model trained through item-space on-policy distillation (OPD). Item-space OPD constructs the student's rollout by sorting its document scores into a ranked sequence. A generative teacher takes the resulting ranking list as inputs and provides conditional feedback to directly supervise those scores. Combined with offline ranking supervision, DecisionRank achieves state-of-the-art performance on both general and reasoning-intensive retrieval benchmarks such as TREC DL, BEIR, and BRIGHT. Across four benchmarks, item-space OPD consistently outperforms alternative training objectives. Even at 1.7B parameters, DecisionRank surpasses its 27B generative teacher on TREC DL19--23. At 8B, it outperforms its generative listwise counterpart while running faster on TREC DL and faster on BRIGHT.
Figures & tables
| General Retrieval | Reasoning-Intensive Retrieval | |||
| Configuration | TREC DL | BEIR | R2MED | BRIGHT |
| Qwen3.8-27B (teacher) | 62.97 | 57.26 | 46.66 | 37.42 |
| Zero-shot (first-token) | 47.23 | 46.99 | 31.15 | 30.88 |
| Zero-shot (sliding-window) | 58.31 | 53.40 | 35.96 | 28.42 |
| RankNet only | 63.44 | 57.39 | 43.06 | 36.06 |
| Item-space OPD only | 63.80 | 57.93 | 42.75 | 35.85 |
| Student | Objective | TREC DL | BRIGHT |
|---|---|---|---|
| Qwen3-4B | RankNet | ||
| + Item-space OPD | |||
| Qwen3-8B | RankNet | ||
| + Item-space OPD |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Learning to Rank
Learning to rank develops models that order items by relevance or utility for search, recommendations, and other decisions.
See all 293 papers