cs.CLNov 2, 2026

Re3^3Rank: Listwise Reranking with Reasoning-Conditioned Retrieval Heads

Authors: Ruiyi Yan, Zhuoyuan Mao, Yugo Murawaki, Yiwen Guo

Organizations: Kyoto University · LIGHTSPEED · Independent Researcher

Abstract

Attention-based rerankers score each candidate by the attention mass that retrieval heads route to it from the question, but the question, serving as a fixed query, carries limited information for locating relevant evidence. We propose Re3^3Rank, a listwise reranker built on reasoning-conditioned retrieval heads (RRHeads). A generator first writes a reasoning trace over the question and the candidates, and a scorer then uses the trace tokens as the attention query of a few selected RRHeads, reading relevance as continuous scores. We present a straight-through estimator to propagate the ranking loss through the discrete trace, so that the generator and the scorer can be trained jointly for ranking. Averaged over 9 benchmarks, Re3^3Rank outperforms 11 baseline rerankers: our 4B variant exceeds the strongest baseline by 3.11 NDCG@10 points on average, and even our 0.6B variant surpasses rerankers with up to 32B parameters.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. HeadRank: Decoding-Free Passage Reranking via Preference-Aligned Attention Heads

    Apr 19, 2026Juyuan Wang, Chenxing Wang, Yuchen Fang +8Attention Head AnalysisSelf-Attention

  2. EKLAV: Elicited Key CoT Learning via Answer only superVision

    Jul 28, 2026Adarsh Singh, Aman Singh, Kushal Raj Bhandari +3Table QACoT Distillation