Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion

Pengcheng Jiang; Judith Yue Li; Moonkyung Ryu; R. Lily Hu; Kun Su; Zhong Yi Wan; Liam Hebert; Hao Peng

cs.IRMar 6, 2026

Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion

Authors: Pengcheng Jiang, Judith Yue Li, Moonkyung Ryu, R. Lily Hu, Kun Su, Zhong Yi Wan, Liam Hebert, Hao Peng,

Organizations: Google Research · University of Illinois Urbana Champaign

Abstract

Many modern retrieval problems are set-valued: given a broad intent, the system must return a collection of results that optimizes higher-order properties (e.g., diversity, coverage, complementarity, coherence) while remaining grounded with respect to a fixed database. Set-valued objectives are typically non-decomposable and are not captured by existing supervised (query, content) datasets which only prioritize top-1 retrieval. Consequently, fan-out retrieval is often employed to generate diverse subqueries to retrieve item sets. While reinforcement learning (RL) can optimize set-level objectives via interaction, deploying an RL-tuned LLM for fan-out retrieval is prohibitively expensive at inference time. Conversely, diffusion-based generative retrieval enables efficient single-pass fan-out in embedding space, but requires objective-aligned training targets. To address these issues, we propose R4T (Retrieve-for-Train), which uses RL once as an objective transducer in a three-step process: (i) train a fan-out LLM with composite set-level rewards, (ii) synthesize objective-consistent training pairs, and (iii) train a lightweight diffusion retriever to model the conditional distribution of set-valued outputs. Across large-scale fashion and music benchmarks consisting of curated item sets, we show that R4T improves retrieval quality relative to strong baselines while reducing query-time fan-out latency by an order of magnitude.

Figures & tables

Figure 1: Overview of the R4T. Step 1 (§ 2.3 ) trains a fan-out language model (FOLM) using RL to produce property-aligned sub-queries. Step 2 (§ 2.4 ) uses the trained FOLM to synthesize $(q,c)$ supervision data. Step 3 (§ 2.5 ) trains a diffusion-based fan-out retriever that samples content embeddings directly from query embeddings.

Figure 2: Illustration of rewards used in OAR.

Gemini-2.5-Flash
	Polyvore				Music
	Groundedness	Diversity	Alignment	Average	Groundedness	Diversity	Alignment	Average
No Fan-out	22.4	34.4	21.4	26.1	48.8	20.0	41.8	36.9
Zero-shot	24.0	47.0	23.6	31.5	45.8	45.2	44.4	45.1
Best-of-N	26.1 $\pm$ 1.6	52.2 $\pm$ 2.3	25.2 $\pm$ 1.9	34.5 $\pm$ 1.8	48.2 $\pm$ 2.1	48.4 $\pm$ 2.6	49.0 $\pm$ 2.0	48.5 $\pm$ 2.2
Gemma3-4B

Table 1: Performance on Task 1: Open-Ended Abstract Retrieval (OAR). Results are reported as mean

\pm

standard deviation. Bold denotes the best performance within each model family. R4T consistently outperforms fan-out baselines across datasets and metrics. Note that Groundedness is not applicable to R4T-Diffusion (indicated by “\”) due to the absence of intermediate sub-query.

LLM-Based Retrieval
	Recall@5K	Hit@5K	VS
Gemini-2.5-Flash	15.7	52.1	33.4
Gemma3-4B	6.0	25.9	44.2
Qwen3-4B	10.1	33.9	46.4
R4T-FOLM (Gemma)	16.9	54.4	40.5
R4T-FOLM (Qwen)	20.9	64.6	27.5

Table 2: Performance on Task 2: Weakly Supervised Compositional Retrieval (WSCR) . We use an LLM (Gemini-2.5-Pro) to generate a broad query for each outfit in Polyvore, and set the items in each as the ground truths. For diversity evaluation with normalized Vendi Score, we set temperature=0.9 for LLM-based retrieval, and set #samples=5 (i.e., 5

\times

10=50 sub-queries in total) for all the methods.

Figure 3: Qualitative comparison on the Polyvore dataset for the open-ended abstract retrieval task given the broad query “Bohemian festival style” . R4T generates semantically distinct, on-topic sub-queries that retrieve diverse outfit collections, while the Qwen3-4B zero-shot baseline produces largely paraphrastic sub-queries, leading to more homogeneous results.

Panel of Figure 4 — Figure 4: Reward & Scores over Time (FOLM Training) for Open-Ended Abstract Retrieval.

Figure 5: Efficiency comparison between Autoregressive LLM and Diffusion Model for query fan-out (

k=10

) generation. Note that the x-axis follows a logarithmic scale (doubling at each step). The Diffusion Model demonstrates superior scalability and lower latency, maintaining sub-second performance for small batches and achieving an order-of-magnitude speedup at larger batch sizes.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Fan-Out LM (RL Training)
Optimizer	AdamW ( $\beta_{1}=0.9,\beta_{2}=0.95$ )
Learning Rate	$1\times 10^{-7}$
Global Batch Size	512
Micro-batch Size	64
Grad. Accum. Steps	8

Appendix

Table 3: Implementation Hyperparameters for R4T.

Figure 6: Examples of generated subqueries and retrieved images from Polyvore dataset given a broad query “Labor day picnic outfit” for the open-ended abstract retrieval task.

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 471 papers

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA
Apr 25, 2026Teng Chen, Sheng Xu, Feixiang Guo +4Language Model Distillation Information Retrieval
Geely AI Lab Ningbo, Zhejiang, China
Retriever Portfolios: A Principled Approach to Adaptive RAG
May 29, 2026Miltiadis Stouras, Vincent Cohen-Addad, Silvio Lattanzi +1Retrieval-Augmented Generation Adaptive RAG
1EPFL · 2Google Research
Aligning Dense Retrievers with LLM Utility via Distillation
Apr 24, 2026Rajinder Sandhu, Di Mu, Cheng Chang +4Retrieval-Augmented Generation Dense Retrieval
Layer 6 AI · Toronto, ON, Canada · Dalhousie University +1

Apr 25, 2026J/K move · Enter open · S save

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA

Teng Chen, Sheng Xu, Feixiang Guo, Xiaoyu Wang+3

Geely AI Lab Ningbo, Zhejiang, China

Language Model Distillation Information Retrieval Dense Retrieval

Open paper AskPDF arXiv

Unlike traditional fact-based retrieval, rationale-based retrieval typically necessitates cross-encoding of query-document pairs using large language models, incurring substantial computational costs. To address this limitation, we propose Rabtriever, which independently encodes queries and documents, while providing comparable cross query-document comprehension capabilities to rerankers. We start from training a LLM-based generative reranker, which puts the document prior to the query and prompts the LLM to generate the relevance score by log probabilities. We then employ it as the teacher of an on-policy distillation framework, with Rabtriever as the student to reconstruct the teacher's contextual-aware query embedding. To achieve this effect, Rabtriever is first initialized from the teacher, with parameters frozen. The Joint-Embedding Predictive Architecture (JEPA) paradigm is then adopted, which integrates a lightweight, trainable predictor between LLM layers and heads, projecting the query embedding into a new hidden space, with the document embedding as the latent vector. JEPA then minimizes the distribution difference between this projected embedding and the teacher embedding. To strengthen the sampling efficiency of on-policy distillation, we also add an auxiliary loss on the reverse KL of LLM logits, to reshape the student's logit distribution. Rabtriever optimizes the teacher's quadratic complexity on the document length to linear, verified both theoretically and empirically. Experiments show that Rabtriever outperforms different retriever baselines across diverse rationale-based tasks, including empathetic conversations and robotic manipulations, with minor accuracy degradation from the reranker. Rabtriever also generalizes well on traditional retrieval benchmarks such as MS MARCO and BEIR, with comparable performance to the best retriever baseline.

Abstract

Information Retrieval

Explore similar work

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA

Retriever Portfolios: A Principled Approach to Adaptive RAG

Aligning Dense Retrievers with LLM Utility via Distillation