No Single Best Model for Diversity: Learning a Router for Sample Diversity

Yuhan Liu; Fangyuan Xu; Vishakh Padmakumar; Daphne Ippolito; Eunsol Choi

cs.CLApr 2, 2026

No Single Best Model for Diversity: Learning a Router for Sample Diversity

Authors: Yuhan Liu, Fangyuan Xu, Vishakh Padmakumar, Daphne Ippolito, Eunsol Choi

Organizations: New York University · Stanford University · Carnegie Mellon University

Abstract

When posed with prompts that permit a large number of valid answers, comprehensively generating them is the first step towards satisfying a wide range of users. In this paper, we study methods to elicit a comprehensive set of valid responses. To evaluate this, we introduce diversity coverage, a metric that measures the total quality scores assigned to each unique answer in the predicted answer set relative to the best possible answer set with the same number of answers. Using this metric, we evaluate 18 LLMs, finding no single model dominates at generating diverse responses to a wide range of open-ended prompts. Yet, per each prompt, there exists a model that outperforms all other models significantly at generating a diverse answer set. Motivated by this finding, we introduce a router that predicts the best model for each query. On NB-WildChat, our trained router outperforms the single best model baseline (26.3% vs 23.8%). We further show generalization to an out-of-domain dataset (NB-Curated) as well as different answer-generation prompting strategies. Our work lays foundation for studying generating comprehensive answers when we have access to a suite of models.

Figures & tables

Figure 1: Left : LLMs exhibit different diversity coverage. Right : There is no universal best model on NB-WildChat . A model is only considered to be the best model if its diversity scores are $5\%$ higher than the second-best candidate. Queries without a model satisfying this margin are labeled as “No dominant single models”. On Simple Questions , all models perform similarly, resulting in $100\%$ of “No dominant single models”. On NB-WildChat , there is no model that consistently dominates all queries. In Appendix A , we take a look on frontier versions of GPT, Claude and Gemini and find no dominant LLM either.

Dataset	# Q	Example
	# Q	Question	Possible Valid Answers (delimited with ‘/’)
Simple Questions ( Zhang et al., 2024 )	$23$	Output a random country in North America.	United States/ Canada/ Mexico / …
NB-Curated 1 1 1 The original released dataset has 100 questions. We filter out 4 questions that ask for multiple answers (violating our hypothesis) and 4 questions that do not have multiple correct answers. ( Zhang et al., 2025b )	$92$	Tell me a funny joke.	Who is Adam and why he is optimizing my code? / “Pick a lane” “Who’s Elaine?”
NB-WildChat ( Zhang et al., 2025b )	$1$ k	Give me a very simple way to remember the formula for tangent.	Tangent = $\frac{\text{Opposite}}{\text{Adjacent}}$ / Draw a unit circle …
Infinity-Chat ( Jiang et al., 2026 )	$26$ k	Write a story about America.	”… I’m just like my country—I’m young, scrappy, and hungry, and I’m not throwin’ away my shot!” said Hamilton./ America was born in the streets.

Table 1: Dataset statistics. # Q denotes the number of total questions. Simple Questions has a fixed, finite answer set; all other datasets have open-ended answer sets.

	SQ	Curated	WildChat
Top overall model	$96.9\%$	$47.0\%$	$23.8\%$
Top two overall models	$97.1\%$	$45.6\%$	$25.6\%$
Random model / query	$92.7\%$	$37.5\%$	$18.1\%$
Top model / query	97.9%	59.6%	33.0%

Table 2: Diversity coverage scores for ensembling multiple LLMs on Simple Questions (SQ), NB-Curated (Curated) and NB-WildChat (WildChat).

Method	NB-WildChat				NB-Curated (OOD)
Method	#Unq	Qual	Unq Qual	Cov.	#Unq	Qual	Unq Qual	Cov.
Top overall	42.6	3.0	2.9	23.8%	35.4	6.0	5.7	38.6%
Frequency	33.1	3.8	3.6	21.0%	28.2	7.2	7.1	39.6%
Random model per query	27.8	3.7	3.6	18.1%	27.8	7.2	7.0	37.5%
Top model per query (oracle)	38.8	4.5	4.4	33.0%	30.3	7.6	7.4	59.6%
KNN (N=1)	34.3	3.7	3.6	23.1%	28.2	7.3	7.1	39.7%

Table 3: A per-query router selects among 18 models to maximize diversity coverage (Cov.). We train our best MLP router for 5 runs with random seeds to compute statistical significance for our best system ( bolded ) against Top overall , ∗∗ indicating significantly better and ns indicating not significant.

Method	NB-WildChat				NB-Curated (OOD)
Method	#Unq	Qual	Unq Qual	Cov.	#Unq	Qual	Unq Qual	Cov.
Top 2 overall	39.1	3.4	3.2	23.8%	31.6	6.7	6.3	38.3%
Top 2 per query	40.7	4.5	4.5	35.8%	41.3	7.7	7.6	62.6%
Router	38.4	3.8	3.6	26.7% ∗∗	32.3	7.1	6.8	42.2% ∗∗

Table 4: Performance of ensembling two models per query. We report the performance of the best single model router (Binary MLP (spec)) in Table 3 by ensembling the top 2 models ranked by the prediction scores. ∗∗ indicating significantly better compared to Top 2 overall .

Figure 2: Scaling training data improves router performance on Infinity-Chat .

Figure 3: Distribution of quality scores for unique answers on the WildChat dataset across different routing methods. The oracle router achieves the highest mean quality, while the learned router attains relatively high mean quality with low variability.

Figure 4: Efficiency analysis comparing the time (seconds per query) across routing (Router), Top overall (Top) and Top model per query (Oracle). We include routing to one model per query and routing to 2 models per query. Sample n denotes sample

n

answers. Oracle incurs the highest cost, as the routing requires exhaustively comparing all candidate models or model pairs.

Method	Prompt Template
Method	Gen 1	Gen 2	Gen All
Top overall	18.5%	19.7%	23.8%
Random	9.9%	13.2%	18.1%
Frequency	15.6%	17.1%	21.0%
Oracle (G-1)	25.6%	22.3%	20.4%
Oracle (G-2)	19.5%	28.3%	21.0%

Table 5: Div-Cov (%) results on NB-WildChat with various prompting strategies. Training the router under each prompting strategy ( in domain and out-of-domain evaluation).

Figure 5: Generate-one prompt has higher answer quality. Under the generate-all prompts, as more answers are listed, the quality decreases with large variations if using generate-all prompt.

Appendix figures & tables34 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Frequency(%) of each LLM being the most diverse model.A model is only considered to be the best model if its diversity coverage is

10\%

higher than the second best candidate. Queries with no meaningful gap are labeled as “No dominant single models”. On NB-WildChat , there is no model that consistently dominates all queries. On Simple Questions , all models have similar diversity coverage.

Figure 7: Frequency(%) of each frontier LLM being the most diverse model. The threshold for deciding dominance is

5\%

Model	Cov.	Qual	#Unq	Avg #Ans/Gen	Avg Ans. Len
Closed Source
claude-sonnet-4-6	20.83%	5.0	24.2	7.2	220.4
gemini-3.1-flash-lite-preview	21.48%	4.4	27.5	6.6	62.1
gemini-3.1-pro-preview	19.78%	4.4	23.7	6.7	162.1
gpt-4o-mini	18.79%	3.9	28.1	8.7	44.4
gpt-5.4	21.06%	4.7	27.0	7.4	262.2

Appendix

Table 6: Comparing 5 frontier models with 18 open models in our routing pool. Performance is calculated on test split of NB-WildChat .

Model	Mean LL	%Best	%Dom.	Raw Performance (Acc)
gemma-3-1b-it	$-$ 2.553	2.14%	2.05%	38.66%
Qwen3-0.6B	$-$ 2.553	0.28%	0.26%	40.34%
OLMo-2-0425-1B-Instruct	$-$ 1.489	1.05%	0.93%	40.61%
Llama-3.2-1B-Instruct	$-$ 1.245	0.80%	0.67%	48.40%
Qwen3-1.7B	$-$ 2.089	1.67%	1.58%	55.44%
gemma-3-4b-it	$-$ 2.495	4.70%	4.50%	57.57%

Appendix

Table 7: MMLU performance for the 18 open source models.%Best denotes percentage of queries with highest log likelihood. %Dom. denotes percentage of queries it dominates.

Model	Mean LL	%Best	%Dom.	Raw Performance (Acc)
Llama-3.2-1B-Instruct	$-$ 9.752	0.15%	0.15%	16.38%
gemma-3-1b-it	$-$ 11.766	0.15%	0.15%	24.49%
Qwen3-0.6B	$-$ 8.773	0.15%	0.08%	32.83%
Llama-3.2-3B-Instruct	$-$ 7.343	0.23%	0.23%	37.76%
Llama-3.1-8B-Instruct	$-$ 7.059	0.08%	0.08%	47.76%
OLMo-2-0425-1B-Instruct	$-$ 8.811	0.08%	0.08%	50.80%

Appendix

Table 8: GSM8K performance for the 18 open source models.

Panel of Figure 8 — Figure 8: Prompt, generate one, simple questions

Panel of Figure 9 — Figure 9: Prompt, generate two, simple questions

Panel of Figure 10 — Figure 10: Prompt, generate all, simple questions

Panel of Figure 11 — Figure 11: Prompt, generate one, open-ended questions

Panel of Figure 12 — Figure 12: Prompt, generate two, open-ended questions

Panel of Figure 13 — Figure 13: Prompt, generate all, open-ended questions

Panel of Figure 14 — Figure 14: Prompt_verbalized_all, simple questions

Panel of Figure 15 — Figure 15: Prompt_verbalized_all, open-ended questions

Panel of Figure 16 — Figure 16: System_vanilla, simple questions and open-ended questions

Panel of Figure 17 — Figure 17: System_verbalized_all, simple questions and open-ended questions

			Evaluation Data
Method			NB-WildChat	Infinity-Chat
Random			18.13%	18.24%
Top Overall			23.83%	23.13%
Oracle			33.04%	30.50%
	Training Data	Size
Router	NB-WildChat	500	25.28% $\scriptstyle\pm 0.28\%$	22.58% $\scriptstyle\pm 0.30\%$

Appendix

Table 9: Router performance (diversity coverage) steadily improves with more training data. We report the average and variance of 5 training runs with different random seeds.

Method	NB-WildChat					NB-Curated (OOD)
Method	Acc	#U	Q	UQ	Cov.	Acc	#U	Q	UQ	Cov.
Top Overall	19.5%	42.6	3.0	2.9	23.8%	3.4%	35.4	6.0	5.7	38.6%
Random M / Q	5.9%	27.8	3.7	3.6	18.1%	5.6%	27.8	7.2	7.0	37.5%
Frequency	12.0%	33.1	3.8	3.6	21.0%	9.0%	28.2	7.2	7.1	39.6%
Top M / Q (oracle)	100%	38.8	4.5	4.4	33.0%	100%	30.3	7.6	7.4	59.6%
$1$ NN	16.5%	34.3	3.7	3.6	23.1%	5.6%	28.2	7.3	7.1	39.7%

Appendix

Table 10: A per-query router selecting over 18 models to maximize diversity coverage (Cov.).#U denotes number of unique outputs, Q denotes average quality, and UQ denotes quality of unique outputs. Accuracy(Acc) measures how frequently the router predicts the oracle model(ground truth target). Random M / Q denotes random model per query, and Top M / Q denotes top model per query.

Figure 18: Distribution of quality scores for all generated answers on NB-WildChat across different routing methods.

Figure 19: Distribution of raw answer quality scores across the 18 models on NB-WildChat .

Figure 20: Distribution of unique-answer quality scores across the 18 models on NB-WildChat .

Method	Gen 1			Gen 2			Gen All
Method	#Unq	Quality	Cov.	#Unq	Quality	Cov.	#Unq	Quality	Cov.
Random	13.7	4.9	9.9%	18.3	4.5	13.2%	27.8	3.7	18.1%
Frequency	25.4	3.9	15.6%	25.2	4.1	17.1%	33.1	3.8	21.0%
Top Overall	31.8	3.2	18.5%	34.8	3.1	19.7%	42.6	3.0	23.8%
Oracle (G-1)	32.8	4.1	25.6%	32.1	3.7	22.3%	31.0	3.3	20.4%
Oracle (G-2)	24.1	4.3	19.5%	32.3	4.7	28.3%	31.6	3.3	21.0%

Appendix

Table 11: Training the router under different prompting strategies ( in domain and out-of-domain evaluation) on NB-WildChat . Router (X) is a router trained under prompt type X. Oracle (X) denotes that we always use ground truth labels of prompt X as predictions. Training a router improves diversity for all prompts, as all routers beat their Top Overall baselines. However, different prompt templates seem to elicit different levels of diversity in LLMs, as the oracle predictions don’t generalize across prompts.

	Gen 1		Gen 2		Gen All
	Cov.	Len	Cov.	Len	Cov.	Len
Random	$16.7\%$	$49.3$	$22.4\%$	$37.0$	$37.5\%$	$17.1$
Top model	$33.5\%$	$47.7$	$35.4\%$	$38.8$	$47.0\%$	$22.6$
Top 2 models	$32.7\%$	$31.6$	$36.1\%$	$34.2$	$45.6\%$	$24.0$
Top model per query	$43.6\%$	$37.8$	$46.6\%$	$30.3$	$59.6\%$	$21.9$

Appendix

Table 12: Oracle diversity coverage (Cov.) and answer lengths (Len) for different prompt templates on NB-Curated Questions.

Figure 21: Average answer quality for responses generated from different prompts on NB-Curated questions.

Ratio	#Unq	Qual	Unq Qual	Cov.
Oracle model pair
0:50	42.50	4.16	3.95	35.40%
5:45	43.90	4.11	3.94	36.82%
10:40	45.00	4.13	3.99	37.56%
15:35	45.50	4.09	3.88	37.10%
20:30	46.30	3.93	3.80	36.46%

Appendix

Table 13: Exploring different ratios while varying models per question. Performance is reported on 10 questions sampled from NB-WildChat . Top 2 models are olmo-2-0425-1b and olmo-2-0325-32b .

Strategy	#Unq	Qual	Unq Qual	Cov.
Oracle ratio	44.80	3.66	3.55	32.58%
Overall best (40:10)	46.20	2.99	2.98	27.62%
Half/half	43.60	3.26	3.12	26.98%

Appendix

Table 14: Always use top 2 models ( olmo-2-0425-1b and olmo-2-0325-32b ), while varying ratios per question.

$N$	#Unq	Qual	Unq Qual	Cov.
1	42.50	4.16	3.95	35.40%
2	45.30	4.00	3.87	36.08%
3	44.00	4.04	3.86	35.72%
4	44.20	4.13	3.79	35.96%
5	43.80	4.13	3.79	35.74%
6	44.30	3.93	3.58	35.04%

Appendix

Table 15: Oracle models: fix

N

models to select for all questions and vary model choices per question.

$N$	#Unq	Qual	Unq Qual	Cov.
1	47.20	2.85	2.88	27.10%
2	43.60	3.26	3.12	26.98%
3	41.80	3.21	3.06	24.90%
4	41.90	3.31	3.07	25.40%
5	40.60	3.47	3.27	25.42%
6	40.70	3.53	3.34	26.60%

Appendix

Table 16: Top

N

models: fix

N

models to select for all questions and vary model choices per question.

$N$	#Unq	Qual	Unq Qual	Cov.
1	35.93	3.24	3.23	20.86%
2	37.72	3.22	3.19	21.85%
3	38.63	3.23	3.17	22.38%
4	39.03	3.21	3.13	22.60%
5	39.11	3.22	3.13	22.89%
6	38.89	3.29	3.18	23.36%

Appendix

Table 17: Random models: fix

N

models to select for all questions and vary model choices per question.

Strategy	#Unq	Qual	Unq Qual	Cov.
Oracle $N$	44.10	3.93	3.73	34.24%
Best overall ( $N=8$ )	40.00	3.65	3.54	27.26%
Random $N$	39.23	3.53	3.35	25.25%

Appendix

Table 18: Fix model order to ensemble (ranking of individual performance) and vary

N

per question.

Prompt	Model	Cov. %
Prompt	Model	1	10	20	50	100	1000
prompt_vanilla	Llama 8B	$6.09$	$47.96$	$66.70$	$88.56$	$92.44$	$96.51$
prompt_verbalized_all		$6.09$	$47.96$	$66.69$	$89.83$	$93.81$	$97.76$
system_vanilla		$6.09$	$43.42$	$55.31$	$72.00$	$83.02$	$95.23$
system_verbalized_all		$6.09$	$44.11$	$60.26$	$79.16$	$89.30$	$94.98$
prompt_vanilla	GPT-4o	$6.09$	$48.20$	$67.18$	$89.80$	$92.80$	$94.10$

Appendix

Table 19: Compared results with verbalized sampling on Simple Questions generating up to

1,10,20,...1000

answers. Prompt_vanilla is the existing generate-all prompt. System_verbalized_all is the original prompt proposed in verbalized sampling.

Prompt	Model	Cov. %
Prompt	Model	1	5	10	20	50	100	200
prompt_vanilla	Llama 8B	$0.38$	$1.67$	$3.04$	$5.22$	$10.00$	$16.17$	$26.13$
prompt_verbalized_all		$0.37$	$1.63$	$2.98$	$4.99$	$9.24$	$14.46$	$23.76$
system_vanilla		$0.34$	$1.34$	$2.42$	$4.19$	$8.91$	$14.86$	$24.55$
system_verbalized_all		$0.37$	$1.57$	$2.82$	$4.54$	$9.05$	$15.31$	$25.53$
prompt_vanilla	GPT-4o	$0.42$	$1.84$	$3.51$	$6.30$	$11.01$	$16.47$	$24.78$

Appendix

Table 20: Compared results with verbalized sampling on NB-Curated generating up to

1,5,10,...200

responses.Prompt_vanilla is the existing generate-all prompt. System_verbalized_all is the original prompt proposed in verbalized sampling.

Figure 22: Comparing different prompts for generating diverse outputs. The number of samples (x-axis) vs. the diversity coverage scores (y-axis). We compare three prompts on Simple Questions (top) and NB-Curated (bottom) datasets.

Figure 23: The number of unique answers on NB-Curated . The X-axis is the log of the number of answers generated. Y-axis measures the diversity coverage of all the unique answers divided by the max possible score(generate 200 different good answers).

Figure 24: Smaller models generate more unique answers. As number of inferred answers increases, Llama models have better output uniqueness than Qwen, but also have more within-family variance.

Diverse Text Generation

Diverse text generation studies how to produce varied, useful language outputs without sacrificing coherence, accuracy, or task performance.

See all 50 papers

DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models
Mar 24, 2026Donya Jafari, Farzan FarniaLLM Routing Contextual Bandits
Sharif University of Technology · The Chinese University of Hong Kong
Growing a Tail: Increasing Output Diversity in Large Language Models
Nov 5, 2024Michal Shur-Ofry, Bar Horowitz-Amsalem, Adir Rahamim +1LLM Evaluation LLM Prompting
Law Faculty, Hebrew University of Jerusalem; Jerusalem, Israel. · Faculty of Computer Science, Technion – Israel Institute of Technology; Haifa, Israel.
Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs
May 28, 2026Vinay Samuel, Yapei Chang, Mohit IyyerLLM Alignment Direct Preference Optimization
University of Maryland, College Park

Mar 24, 2026J/K move · Enter open · S save

DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models

Donya Jafari, Farzan Farnia

Sharif University of Technology · The Chinese University of Hong Kong

LLM Routing Contextual Bandits Generative Modeling

Open paper AskPDF arXiv

The expansion of generative AI and LLM services underscores the growing need for adaptive mechanisms to select an appropriate available model to respond to a user's prompts. Recent works have proposed offline and online learning formulations to identify the optimal generative AI model for an input prompt, based solely on maximizing prompt-based fidelity evaluation scores, e.g., CLIP-Score in text-to-image generation. However, such fidelity-based selection methods overlook the diversity of generated outputs, and hence, they can fail to address potential diversity shortcomings in the generated responses. In this paper, we introduce the Diversity-Aware Kernelized Upper Confidence Bound (DAK-UCB) method as a contextual bandit algorithm for the online selection of generative models with diversity considerations. The proposed DAK-UCB method incorporates both fidelity and diversity-related metrics into the selection process. We design this framework based on prompt-aware diversity score functions that decompose to a two-sample-based expectation over prompt-output pairs in the previous generation rounds. Specifically, we illustrate the application of our framework using joint kernel distance and kernel entropy measures. Our experimental results demonstrate the effectiveness of DAK-UCB in promoting diversity-aware model selection while maintaining fidelity in the generations for a sequence of prompts. The code is available at https://github.com/Donya-Jafari/DAK-UCB.

Abstract

Diverse Text Generation

Explore similar work

DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models

Growing a Tail: Increasing Output Diversity in Large Language Models

Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs