Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

Mohammad Jalali; Azim Ospanov; Amin Gohari; Farzan Farnia

cs.LGNov 5, 2024

Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

Authors: Mohammad Jalali, Azim Ospanov, Amin Gohari, Farzan Farnia

Organizations: Department of Computer Science and Engineering, The Chinese University of Hong Kong · Department of Information Engineering, The Chinese University of Hong Kong

Abstract

Generative models guided by text prompts are widely evaluated for fidelity and prompt alignment, yet their ability to produce diverse outputs remains underexplored. Existing diversity metrics such as Vendi and RKE, which are based on the von Neumann and Rényi entropies of kernel matrices, were developed for unconditional models and cannot distinguish prompt-induced from model-induced variability. We address this gap by introducing Conditional-Vendi and Conditional-RKE, diversity measures derived from the conditional entropy of positive semidefinite matrices. These scores isolate model-induced diversity in prompt-guided generation, with Conditional-RKE enjoying an $O(1/\sqrt{n})$ convergence rate. For Conditional-Vendi, we introduce a truncated-spectrum approximation that yields scalable and consistent estimates. Experiments on text-to-image, image-captioning, and LLM tasks show that the conditional scores recover ground-truth diversity orderings and can also guide diffusion models toward more diverse samples. The codebase is available at https://github.com/mjalali/conditional-vendi.

Figures & tables

Figure 1: Illustration of prompt-induced diversity (left), where variation comes from different prompts, and model-induced diversity (right), where variation arises from the generator itself. Unconditional image diversity scores, Vendi and RKE, assign higher diversity to the right side, overlooking prompt effects. Our proposed Conditional-RKE and Conditional-Vendi account for prompts and assign higher diversity to the left side.

Figure 2: Statistical convergence of Conditional-Vendi and RKE scores with different sample sizes on the MS-COCO validation dataset with a finite-dimensional (normalized) linear kernel and infinite-dimensional Gaussian (RBF) kernel.

Figure 3: Statistical convergence of Conditional-Vendi scores with different sample sizes on data generated by SDXL using MS-COCO validation set prompts with finite-dimension cosine similarity and infinite-dimension Gaussian kernel. DINOv2 and CLIP embeddings are used for image and text modalities, respectively.

Figure 4: Conditional and Information Vendi and RKE score comparison across text-to-image models. We clustered MS-COCO prompts into k groups and generated images for each cluster center. Within each cluster, we paired prompts with identical images. The results show increasing diversity and stronger correlation as the number of clusters grows, indicating that clusters become more relevant and diverse with finer partitioning.

Panel of Figure 5 — Figure 5: Evaluated Conditional-Vendi and Vendi scores on animal samples generated by SDXL. (Left Plot) We do not specify the animal types in the prompt (Right Plot) we specify the animal types in the prompt.

Figure 6: Quantifying image diversity of PixArt- $\alpha$ –generated outputs for the top-2 prompt clusters (from 10,000 GPT-4o–generated prompts).

Guidance Method	CLIPScore $\uparrow$	KD $\times 10^{2}\downarrow$	Cond-Vendi ${}_{\text{DINOv2 }}$ $\uparrow$	Vendi ${}_{\text{DINOv2 }}$ $\uparrow$	In-batch Sim. $\times 10^{2}\downarrow$
Vendi ${}_{\text{Latent }}$	30.41	35.40	29.85	309.45	80.06
Conditional-Vendi ${}_{\text{Latent }}$	30.47	29.14	32.45	325.20	78.02

Table 1: Quantitative comparison of guidance methods on Stable Diffusion XL

Figure 7: Qualitative comparison of Conditional-Vendi score guidance vs. Vendi score guidance using SD-XL.

Figure 8: Measuring Conditional-Vendi and Information-Vendi for text-to-video models

Method	$T=0.4$	$T=0.7$	$T=1.0$	$T=1.3$
Conditional-Vendi	45.38	46.73	48.26	49.60
Conditional-RKE	41.45	44.67	46.29	48.45

Table 2: Conditional Vendi and RKE Scores evaluated for Llama 2 with different temperature parameters.

Figure 9: Conditional-Vendi and Information-Vendi of image-captioning models for 3 image types

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Qualitative comparison of Conditional-Vendi score guidance vs. Vendi score guidance using SD-XL.

Figure 11: Qualitative comparison of Conditional-Vendi score guidance vs. Vendi score guidance using PixArt- $\Sigma$ .

Guidance Method	CLIPScore $\uparrow$	KD $\times 10^{2}\downarrow$	Cond-Vendi ${}_{\text{DINOv2 }}$ $\uparrow$	Vendi ${}_{\text{DINOv2 }}$ $\uparrow$	In-batch Sim. $\times 10^{2}\downarrow$
Vendi ${}_{\text{CLIP }}$	29.63	38.14	26.28	261.73	83.26
Vendi ${}_{\text{Latent }}$	30.39	36.20	28.95	298.15	81.50
Conditional-Vendi ${}_{\text{Latent }}$	30.44	29.80	31.50	312.80	79.45

Appendix

Table 3: Quantitative comparison of guidance methods on PixArt

Figure 12: Comparing Conditional-Vendi and Information-Vendi on 2-D Gaussian Distribution. We used 1000 pair of points and used a Gaussian Kernel with bandwidth $\sigma=0.6$ .

Figure 13: Substituting images generated from models trained on MS- dataset.

Figure 14: Quantifying image diversity for different clusters of text prompts. Images are generated using the Stable Diffusion XL model.

Figure 15: Quantifying image diversity for different clusters of text prompts. Images are generated using the PixArt- $\alpha$ model.

Figure 16: Quantifying image diversity for different clusters of text prompts. Images are generated using the Flux model.

Panel of Figure 17 — Figure 17: Evaluated Conditional-Vendi, Vendi, Conditional-RKE, and RKE scores on animal samples generated by PixArt $\Sigma$ . (Left Plot) We do not specify the animal types in the prompt (Right Plot) we specify the animal types in the prompt.

Panel of Figure 18 — Figure 18: Comparing Conditional-Vendi with Vendi on different fruit types generated by Stable Diffusion-XL.

Panel of Figure 19 — Figure 19: Comparing Conditional-Vendi with Vendi on different fruit types generated by PixArt- $\Sigma$ .

Figure 20: Statistical convergence of Conditional-Vendi scores with different sample sizes on data generated by Stable Diffusion-XL and Kandinsky using MS-COCO validation set prompts with finite-dimension cosine similarity and infinite-dimension Gaussian kernel. DINOv2 and CLIP embeddings are used for image and text modalities, respectively.

Figure 21: Conditional-Vendi and Information-Vendi of image-captioning models for 3 image types

Figure 22: Llama 2 Samples generated with different temperature parameters.

Method	$T=0.4$	$T=0.7$	$T=1.0$	$T=1.3$
Conditional-Vendi	40.82	42.82	44.03	48.23
Conditional-RKE	38.42	41.82	43.16	45.93

Appendix

Table 4: Conditional Vendi and RKE Scores evaluated for Gemma 3 with different temperature parameters.

Method	$T=0.4$	$T=0.7$	$T=1.0$	$T=1.3$
Conditional-Vendi	41.02	45.93	49.93	51.60
Conditional-RKE	39.43	41.74	47.27	49.82

Appendix

Table 5: Conditional Vendi and RKE Scores evaluated for Phi 4 Mini with different temperature parameters.

Figure 23: Measuring Conditional-Vendi and Information-Vendi for text-to-video models

Panel of Figure 24 — Figure 24: Conditional and Information Vendi and RKE score comparison across text-to-image models. We clustered MS-COCO prompts into k groups and generated images for each cluster center. Within each cluster, we paired prompts with identical images. The results show increasing diversity and stronger correlation as the number of clusters grows, indicating that clusters become more relevant and diverse with finer partitioning.

Figure 25: 3 clusters of MS-COCO generated samples

Generative AI Evaluation

Generative AI evaluation measures the quality, reliability, diversity, and real-world usefulness of generated outputs and AI-assisted workflows.

See all 300 papers

DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models
Mar 24, 2026Donya Jafari, Farzan FarniaLLM Routing Contextual Bandits
Sharif University of Technology · The Chinese University of Hong Kong
Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error
Feb 16, 2026Farzan Farnia, Mohammad Jalali, Azim OspanovGenerative Modeling Generative AI Evaluation
Department of Computer Science and Engineering, The Chinese University of Hong Kong.
Gacha Decoding: Eliciting Diverse Generations Through Instruction Following
Oct 1, 2026Scott Geng, Yufei Zhang, Joseph Lee +3Language Model Decoding Diverse Text Generation
University of Washington · Meta Superintelligence Labs

Mar 24, 2026J/K move · Enter open · S save

DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models

Donya Jafari, Farzan Farnia

Sharif University of Technology · The Chinese University of Hong Kong

LLM Routing Contextual Bandits Generative Modeling

Open paper AskPDF arXiv

The expansion of generative AI and LLM services underscores the growing need for adaptive mechanisms to select an appropriate available model to respond to a user's prompts. Recent works have proposed offline and online learning formulations to identify the optimal generative AI model for an input prompt, based solely on maximizing prompt-based fidelity evaluation scores, e.g., CLIP-Score in text-to-image generation. However, such fidelity-based selection methods overlook the diversity of generated outputs, and hence, they can fail to address potential diversity shortcomings in the generated responses. In this paper, we introduce the Diversity-Aware Kernelized Upper Confidence Bound (DAK-UCB) method as a contextual bandit algorithm for the online selection of generative models with diversity considerations. The proposed DAK-UCB method incorporates both fidelity and diversity-related metrics into the selection process. We design this framework based on prompt-aware diversity score functions that decompose to a two-sample-based expectation over prompt-output pairs in the previous generation rounds. Specifically, we illustrate the application of our framework using joint kernel distance and kernel entropy measures. Our experimental results demonstrate the effectiveness of DAK-UCB in promoting diversity-aware model selection while maintaining fidelity in the generations for a sequence of prompts. The code is available at https://github.com/Donya-Jafari/DAK-UCB.

Abstract

Generative AI Evaluation

Explore similar work

DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models

Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error

Gacha Decoding: Eliciting Diverse Generations Through Instruction Following