GICDM: Mitigating Hubness for Reliable Distance-Based Generative Model Evaluation

Nicolas Salvy; Hugues Talbot; Bertrand Thirion

cs.LGFeb 18, 2026

GICDM: Mitigating Hubness for Reliable Distance-Based Generative Model Evaluation

Authors: Nicolas Salvy, Hugues Talbot, Bertrand Thirion

Organizations: Inria, Palaiseau, France · CentraleSupélec, Gif-sur-Yvette, France · Université Paris-Saclay, Gif-sur-Yvette, France · CEA, Gif-sur-Yvette, France

Abstract

Generative model evaluation commonly relies on high-dimensional embedding spaces to compute distances between samples. We show that dataset representations in these spaces are affected by the hubness phenomenon, which distorts nearest-neighbor relationships and biases distance-based metrics. Building on the classical Iterative Contextual Dissimilarity Measure (ICDM), we introduce Generative ICDM (GICDM), a method to correct neighborhood estimation for both real and generated data. We introduce a multi-scale extension to improve empirical behavior. Extensive experiments on synthetic and real benchmarks demonstrate that GICDM resolves hubness-induced failures, restores reliable metric behavior, and improves alignment with human assessment.

Figures & tables

Panel of Figure 1 — Figure 1: High dimension challenges distance-based metrics : The scenario in (a) consists of real samples uniformly drawn from a 60/40 mixture of two hyperspheres, and generated samples uniformly drawn from a mixture with swapped radii and proportions. Standard metrics for this scenario are plotted in (b) as the dimension increases. The real and generated sets are disjoint, so all metrics should score 0; however, none of the standard metrics (top) do in high dimensions due to hubness: on the right side generated points spuriously appear closer to real points than real points are to each other. After applying GICDM (bottom), their scores correctly remain at 0. See Appendix I for individual metric results.

Panel of Figure 2 — Figure 2: Distribution of $k$ -occurrences ( $k=5$ ) for $20000$ samples from a standard Gaussian in (a) $d=4$ and (b) $d=32$ . The y-axis shows the count (log scale) for each $k$ -occurrence value (x-axis). As dimension increases, the distribution skews right, leading to the emergence of hubs (frequent nearest-neighbors, high $O_{k}$ ) and antihubs (never neighbors, $O_{k}=0$ ).

Panel of Figure 3 — Figure 3: $5$ -Occurrence vs Centrality ( $N=20000$ , $d=32$ ): (a) (adapted from Radovanovic et al. (2010) , Fig. 3) Standard Gaussian: hubness aligns with centrality; central points have higher occurrences. (b) Uniform sphere: removing centrality eliminates hubness. So, high dimensionality alone does not cause hubness.

Panel of Figure 4 — Figure 4: Density Gradient : (a) (adapted from Hara et al. (2016) , Fig. 2c): Illustrative Gaussian points (blue density) with $1$ -occurrence counts; arrows indicate nearest-neighbors. Occurrences accumulate in denser regions, as arrows follow the gradient of the density. (b) : $5000$ samples from a uniform square. Color indicates $200$ -occurrence count. Boundaries force neighbors inward, creating density gradients and hubness; both effects intensify with increasing dimension. After applying (c) NICDM and (d) ICDM: Left, arrows show updated neighbors. The maximum $O_{1}$ decreases from $4$ to $3$ (NICDM) and $2$ (ICDM); the number of antihubs drops from $6$ to $5$ and $2$ . Right, the occurrence distribution becomes more uniform after NICDM, and even more so after ICDM.

Figure 5: Crossover dimension

d^{*}

: At

d^{*}

for a standard Gaussian, the median

K

-NN squared distance and the expected squared distance to the center equalize. Points are closer to each other below

d^{*}

and closer to the center above. The plot shows

d^{*}

K

using Proposition 5.3 , for various sample sizes

N

. As

N

increases, points become closer so

d^{*}

increases; as

K

increases,

K

-NN distances increase so

d^{*}

decreases. Using two sufficiently different

K

values ensures at least one avoids the crossover regime.

Figure 6: Hypersphere Test ablation : In the hypersphere scenario from Figure 1 , GICDM components are incrementally added to Precision. Unfiltered GICDM overcorrects generated points. Single-scale filtering reduces this effect, but a spike remains when

5

-NN distances between real points and to generated points are similar. Finally, multi-scale filtering achieves the ideal result.

	Purpose	Bounds	Other
Precision	6/14	3/13	2 /3
Clipped Density	8/14	8/13	2 /3
+ GICDM (ours)	10 /14	11 /13	2 /3
Recall	5/14	2/13	0/3
Clipped Coverage	8/14	9/13	1 /3
+ GICDM (ours)	10 /14	11 /13	1 /3

Table 1: Benchmark success count. Passed tests count (higher is better) on the synthetic benchmark of Räisä et al. (2025) , before and after applying GICDM. Test categories: Purpose (does the score evolution match the intended test objective), Bounds (do metrics yield the exact expected values, e.g., 1 in the ideal case), and Other (additional checks). Results are shown for the best-performing fidelity and coverage metrics: Clipped Density and Clipped Coverage, as well as for Precision and Recall.

Embed.	Metric	CIFAR 10	Image Net	LSUN Bedroom
DINOv2	Precision	$-0.82$	$0.76$	-
	Clipped Density	$0.93$	$0.75$	$0.81$
	+ GICDM	$0.97$	$0.80$	$0.80$
DINOv3	Precision	$0.86$	$0.74$	$0.96$
	Clipped Density	$0.82$	$0.67$	$0.94$
	+ GICDM	$0.95$	$0.82$	$0.97$

Table 2: Correlation with human scores. Pearson correlation between metric scores and human error rates. Results are not significant for FFHQ or other embeddings. GICDM maintains or improves Clipped Density’s (the best-performing metric) correlation with human scores, especially for DINOv3, which exhibits more hubness.

Metric	DiT-XL-2	+ Guidance
Clipped Density	$0.74$	$1.62$ $\nearrow$
+ GICDM	$0.86$	$1.15$ $\nearrow$
Clipped Coverage	$0.64$	$0.83$ $\nearrow$
+ GICDM	$0.79$	$0.55$ $\searrow$

Table 3: Effect of guidance. Metric scores for DINOv2 embeddings of ImageNet-trained DiT-XL-2, with and without classifier-free guidance. Guidance is expected to increase fidelity and decrease coverage. Clipped Density increases both with and without GICDM, but Clipped Coverage decreases only with GICDM.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Hubness Evolution for standard Gaussian data ( $N=20000$ ) as dimensionality increases. Top : Intuitive 2D visualization of a Gaussian using a wrapped chi-squared distribution with $d$ degrees of freedom (lighter colors indicate higher density). As the dimension increases, samples concentrate on a thin spherical shell. Bottom : The y-axes are not shared. Both hubness measures ( $A^{5}$ and $h_{1}^{5}(1\%)$ ) indicate that hubness increases with dimensionality.

Embedding	Dimension	Dataset	$h_{1}^{5}(1\%)$	$A^{5}$
VGG16	4096	Cifar10	10.4	0.21
		FFHQ	9.8	0.23
		ImageNet	12.6	0.20
		LSUN Bedroom	16.6	0.31
Inceptionv3	2048	Cifar10	9.0	0.22
		FFHQ	8.3	0.20

Appendix

Table 4: Hubness measures for various datasets and embedding spaces .

A^{5}

is the proportion of antihubs (

O_{5}(x)=0

), and

h_{1}^{5}(1\%)

quantifies how much more frequently the top 1% of points appear among

k

-nearest-neighbors compared to the average. In the absence of hubness, the

k

-occurrence distribution would be nearly symmetric around its mean (as in Figure 2(a) ), yielding

h_{1}^{5}(1\%)

values slightly above

2

and

A^{5}

values below

0.01

. All reported values are at least twice as large, indicating pronounced hubness.

Method	$h_{1}^{5}(1\%)$		$A^{5}$
Method	Rank	Value	Rank	Value
Original	7.8	8.8	7.9	0.20
Sphere	6.9	6.0	7.1	0.12
$\text{MP}^{\text{Gauss}}$	5.7	4.5	5.9	0.08
LS	5.2	4.1	5.1	0.05
CSLS	3.7	3.2	4.1	0.04

Appendix

Table 5: Hubness reduction comparison. For each method, we report its rank and value for

A^{5}

and

h_{1}^{5}(1\%)

, averaged over five datasets and embeddings from Table 4 . ”Sphere” is projection onto the unit sphere. Lower values are better. ICDM is best in all

17

cases. For methods needing a neighborhood size

K

, we took the

K

\{5,10,20,50\}

with lowest

h_{1}^{5}(1\%)

. With

h_{1}^{5}(1\%)

below

2

and

A^{5}

near

0

, ICDM eliminates hubness effectively.

Figure 8: Empirical convergence of ICDM. Maximum relative deviation of individual neighborhood distance averages

\mu_{i}^{r}

from their overall mean

\bar{\mu}^{r}

across iterations, for 16 combinations of datasets and embeddings, with

K=10

and

K=100

(the two values used when evaluating metrics with

k=5

). Values from the above results at iterations 1, 5, 10, 15 and 20 show the biggest relative difference. After 10 iterations, the maximum deviation is less than

0.17\%

across all scenarios, confirming fast pointwise convergence.

Desiderata

Sanity Check

\alpha

-Precision

Precision Cover

Precision

+ GICDM

Density

+ GICDM

symPrecision

+ GICDM

P-precision

+ GICDM

Clipped Density

+ GICDM

Purpose

Discrete Num. vs. Continuous Num.

Gaussian Mean Difference

Gaussian Mean Difference + Outlier

Gaussian Mean Difference + Pareto

Gaussian Std. Deviation Difference

Hypercube, Varying Sample Size

Appendix

Table 6: Fidelity metrics benchmark with and without GICDM . Tests are grouped by desiderata type: Purpose checks whether the curve shape of the evaluated metric matches the intended test objective, while Bounds evaluates whether the metric yields the expected values (e.g., 1 for perfect generated data). For each metric, T indicates a passed test, and F indicates failure. Adding GICDM improves Purpose results on several tests, such as ”Gaussian Std. Deviation Difference” and ”Hypersphere Surface”, where the real data is fixed and the generated data varies in radius (variance for a Gaussian or radius for a sphere). GICDM improves Bounds results for the normalized metric Clipped Density.

Desiderata

Sanity Check

\beta

-Recall

Recall Cover

Recall

+ GICDM

Coverage

+ GICDM

symRecall

+ GICDM

P-recall

+ GICDM

Clipped Coverage

+ GICDM

Purpose

Discrete Num. vs. Continuous Num.

Gaussian Mean Difference

Gaussian Mean Difference + Outlier

Gaussian Mean Difference + Pareto

Gaussian Std. Deviation Difference

Hypercube, Varying Sample Size

Appendix

Table 7: Coverage metrics benchmark with and without GICDM . This table, analogous to Table 6 , reports results for coverage metrics. T indicates a passed test, while F indicates failure. For coverage metrics, L and H denote that the metric correctly identifies low (L) or high (H) coverage scenarios; either outcome is considered a pass. As with fidelity metrics, GICDM improves Purpose results on several tests, such as ”Gaussian Std. Deviation Difference” and ”Hypersphere Surface.” GICDM also improves Bounds results for the normalized metric Clipped Coverage.

Panel of Figure 9 — Figure 9: Fidelity vs Coverage in DINOv2 embeddings : Clipped Density vs Clipped Coverage (top row) and Clipped Density + GICDM vs Clipped Coverage + GICDM (bottom row) for CIFAR-10, ImageNet, LSUN Bedroom, and FFHQ datasets. Each point represents a generative model. For DiT-XL-2 on ImageNet, in the top row, classifier-free guidance increases both fidelity and coverage, whereas with GICDM (bottom row), it increases fidelity but decreases coverage, as expected (red arrows).

Panel of Figure 10 — Figure 10: Fidelity vs Coverage in DINOv3 embeddings : Same setup as Figure 9 , but using DINOv3 embeddings. GICDM tends to increase the spread of points.

Panel of Figure 11 — Figure 11: Fidelity vs Human error rates in DINOv2 embeddings : Human were tasked with discriminating real from generated images. Top row: Clipped Density vs human error rates. Bottom row: Clipped Density + GICDM vs human error rates. Each plot reports the Pearson correlation coefficient $r$ and its $p$ -value. Results are not significant for FFHQ, consistent with what was reported for DINOv2 previously ( Stein et al., 2023 ) . For other datasets, correlation remains stable or improves slightly with GICDM.

Panel of Figure 12 — Figure 12: Fidelity vs Human error rates in DINOv3 embeddings : Same setup as Figure 11 , but using DINOv3 embeddings. As with DINOv2, results are not significant for FFHQ. For other datasets, correlation increases with GICDM, and the improvement is more pronounced than with DINOv2, since DINOv3 embeddings are more affected by hubness.

Embedding	Metric	Cifar10	ImageNet	LSUN Bedroom
DINOv2	Precision	$-0.82$	$0.76$	–
	+ GICDM	$0.82$	–	$-0.78$
	Density	$-0.91$	$0.68$	–
	+ GICDM	$0.93$	–	$-0.72$
	P-precision	$0.98$	$0.75$	$0.69$
	+ GICDM	$0.82$	–	$-0.78$

Appendix

Table 8: Correlation with human scores. Significant Pearson correlation coefficients between metric scores and human ratings for each dataset and embedding; non-significant values are indicated by –. The FFHQ column is omitted due to the absence of significant correlations. This is consistent with prior findings using DINOv2 embeddings ( Stein et al., 2023 ) . For metrics robust to real outliers (Clipped Density and symPrecision, which equals cPrecision in this context), adding GICDM maintains or improves correlation. For other metrics, adding GICDM generally worsens results, indicating that robustness to real outliers is necessary for GICDM to be effective.

Panel of Figure 13 — Figure 13: Real data tests for fidelity metrics . Fidelity metrics with GICDM are evaluated in several scenarios. (a) Simultaneous mode dropping: synthetic data from all but one CIFAR-10 class is progressively replaced with data from the remaining class. This test is related to ”Dropping + Invention” in Table 6 , but on real data. Similarly, only symPrecision fails. (b) Introducing real and synthetic out-of-distribution samples at equal rates: this tests the stability of the metrics. All succeed.

Panel of Figure 14 — Figure 14: Real data tests for coverage metrics . Coverage metrics with GICDM are evaluated in several scenarios, analogous to those in Figure 13 . (a) Introducing real and synthetic out-of-distribution samples at equal rates: all metrics succeed. (b) Progressively replacing good CIFAR-10 synthetic samples with bad ones. Results are consistent with those reported without GICDM: only Clipped Coverage succeeds ( Salvy et al., 2026 ) .

Panel of Figure 15 — Figure 15: Hubness in DINOv2-embedded LSUN Bedroom and generated sets. The first column shows the most occurring real (top) and generated (bottom, ADM-dropout) images within the 5-nearest neighborhoods of real images. The middle column displays the real images that include them as neighbors. In both cases, there are more than 60 such real images, identifying the images in the first column as hubs and highlighting the asymmetry of neighborhood relationships. After applying hubness reduction (ICDM for real data, GICDM for generated data), the number of real images including these hubs in their neighborhoods drops significantly to 5 and 4, respectively. In both cases, the remaining images listing the hub as a neighbor are more semantically related to it than many of the original ones.

Panel of Figure 16 — Figure 16: Metrics without GICDM on the Hypersphere Test : Each subplot shows how a standard metric behaves as the dimension increases in the hypersphere test scenario described in Figure 1 . Ideally, all metrics should remain at zero for all dimensions, since the real and generated distributions are disjoint.

Panel of Figure 17 — Figure 17: Metrics with GICDM on the Hypersphere Test : Each subplot shows how a metric, after applying GICDM, behaves as the dimension increases in the hypersphere scenario described in Figure 1 . GICDM successfully corrects hubness-related failures, keeping metric values at 0.

	DINOv2 Embeddings
Dataset	Method	K	$h_{1}^{5}(1\%)$	$A^{5}$
CIFAR10	Original	–	6.5	0.14
	Sphere	–	6.2	0.12
	$\text{MP}^{\text{Gauss}}$	–	4.5	0.08
	LS	10	4.0	0.03
	CSLS	20	2.9	0.02

Appendix

Table 9: Hubness reduction comparison : using either DINOv2 (left) or DINOv3 (right). The metrics are

h_{1}^{5}(1\%)

, which measures how much more frequently the top

1\%

of hubs appear in the 5-nearest neighborhoods compared to the average, and

A^{5}

, the proportion of antihubs. In both cases, lower values are better. On ImageNet with DINOv2 embeddings, some methods perform worse than no correction.

	Inceptionv3 Embeddings
Dataset	Method	K	$h_{1}^{5}(1\%)$	$A^{5}$
CIFAR10	Original	–	9.0	0.22
	Sphere	–	6.2	0.12
	$\text{MP}^{\text{Gauss}}$	–	4.5	0.07
	LS	20	3.8	0.04
	CSLS	50	3.3	0.04

Appendix

Table 10: Hubness reduction comparison : using either Inceptionv3 (left) or VGG16 (right) embeddings. The metrics and datasets are the same as in Table 9 . The initial hubness levels are generally higher than with DINO embeddings, but ICDM still consistently achieves the lowest hubness scores across all datasets and embeddings.

	CLAP Embeddings
Dataset	Method	K	$h_{1}^{5}(1\%)$	$A^{5}$
FMA	Original	–	4.9	0.09
	Sphere	–	4.6	0.08
	$\text{MP}^{\text{Gauss}}$	–	3.9	0.06
	LS	20	3.3	0.02
	CSLS	20	2.6	0.01

Appendix

Table 11: Hubness reduction comparison : for CLAP embeddings of 7-second music samples from the Free Music Archive (FMA) dataset. The metrics are the same as in Tables 9 and 10 . Even in this different modality, hubness is present, and ICDM achieves the best performance.

Generative AI Evaluation

Generative AI evaluation measures the quality, reliability, diversity, and real-world usefulness of generated outputs and AI-assisted workflows.

See all 309 papers

May 7, 2026

MIND: Monge Inception Distance for Generative Models Evaluation

We propose the Monge Inception Distance (MIND), a metric for evaluating generative models that addresses key limitations of the widely adopted Fréchet Inception Distance (FID). The MIND metric leverages the sliced Wasserstein distance to compare distributions by averaging one-dimensional optimal transport distances, efficiently computed via sorting. This approach circumvents the estimation of high-dimensional means and covariance matrices, which underlie FID's poor sample complexity and vulnerability to adversarial attacks. We empirically demonstrate three primary advantages: (i) it is more sample-efficient by one order of magnitude, (ii) it is faster to compute by two orders of magnitude, (iii) it is more robust to adversarial attacks such as moment-matching. We show that MIND with 5k samples can replace the evaluation performance of FID with 50k samples, providing high correlation with this standard benchmark and superior discriminative performance. We further demonstrate that even smaller sample sizes (e.g., 1k or 2k) remain highly informative for rapid model iteration.

Quentin Berthet, Yu-Han Wu, Clement Crepy +3

Google DeepMind · LPSM, Sorbonne Université

Image Generation Evaluation Sliced Wasserstein Distance Generative AI Evaluation

Ask PDF

May 28, 2026

Rethinking FID Through the Geometry of the Reference Dataset

Fréchet Inception Distance (FID) is widely used to evaluate image generators, yet lower FID does not always correspond to better sample quality. We show that this mismatch depends in part on the geometry of the reference dataset. In a controlled study across six datasets, distributional density and effective rank significantly explain how FID changes as sample quality improves. Concentrated datasets tend to yield more favorable FID trends, whereas more dispersed datasets can make FID worsen despite better samples. Attribution to precision and recall and ablations with alternative feature spaces and distances support the same conclusion. These results suggest that distributional metrics should be interpreted together with the geometry of the reference dataset for more reliable benchmarking.

Yunghee Lee, Byeonghyun Pak

Image Generation Evaluation

Ask PDF

Apr 7, 2026

A Theoretical Framework for Statistical Evaluability of Generative Models

Statistical evaluation aims to estimate the generalization performance of a model using held-out i.i.d. test data sampled from the ground-truth distribution. In supervised learning settings such as classification, performance metrics such as error rate are well-defined, and test error reliably approximates population error given sufficiently large datasets. In contrast, evaluation is more challenging for generative models due to their open-ended nature: it is unclear which metrics are appropriate and whether such metrics can be reliably evaluated from finite samples. In this work, we introduce a theoretical framework for evaluating generative models and establish evaluability results for commonly used metrics. We study two categories of metrics: test-based metrics, including integral probability metrics (IPMs), and Rényi divergences. We show that IPMs with respect to any bounded test class can be evaluated from finite samples up to multiplicative and additive approximation errors. Moreover, when the test class has finite fat-shattering dimension, IPMs can be evaluated with arbitrary precision. In contrast, Rényi and KL divergences are not evaluable from finite samples, as their values can be critically determined by rare events. We also analyze the potential and limitations of perplexity as an evaluation method.

Shashaank Aiyer, Yishay Mansour, Shay Moran +1

University of Maryland · Tel Aviv University and Google Research · Technion and Google Research

Generative AI Evaluation Statistical Learning Theory Integral Probability Metrics

Ask PDF

Abstract

Generative AI Evaluation

Explore similar work

MIND: Monge Inception Distance for Generative Models Evaluation

Rethinking FID Through the Geometry of the Reference Dataset

A Theoretical Framework for Statistical Evaluability of Generative Models