How Geometry Shapes Generative Model Evaluation on Riemannian Manifolds

Misha Oberski; Valentijn Oldenburg; Xanthos Kekkos; Soham Chaudhuri; Emma Oosterhuis; Floris de Kam; Sagi Schwartz; Alison Pouplin

cs.LGOct 24, 2026

How Geometry Shapes Generative Model Evaluation on Riemannian Manifolds

Authors: Misha Oberski, Valentijn Oldenburg, Xanthos Kekkos, Soham Chaudhuri, Emma Oosterhuis, Floris de Kam, Sagi Schwartz, Alison Pouplin,

Organizations: University of Amsterdam · Independent Researcher · AMLab, University of Amsterdam

Abstract

On Riemannian manifolds, generative-model evaluation is geometry-dependent since curvature and dimension jointly determine distances and volume growth. Our framework systematically compares geodesic, ambient, and tangent-space versions of Wasserstein and MMD across different geometries. Using controlled-noise perturbations and Riemannian Flow Matching samples, we study metric agreement, noise sensitivity, and their relation to geometry-induced volume scaling. We find that no metric is uniformly reliable: evaluation distance can affect both measured discrepancies and noise sensitivity, varying across curvature and dimension. Our analysis leads to practical recommendations for reporting evaluation distances, comparing metric types, and calibrating metrics to the geometry of the samples.

Figures & tables

Figure 1: Metric discrepancies depend on evaluation geometry. Cells show $\log_{2}(D_{\mathrm{geodesic}}/D_{\mathrm{other}})$ over $(\kappa,d)$ for GtG, comparing geodesic evaluation with ambient (top) and tangent-space (bottom) distances. Positive values mean geodesic eval. yields a larger discrepancy. $W_{1}$ is mostly larger under geodesic, whereas $\mathrm{MMD}$ decreases in high $d$ .

Figure 2: Evaluation distance changes metric trends. Curves show

\log_{2}(D_{\mathrm{geodesic}}/D_{\mathrm{other}})

across dimensions for four curvatures.

W_{1}

and

\mathrm{MMD}^{2}

respond differently to the same change from intrinsic to non-intrinsic evaluation.

Table 3

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: The three evaluation distances, illustrated for spherical and hyperbolic space. Samples are compared intrinsically by geodesic distance, extrinsically by ambient Euclidean distance, or locally after log-mapping to the tangent space at $o$ . The same pair of points can thus produce different error.

Figure 6: Illustration of the evaluation metrics. The same predicted samples can be assessed by transport cost ( $W_{1}$ ), kernel similarity ( $\mathrm{MMD}^{2}$ ), and dispersion via Fréchet variance.

Figure 7:

J_{\kappa}

versus

r

for different

\kappa

Figure 8: Overview of the target distributions. Columns show GtG, SGR, and MoG targets; rows show Euclidean, spherical, and hyperbolic geometries. The same tangent-space distributions lead to different sample geometries after mapping to manifolds with different curvature.

Parameter	Value
Curvatures ( $\kappa$ )	-10, -5, -3, -2, -1, 0, 1, 2, 3, 5, 10
Dimensions ( $d$ )	2, 3, 4, 8, 16, 32, 64, 256, 512
Seeds	$34,\ 42,\ 50$

Appendix

Table 3: General hyperparameters.

Parameter	Value
Base standard deviation ( $\sigma_{0}$ )	0.3
Target standard deviation ( $\sigma_{1}$ )	0.3
Separation distance ( $\delta$ )	0.5

Appendix

Table 4: Additional hyperparameters for the GtG experiments.

Parameter	Value
Radius ( $r_{\mathrm{SGR}}$ )	$0.35$ , $0.60$ , $0.85$
Radial standard deviation ( $\sigma_{\mathrm{SGR}}$ )	0.05

Appendix

Table 5: Additional hyperparameters for the SGR experiments.

Parameter	Value
Number of target modes ( $n$ )	2, 3, 5
Base standard deviation ( $\sigma_{0}$ )	0.3
Target standard deviation ( $\sigma_{1}$ )	0.3
Balanced weights ( $w_{n}$ )	$[1/n,\dots,1/n]$
Unbalanced weights ( $w_{n}$ )	$w_{2}=\frac{1}{3}[1,2],w_{3}=\frac{1}{4}[1,1,2],w_{5}=\frac{1}{8}[1,2,3,1,1]$

Appendix

Table 6: Additional hyperparameters for the MoG experiments.

Experiment	Hidden layers	Batch size	Learning rate	$d$	$\|\kappa\|$	Seeds
GtG, SGR, MoG	8	512	$1\times 10^{-4}$	2, 3, 4, 8, 16, 32, 64, 256, 512	0, 1, 2, 3, 5, 10	34, 42, 50

Appendix

Table 7: Model hyperparameters and sweep configurations.

Panel of Figure 9 — Figure 9: Metric-type agreement is target-dependent. Spearman correlations are computed across all $(\kappa,d,\mathrm{seed})$ -configurations for each target. For Fréchet variance we use $|\log\rho_{F}|$ so lower values indicate better quality. Distance representations are strongly correlated within a metric type, but agreement across $W_{1}$ , $\mathrm{MMD}^{2}$ , and Fréchet variance is weaker and depends on target structure.

Curvature	$d=3$	$16$	$64$	$256$	$512$
Hyperbolic	1.00	0.99	0.77	0.18	0.11
Spherical	1.00	1.01	0.95	1.01	1.01

Appendix

Table 8: Ambient Fréchet variance underestimates high-

d

hyperbolic dispersion. Entries show

\rho_{F}^{d_{\mathrm{ambient}}}/\rho_{F}^{d_{\mathrm{geodesic}}}

for RFM SGR samples. Values below

1

mean ambient evaluation reports a more small dispersion error than geodesic evaluation. The gap grows with dimension in hyperbolic space but stays near

1

on spheres.

Figure 10: Dispersion errors depend on target structure and representation. Ratio of predicted-to-target Fréchet variance across signed curvature

\kappa

and dimension

d

, using geodesic, tangent-space, and ambient distances. GtG remains close to

1

across the sweep, while SGR develops large high-

d

overdispersion under geodesic and tangent-space evaluation. Ambient hyperbolic evaluation compresses this magnitude, consistent with Table 8 .

$d$	$\rho_{W_{1}}$	$\rho_{\mathrm{MMD}}$
2	1.0000	0.9903
3	1.0000	0.9894
4	1.0000	0.9940
8	1.0000	0.9949
16	0.9997	0.9940
64	0.9959	0.9760

Appendix

Table 9: Geodesic

W_{1}

remains monotone, while geodesic

\mathrm{MMD}^{2}

weakens at high

d

. Mean Spearman correlations between noise level

\sigma

and metric value on GtG, averaged over curvatures and seeds. Values near

1

indicate that the metric increases consistently with controlled noise. Spherical configurations at

d\in\{256,512\}

are excluded (see Appendix D.3 ).

Panel of Figure 11 — Figure 11: Noise responses depend on dimension and evaluation distance. At $d=2$ and $d=64$ , both $W_{1}$ and $\mathrm{MMD}^{2}$ generally increase with controlled noise. However, the responses differ per evaluation distance, especially in hyperbolic space.

Figure 12: Noise-detection reliability varies by metric and target structure. SNR is shown as a function of dimension for GtG, SGR, and MoG, using geodesic, tangent-space, and ambient distances. This supports the main-text claim that reliability is not a property of the metric alone.

Figure 13: SGR radius alone does not determine geodesic SNR. Mean geodesic SNR is shown against the target radius $r_{p_{1}}$ in spherical and hyperbolic geometries, coloured by dimension. SNR remains high across radii, while dimension explains much of the variation.

Figure 14: Volume scaling explains some, but not all, SNR gaps. Each point is a

(\kappa,d)

-configuration on GtG. The x-axis shows target volume scaling,

\log\mu_{J_{\kappa}}(p_{1})

, and the y-axis shows the geodesic SNR advantage over the compared distance. Positive values mean geodesic evaluation detects controlled noise more reliably. Volume scaling correlates with the geodesic advantage over ambient

W_{1}

and over both MMD alternatives, with the relationship weaker for tangent space

W_{1}

Target	# runs	finite (%)	failed (%)	median	max	median	max
		Sample validity		Relative tangency violation		$\\|v_{t}(x)\\|_{g_{\kappa}}$ (p95)
GtG	330	100.0	0.0	$4.5\times 10^{-6}$	$4.6\times 10^{-5}$	0.45	20.40
SGR	1350	100.0	0.0	$2.3\times 10^{-6}$	$5.3\times 10^{-4}$	0.71	5.20
MoG	1980	100.0	0.0	$1.0\times 10^{-5}$	$2.9\times 10^{-4}$	0.45	12.76
All	3660	100.0	0.0	$7.1\times 10^{-6}$	$5.3\times 10^{-4}$	0.52	20.40

Appendix

Table 10: Numerical stability for RFM sweeps. All completed runs produce finite samples, and relative tangency violations remain very small. This supports the interpretation that the metric differences in the text are not caused by invalid samples or off-manifold vector fields. The p95 vector-field norm is included as a secondary scale check.

Generative AI Evaluation

Generative AI evaluation measures the quality, reliability, diversity, and real-world usefulness of generated outputs and AI-assisted workflows.

See all 311 papers

MIND: Monge Inception Distance for Generative Models Evaluation
May 7, 2026Quentin Berthet, Yu-Han Wu, Clement Crepy +3Image Generation Evaluation Sliced Wasserstein Distance
Google DeepMind · LPSM, Sorbonne Université
When Riemann flows with Wasserstein: Generative Modeling of Probability Distributions on Manifolds
Sep 22, 2026Doron Haviv, Edward De Brouwer, Rishabh Anand +4Riemannian Flow Matching Generative Modeling
Genentech Inc., South San Francisco, CA. · Yale University, New Haven, CT.
A Likely Geometry of Generative Models
Oct 30, 2025Frederik Möbius Rygaard, Shen Zhu, Yinzhu Jin +2Generative Modeling
DTU Compute, Technical University of Denmark, Kongens Lyngby, Denmark · Department of Computer Science, University of Virginia, Charlottesville (VA), USA · Department of Electrical and Computer Engineering, University of Virginia, Charlottesville (VA), USA

May 7, 2026J/K move · Enter open · S save

MIND: Monge Inception Distance for Generative Models Evaluation

Quentin Berthet, Yu-Han Wu, Clement Crepy, Romuald Elie+2

Google DeepMind · LPSM, Sorbonne Université

Image Generation Evaluation Sliced Wasserstein Distance Generative AI Evaluation

Open paper AskPDF arXiv

We propose the Monge Inception Distance (MIND), a metric for evaluating generative models that addresses key limitations of the widely adopted Fréchet Inception Distance (FID). The MIND metric leverages the sliced Wasserstein distance to compare distributions by averaging one-dimensional optimal transport distances, efficiently computed via sorting. This approach circumvents the estimation of high-dimensional means and covariance matrices, which underlie FID's poor sample complexity and vulnerability to adversarial attacks. We empirically demonstrate three primary advantages: (i) it is more sample-efficient by one order of magnitude, (ii) it is faster to compute by two orders of magnitude, (iii) it is more robust to adversarial attacks such as moment-matching. We show that MIND with 5k samples can replace the evaluation performance of FID with 50k samples, providing high correlation with this standard benchmark and superior discriminative performance. We further demonstrate that even smaller sample sizes (e.g., 1k or 2k) remain highly informative for rapid model iteration.

Abstract

Generative AI Evaluation

Explore similar work

MIND: Monge Inception Distance for Generative Models Evaluation

When Riemann flows with Wasserstein: Generative Modeling of Probability Distributions on Manifolds

A Likely Geometry of Generative Models