cs.CVOct 31, 2026

Regularized Wasserstein Distance for Discrete Generative Modeling and Clustering Analysis

Authors: Senmao Ye, Fei Liu, Dawei Zhang, Yuqian Zheng, Jun Zhang, Zhonglong zheng

Organizations: Zhejiang Normal University, Jinhua, 321004, Zhejiang, China · South China University of Technology, Guangzhou, 510006, Guangdong, China · Jinhua University of Vocational Technology, Jinhua, 321007, Zhejiang, China

Abstract

In this paper, we formulate clustering analysis as discrete generative modeling. Specifically, when the generator is restricted to a finite candidate space, the generated samples naturally emerge as the most representative prototypes of the underlying data distribution. To model such a discrete candidate space, we employ a Discrete Distribution Network (DDN). However, DDN suffers from unstable training due to its hard nearest-neighbor selection. To stabilize training and encourage geometric consistency, we replace DDN's l2 loss between images with regularized Wasserstein distance between the empirical distributions of real and generated images. The resulting framework, termed Discrete Generative Clustering (DGC), offers four notable advantages: 1) the learned cluster centers are directly interpretable, as they are realistic synthetic samples; 2) the inference cost scales logarithmically with the number of clusters, i.e., O(log K), making the method scalable to extremely large K; 3) the regularized Wasserstein distance provides a smooth and well-behaved optimization objective that facilitates stable convergence, whereas DDN's l2 loss tends to converge to poor local optima; and 4) DGC replaces DDN's nearest-neighbor selection with random-neighbor selection, which avoids the density shift problem during multi-step forward. Extensive experiments on several benchmarks demonstrate the effectiveness and desirable properties of DGC. The code is publicly available at GitHub.

Figures & tables

Explore similar work

May 12, 2026

One-Step Generative Modeling via Wasserstein Gradient Flows

Diffusion models and flow-based methods have shown impressive generative capability, especially for images, but their sampling is expensive because it requires many iterative updates. We introduce W-Flow, a framework for training a generator that transforms samples from a simple reference distribution into samples from a target data distribution in a single step. This is achieved in two steps: we first define an evolution from the reference distribution to the target distribution through a Wasserstein gradient flow that minimizes an energy functional; second, we train a static neural generator to compress this evolution into one-step generation. We instantiate the energy functional with the Sinkhorn divergence, which yields an efficient optimal-transport-based update rule that captures global distributional discrepancy and improves coverage of the target distribution. We further prove that the finite-sample training dynamics converge to the continuous-time distributional dynamics under suitable assumptions. Empirically, W-Flow sets a new state of the art for one-step ImageNet 256×\times256 generation, achieving 1.29 FID, with improved mode coverage and domain transfer. Compared to multi-step diffusion models with similar FID scores, our method yields approximately 100×\times faster sampling. These results show that Wasserstein gradient flows provide a principled and effective foundation for fast and high-fidelity generative modeling.
Oct 1, 2026

Discrete Wasserstein Flows for One-Step Generative Modeling

We introduce a new framework for one-step generative modelling on finite state spaces. To extend drifting beyond continuous domains, we use discrete Wasserstein geometry to define a target-relative KL gradient flow over the transitions of a reversible Markov kernel. We realize this probability flow at the particle level through Markov jumps and amortize the resulting transport updates into a latent-conditioned generator, so that the iterative dynamics are required only during training while inference remains one-step. In a controlled setting where the underlying distributions and transport dynamics can be computed exactly, we verify KL dissipation, consistency between the particle dynamics and the probability flow, and the predicted numerical scaling. We further show that a finite-capacity neural generator can track these exact transport targets while retaining one-step generation. These results validate the basic construction and provide a foundation for scaling Discrete Drifting to structured discrete data.
Feb 3, 2026

Distance Marching for Generative Modeling

Modern generative models based on diffusion or flow matching transport noise toward the data manifold, without representing the data manifold explicitly. We propose Distance Marching that explores an alternative design by explicitly modeling a distance field of the data manifold. A distance field offers a geometric view of generation as a descent process toward the data manifold using only the current position. The negative gradient of the distance field provides the direction, and the distance value can optionally determine the step size. However, sparse training data in high-dimensional space make it difficult to identify the closest points on the data manifold needed to train the distance field. To address this challenge, we design new loss functions that emphasize denoising targets from closer data points. Our analysis shows that, without time input, emphasizing closer targets reduces denoising ambiguity and the bias toward the data mean. This distance-field perspective naturally motivates two principled deterministic samplers, showing fast convergence in inference. As a result, our model is the only one without time input that remains competitive with time-conditioned models across datasets and architectures. Moreover, our distance prediction is also helpful for early stopping during sampling and for out-of-distribution detection. We hope distance-field modeling can open a promising direction for exploring generative modeling from a geometric perspective.