Overview

Optimal transport studies how probability mass can be moved between distributions at minimum cost, giving a geometric way to compare data even when samples do not correspond one-to-one. Wasserstein distances and transport maps describe this geometry, while Gromov–Wasserstein methods compare relational structure when data live in different spaces. Practical approaches include entropic regularization with Sinkhorn algorithms, sliced and low-rank approximations, and neural transport maps used in generative modeling, domain adaptation, and trajectory inference. A central challenge is balancing computational and statistical scalability with faithful transport, especially when choosing regularization or relaxing exact mass constraints.

Latest papers

315 papers24 papers in the last four weeks, down 61% on the four weeks before. 0.2% of all new papers.

Oct 29, 2025

Orlicz-Sobolev with Musielak: An Efficient Regularization Approach for Graph-based IPM

We study the Sobolev IPM problem for probability measures supported on a graph metric space, where critic function is constrained to lie within the unit ball defined by Sobolev norm. Sobolev IPM is intrinsically coupled with LpL^p geometric structure within its definition, limiting its ability to incorporate other prior geometry beyond the LpL^p paradigm. Conversely, the classic optimal transport is flexible, easy to adapt to various geometric structures by simply changing its ground cost. An important example is Orlicz-Wasserstein (OW) which utilizes Orlicz geometric structure to generalize the LpL^p within the standard pp-order Wasserstein, and remarkably play a vital role to advance machine learning methodologies. Inspired by recent advantages of OW, in this work, we leverage a specific class of convex functions for Orlicz geometry to mitigate such limitation for Sobolev IPM, and propose the generalized Sobolev IPM (GSI). Our GSI approach encompasses Sobolev IPM as a special case while accommodating diverse geometric priors beyond LpL^p. It however brings up significant computational hurdles that compound those already notoriously inherent in Sobolev IPM. To address these challenges, we theoretically establish a novel connection between Orlicz-Sobolev norm and Musielak norm which facilitates a novel efficient regularization for GSI. By further exploiting the underlying graph structure, we show that the regularized GSI reduces to a simple univariate optimization problem, achieving notably computational efficiency, enabling its usage in practical applications. We empirically illustrate that the regularized GSI is several-order faster than the popular OW in computation, and its performances compare favorably to other transport baselines for comparing graph-based measures in document classification and topological data analysis.
Oct 28, 2025

DistDF: Time-Series Forecasting Needs Joint-Distribution Wasserstein Alignment

Training time-series forecasting models requires aligning the conditional distribution of model forecasts with that of the label sequence. The standard direct forecast (DF) approach resorts to minimizing the conditional negative log-likelihood, typically estimated by the mean squared error. However, this estimation proves biased when the label sequence exhibits autocorrelation. In this paper, we propose DistDF, which achieves alignment by minimizing a distributional discrepancy between the conditional distributions of forecast and label sequences. Since such conditional discrepancies are difficult to estimate from finite time-series observations, we introduce a joint-distribution Wasserstein discrepancy for time-series forecasting, which provably upper bounds the conditional discrepancy of interest. The proposed discrepancy is tractable, differentiable, and readily compatible with gradient-based optimization. Extensive experiments show that DistDF improves diverse forecasting models and achieves leading performance. Code is available at https://anonymous.4open.science/r/DistDF-F66B.
Oct 16, 2025

AlignFlow: Improving Flow-based Generative Models with Semi-Discrete Optimal Transport

Flow-based Generative Models (FGMs) effectively transform noise into complex data distributions. Incorporating Optimal Transport (OT) to couple noise and data during FGM training has been shown to improve the straightness of flow trajectories, enabling more effective inference. However, existing OT-based methods estimate the OT plan using (mini-)batches of sampled noise and data points, which limits their scalability to large and high-dimensional datasets in FGMs. This paper introduces AlignFlow, a novel approach that leverages Semi-Discrete Optimal Transport (SDOT) to enhance the training of FGMs by establishing an explicit, optimal alignment between noise distribution and data points with guaranteed convergence. SDOT computes a transport map by partitioning the noise space into Laguerre cells, each mapped to a corresponding data point. During FGM training, i.i.d. noise samples are paired with data points via the SDOT map. AlignFlow scales well to large datasets and model architectures with negligible computational overhead. Experimental results show that AlignFlow improves the performance of a wide range of state-of-the-art FGM algorithms and can be integrated as a plug-and-play component. Code is available at: https://github.com/konglk1203/AlignFlow.
Sep 26, 2025

OT-MeanFlow3D: Bridging Optimal Transport and Meanflow for Efficient 3D Point Cloud Generation

Flow-matching models have recently emerged as a powerful framework for generative modeling, including 3D point cloud synthesis. However, their deployment is limited by the need for multiple sequential sampling steps at inference time. MeanFlow enables single-step generation by learning an average velocity field, but its quality depends heavily on the geometry of the training pairs. In this work, we propose an Optimal Transport-enhanced MeanFlow framework (OT-MF3D) for efficient and accurate 3D point cloud generation and completion. Beyond incorporating OT into MeanFlow, our method introduces a latent-conditioned training algorithm that decouples shape-level sampling from point-level matching. This enables exact point-level OT within each conditioned shape while performing shape-level transport in a compact latent space, avoiding the expensive nested OT computation in point-cloud space. Experiments on standard benchmarks show improved generation and completion quality over fast baselines, with substantially reduced training and inference costs compared with diffusion and other flow-based models.
Sep 2, 2025

Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport

The Expectation-Maximisation (EM) algorithm is a central tool in statistics and machine learning, widely used for latent-variable models such as Gaussian Mixture Models (GMMs). Despite its ubiquity, EM is typically treated as a non-differentiable black box, preventing its integration into modern learning pipelines where end-to-end gradient propagation is essential. In this work, we present and compare several differentiation strategies for EM, from full automatic differentiation to approximate methods, assessing their accuracy and computational efficiency. As a key application, we leverage this differentiable EM in the computation of the Mixture Wasserstein distance MW2\mathrm{MW}_2 between GMMs, allowing MW2\mathrm{MW}_2 to be used as a differentiable loss in imaging and machine learning tasks. To complement our practical use of MW2\mathrm{MW}_2, we contribute a novel stability result which provides theoretical justification for the use of MW2\mathrm{MW}_2 with EM, and also introduce a novel unbalanced variant of MW2\mathrm{MW}_2. Numerical experiments on barycentre computation, colour and style transfer, image generation, and texture synthesis illustrate the versatility of the proposed approach in different settings.
May 11, 2025

Learning from samples: inverse problems over measures

We study inverse problems where an unknown potential is observed only through samples from the measure it induces by a convex variational principle. Such problems arise in learning costs, energies, and dynamics from distributional data, but the associated forward solution map is typically nonlinear and implicit. We show that its optimality gap nevertheless yields convex empirical objectives for finite-dimensional potential classes, and we introduce sharpened Fenchel--Young losses that add a data-dependent discrepancy inside the forward problem. This keeps the estimator calibrated while improving the local geometry of the loss. Our main stability theorem separates the inverse error analysis into measurement error, forward perturbation, and empirical curvature. We instantiate this principle for inverse entropic unbalanced optimal transport and for inverse Jordan--Kinderlehrer--Otto (JKO) learning from independent snapshot samples, obtaining high-probability parameter recovery bounds. JKO schemes discretize Wasserstein gradient flows through a sequence of variational problems over measures, making them a natural language for population dynamics observed through snapshots. In this JKO case, the sharpened objective reduces to an unbalanced transport problem, which also clarifies the connection between variational gap losses and quadratic iJKO⋆^\star surrogates. Numerical experiments illustrate the conditioning effect of sharpening and its benefits for sparse inverse-gradient-flow recovery.
Mar 19, 2025

Regularized Unbalanced Optimal Transport with Pearson Marginal Penalties

We study quadratically regularized unbalanced optimal transport with Pearson marginal penalties and a lower bound δ≥0δ\ge0 on the plan, including the sparse case δ=0δ=0. We organize the analysis around the active interface, where the plan switches from its lower bound to larger values. We prove a Newton remainder estimate that depends linearly on the marginal measure of the boundary layer around this interface, and a maximum-norm inverse estimate uniform over all Newton derivatives and cell volumes. Together they give local superlinear convergence of semismooth Newton iteration in function space, of q-order at least 1+α1+α when the boundary-layer measure is O(tα)O(t^α), and a common convergence neighborhood along each refinement sequence whose data converge uniformly. Transverse interfaces give second-order supercloseness of the discrete potentials to the cell averages of the continuum potentials. A directional curvature condition on the input data verifies the interface condition before the solution is known and yields an explicit Newton convergence radius on all sufficiently fine meshes, uniformly in the lower bound. A globalized inexact semismooth Newton iteration has mesh-independent numbers of outer iterations and of full-array operations, that is, passes over the plan array. Its output satisfies an a posteriori error bound relative to the continuum optimal plan. Experiments test the rates at transverse interfaces and at interfaces with tangencies, the curvature criterion, and the algorithmic work. All estimates hold for a fixed regularization parameter and a fixed lower bound on the densities.
Jan 30, 2025

Double-Bounded Nonlinear Optimal Transport for Size Constrained Min Cut Clusterin

Min cut is an important graph partitioning method. However, current solutions to the min cut problem suffer from slow speeds, difficulty in solving, and often converge to simple solutions. To address these issues, we relax the min cut problem into a double-bounded constraint and, for the first time, treat the min cut problem as a double-bounded nonlinear optimal transport problem. Additionally, we develop a method for solving double bounded nonlinear optimal transport based on the Frank-Wolfe method (abbreviated as DNF). We prove that for convex problems satisfying Lipschitz smoothness, the DNF method can achieve a convergence rate of O(1t)\mathcal{O}(\frac{1}{t}). We apply DNF to size-constrained min-cut clustering and evaluate it on eight benchmark datasets. DNF achieves competitive clustering performance and matches or outperforms the compared baselines on several datasets and metrics.
Oct 2, 2024

Robustness and Structure Preservation in Flow-Based Generative Models via Wasserstein Path-Space Divergences

We introduce a novel Wasserstein-1 (W1W_1) path-space divergence for stochastic and deterministic dynamics and establish a Wasserstein Uncertainty Propagation (WUP) theorem that bounds the W1W_1 distance between terminal distributions by the proposed divergence, equivalently characterized by a weighted L2L^2 discrepancy between the underlying drifts and the W1W_1 distance between their initial measures. A key ingredient is a probabilistic framework combining adjoint Feynman-Kac representations with synchronous coupling (and reflection coupling on bounded domains), yielding Wasserstein stability estimates beyond existing PDE- and Girsanov-based approaches. The framework accommodates time-varying and possibly degenerate diffusion coefficients, empirical and singular measures, and remains valid in the deterministic limit of flow matching. Unlike KL-based uncertainty quantification bounds, it does not require absolute continuity of path measures and therefore remains well-defined in singular settings. As consequences of the WUP theorem, we derive W1W_1 robustness and generalization bounds for score-based generative models and flow matching at both population and finite-sample levels. We further specialize the framework to group-symmetric targets, providing the first error analysis of equivariant flow-based models and the first quantitative comparison between data augmentation and equivariant inductive bias. Our analysis identifies a symmetry-aware Wasserstein path-space divergence that quantifies the model-form error induced by non-equivariant parametrizations. We prove that this error cannot be removed by additional data or training and vanishes only under equivariant architectures, establishing a precise theoretical advantage of equivariant inductive bias over data augmentation. Numerical experiments on group-symmetric Gaussian mixtures corroborate the theory.
Sep 3, 2024

Multi-view Optimal Transport for Multi-view Crowd Localization and Tracking

Multi-view crowd localization and tracking aim to predict the ground-plane positions of all individuals in the scene and estimate their trajectories over time. Existing multi-view localization methods typically rely on Gaussian-smoothed density map supervision, whose inherent ambiguity in crowded regions may blur local peaks and degrade localization accuracy. In this work, we introduce a multi-view optimal transport framework that directly supervises point-level predictions on the ground plane, thereby avoiding the limitations of density map regression. Specifically, we propose a Mahalanobis distance-based multi-view optimal transport loss (M-MVOT), whose transport cost incorporates view ray direction and object-to-camera distance within a Mahalanobis formulation, with the direction guiding the ellipse orientation and the distance modulating the cost magnitude. For multi-view tracking, we first extend the proposed localization framework in a tracking-by-detection manner (MVOT-TBD). To further improve performance, we introduce a tracking-specific optimal transport loss (MVOT-Track) that explicitly supervises inter-frame association. Extensive experiments demonstrate that the proposed optimal transport-based framework consistently outperforms density map-based localization and tracking on multiple multi-view crowd datasets. Code is available at https://github.com/zqyq/MVOT-Track.
Jun 19, 2024

CombAlign: Enhancing Model Expressiveness in Unsupervised Graph Alignment

Unsupervised graph alignment finds the node correspondence between a pair of attributed graphs by only exploiting graph structure and node features. One category of recent studies first computes the node representation and then matches nodes with the largest embedding-based similarity, while the other category reduces the problem to optimal transport (OT) via Gromov-Wasserstein learning. However, it remains largely unexplored in the model expressiveness, as well as how theoretical expressivity impacts prediction accuracy. We investigate the model expressiveness from two aspects. First, we characterize the model's discriminative power in distinguishing matched and unmatched node pairs across two graphs. Second, we study the model's capability of guaranteeing node matching properties such as one-to-one matching and mutual alignment. Motivated by our theoretical analysis, we put forward a hybrid approach named CombAlign with stronger expressive power. Specifically, we enable cross-dimensional feature interaction for OT-based learning and propose an embedding-based method inspired by the Weisfeiler-Lehman test. We also apply non-uniform marginals obtained from the embedding-based modules to OT as priors for more expressiveness. Based on that, we propose a traditional algorithm-based refinement, which combines our OT and embedding-based predictions using the ensemble learning strategy and reduces the problem to maximum weight matching. With carefully designed edge weights, we ensure these matching properties and further enhance prediction accuracy. By extensive experiments, we demonstrate a significant improvement of 14.5% in alignment accuracy compared to state-of-the-art approaches and confirm the soundness of our theoretical analysis.
Dec 15, 2023

Optimal Transport Aggregation for Distributed Mixture-of-Experts

This paper considers the setting in which local Mixture-of-experts (MoE) models are fitted independently on distributed datasets and must be combined into a single global MoE model. Weighted averaging the local mixtures, whose experts are not aligned across machines, often yields a large mixture with many experts instead of KK and does not provide an estimator of the true model parameters. We propose to reduce this large mixture to a KK-expert MoE by minimizing an expected transportation divergence, in which the transport plan depends on the covariates. A relaxation of this problem separates the experts from the gating network: the experts are computed by a majorization--minimization algorithm, which decreases the objective monotonically, and the gating network is then fitted to the gates implied by the optimal plan, with an explicit bound on the resulting loss. We show that the aggregated estimator is consistent whenever the local estimators are, without requiring the local models to have the same labeling across machines. The procedure only needs a single one-way communication round. Experiments on simulated and real data show accuracy close to centralized estimation at a fraction of its computation time. Our source code is available at https://github.com/nhat-thien/Distributed-Mixture-Of-Experts.
Oct 13, 2023

Structured Approximations of Measures

We study the approximation of probability measures in the Wasserstein-pp distance by structured classes of approximators, motivated by applications in imaging, machine learning, and physical measurement under sensor constraints. We obtain three sets of results. First, for measures with densities bounded away from zero on a bounded Lipschitz domain ΩΩ, we prove that any approximation scheme for functions in Lp(Ω)\mathrm{L}_p(Ω) transfers, with linear rate, to a corresponding approximation scheme for measures in Wp(Ω)\mathrm{W}_p(Ω). The argument applies a theorem of Bogovskii on regularity of solutions to the continuity equation in the Benamou-Brenier formulation of optimal transport. We exhibit concrete approximation schemes (polynomials, shift-invariant spaces, cardinal interpolation with radial basis functions, kernel density estimators, and piecewise approximations on nonuniform Voronoi partitions) that fit the framework. As a matter of independent interest, we prove a negative Sobolev lower bound that generalizes existing bounds from p=2p=2 to all p∈(1,∞)p\in(1,\infty). We also consider deterministic bounds for discrete approximations to arbitrary measures in terms of the mesh norm of a quasi-uniform set of points. We specialize these bounds to show that compactly supported measures admit a deterministic NN-term approximation μNμ_N such that Wp(μ,μN)=O(N−1d)\mathrm{W}_p(μ,μ_N) = O(N^{-\frac{1}{d}}) for all d≥1d\geq 1, which matches the asymptotic optimal quantizer rate. We also extend these results to non-compactly supported measures with appropriate tail decay.
Feb 14, 2023

Linearized Wasserstein dimensionality reduction with approximation guarantees

We introduce LOT Wassmap, a computationally feasible algorithm to uncover low-dimensional structures in the Wasserstein space. The algorithm is motivated by the observation that many datasets are naturally interpreted as probability measures rather than points in Rn\mathbb{R}^n, and that finding low-dimensional descriptions of such datasets requires manifold learning algorithms in the Wasserstein space. Most available algorithms are based on computing the pairwise Wasserstein distance matrix, which can be computationally challenging for large datasets in high dimensions. Our algorithm leverages approximation schemes such as Sinkhorn distances and linearized optimal transport to speed-up computations, and in particular, avoids computing a pairwise distance matrix. We provide guarantees on the embedding quality under such approximations, including when explicit descriptions of the probability measures are not available and one must deal with finite samples instead. Experiments demonstrate that LOT Wassmap attains correct embeddings and that the quality improves with increased sample size. We also show how LOT Wassmap significantly reduces the computational cost when compared to algorithms that depend on pairwise distance computations.
Feb 18, 2021

Deep Residual Networks Learn the Geodesic Curve in the Wasserstein Space

Recent studies revealed the mathematical connection between deep neural networks (DNNs) and dynamic systems. However, the specific dynamics that DNNs, especially deep residual networks (ResNets), tend to learn during training remain insufficiently characterized. To this end, we model the forward propagation of deep residual networks using continuity equations, in which the measure is conserved and infinite curves in the measure space connect the input distribution to the output one of a ResNet. We find ResNets with L2L_2 regularization attempt to learn the geodesic curve in the Wasserstein space, induced by the optimal transport map. Compared with plain networks, ResNets can better approximate the geodesic curve, which explains why ResNets can be optimized and generalize better. Numerical experiments show that the data tracks of a ResNet tend to be line-shaped in terms of the line-shape score, and the map learned by a ResNet is closer to the optimal transport map in terms of the optimal transport score. In a word, we conclude that ResNets learn the geodesic curve in the Wasserstein space and discretely engineer the data transformation in high-dimensional spaces.