cs.DSApr 4, 2026

On the Efficiency of Sinkhorn-Knopp for Entropically Regularized Optimal Transport

Authors: Kun He

Organizations: Renmin University of China

Abstract

The Sinkhorn--Knopp (SK) algorithm is a cornerstone method for matrix scaling and entropically regularized optimal transport (EOT). Despite its empirical efficiency, existing theoretical guarantees to achieve a target marginal accuracy ε\varepsilon deteriorate severely in the presence of outliers, bottlenecked either by the global maximum regularized cost η∥C∥∞η\|C\|_\infty (where ηη is the regularization parameter and CC the cost matrix) or the matrix's minimum-to-maximum entry ratio νν. This creates a fundamental disconnect between theory and practice. In this paper, we resolve this discrepancy. For EOT, we introduce the novel concept of well-boundedness, a local bulk mass property that rigorously isolates the well-behaved portion of the data from extreme outliers. We prove that governed by this fundamental notion, SK recovers the target transport plan for a problem of dimension nn in O(log⁡n−log⁡ε)O(\log n - \log \varepsilon) iterations, completely independent of the regularized cost η∥C∥∞η\|C\|_\infty. Furthermore, we show that a virtually cost-free pre-scaling step eliminates the dimensional dependence entirely, accelerating convergence to a strictly dimension-free O(log⁡(1/ε))O(\log(1/\varepsilon)) iterations. Beyond EOT, we establish a sharp phase transition for general (u,v)(\boldsymbol{u},\boldsymbol{v})-scaling governed by a critical matrix density threshold. We prove that when a matrix's density exceeds this threshold, the iteration complexity is strictly independent of νν. Conversely, when the density falls below this threshold, the dependence on νν becomes unavoidable; in this sub-critical regime, we construct instances where SK requires Ω(n/ε)Ω(n/\varepsilon) iterations.

Explore similar work

Aug 12, 2026

Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp

We revisit the Sinkhorn-Knopp (SK) algorithm for the matrix scaling problem. Despite extensive literature on the global convergence of SK and its variants, its local linear convergence behavior remains less understood. We address this gap by providing the first nonasymptotic local analysis of SK that matches the rate obtained from existing asymptotic Jacobian-based arguments. We show that under certain connectivity conditions, SK is a polynomial-time algorithm for doubly stochastic matrix scaling. With the developed tools, we showcase the local suboptimality of SK and provide accelerated variants. Finally, for dense matrices, we improve the complexity of existing first-order matrix scaling algorithms from O(n7/3ε2/3)O(\tfrac{n^{7/3}}{\varepsilon^{2/3}}) to O(n9/4ε)O(\tfrac{n^{9/4}}{\sqrt{\varepsilon}}).
Aug 28, 2026

SinkSLOT: Sinkhorn via Sparse Lifted Optimal Transport

Entropic optimal transport (EOT) has been shown to offer a computationally tractable approximation to exact optimal transport. However, the standard Sinkhorn-Knopp algorithm has two main limitations. First, given discrete measures with NN points, each iteration requires O(N2)O(N^2) operations, which restricts its use on large-scale datasets (e.g. N≥104N\geq10^4). Second, it uses the independent coupling as a reference measure for regularisation. This assigns mass to high-cost transport edges at moderate regularisation strengths. We propose SinkSLOT, which addresses both limitations by putting forth the expected sliced lifted transport plan as a natural way to sparsify the Gibbs kernel with a non-independent prior coupling. We prove that: 1) SinkSLOT converges; 2) with LL slices, each resulting sparse Sinkhorn iteration costs O(LN)O(LN); and 3) the resulting objective is a divergence requiring no debiasing. Experiments on synthetic benchmarks show that SinkSLOT delivers substantial speedups over state-of-the-art dense and sparse EOT methods. We also demonstrate the applicability of the proposed divergence in a gradient flow experiment. The code is publicly available at https://github.com/cai4cai/SinkSLOT.
Apr 4, 2026

Fast Log-Domain Sinkhorn Optimal Transport with Warp-Level GPU Reductions

Entropic regularized optimal transport (OT) via the Sinkhorn algorithm has become a fundamental tool in machine learning, yet existing implementations either suffer from numerical instability for small regularization parameters or incur significant overhead from deep learning frameworks. We present FastSinkhorn, a lightweight, native CUDA implementation of the log-domain Sinkhorn algorithm that combines warp-level shuffle reductions with shared-memory tiling to achieve high GPU utilization without sacrificing numerical stability. Our solver operates entirely in the log-domain, enabling robust computation for regularization parameters as small as epsilon = 10^{-4} where standard-domain methods fail. On dense OT problems with n = m = 8192, our implementation achieves 12x speedup over the widely-used POT library and 5.9x speedup over GPU-accelerated PyTorch baselines, while consuming only 256 MB of GPU memory. We validate our solver on image color transfer, 3D point cloud matching, and convergence analysis, demonstrating that native CUDA kernels with careful numerical treatment provide a practical and efficient foundation for large-scale optimal transport computation.