stat.MLMay 30, 2025

Equilibrium Distribution for t-Distributed Stochastic Neighbor Embedding with Generalized Kernels

Authors: Yi Gu, Antonio Auffinger

Organizations: Department of Mathematics, Northwestern University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We extend the results by Auffinger and Fletcher in 2023 on the large-sample limit of t-SNE to a broader class of input and output kernels. We prove that under natural assumptions on the input distribution and the decay of the output kernel, if perplexity scales linearly with data size, the empirical measures and values of minimizers converge to an equilibrium distribution. We conclude with numerical examples comparing different kernels.

Figures & tables

Output kernelType of decayRestriction on ss
(1+r2/α)−α(1+r^{2}/\alpha)^{-\alpha}Regular, β=2α\beta=2\alphas≤2αs\leq 2\alpha
(1+r4)−1(1+r^{4})^{-1}Regular, β=4\beta=4s≤4s\leq 4
e−r2e^{-r^{2}} and e−r4e^{-r^{4}}RapidAll ss
e2−21+r2e^{2-2\sqrt{1+r^{2}}}RapidAll ss
e−r2−0.8sin⁡(r2)e^{-r^{2}-0.8\sin(r^{2})}RapidAll ss
Table 1: Examples satisfying Conditions 2 and 4 .
Namek(r)k(r)
Student-type(1+r2/α)−α(1+r^{2}/\alpha)^{-\alpha} , α∈{1/2,1,2}\alpha\in\{1/2,1,2\}
Gaussiane−r2e^{-r^{2}}
Quartic exponentiale−r4e^{-r^{4}}
Quartic rational(1+r4)−1(1+r^{4})^{-1}
Smooth exponentiale2−21+r2e^{2-2\sqrt{1+r^{2}}}
Rippled Gaussiane−r2−0.8sin⁡(r2)e^{-r^{2}-0.8\sin(r^{2})}
Table 2: The eight output kernels used in the simulations.
Figure 1: Two-dimensional outputs for 30003000 observations from N(0,I50)N(0,I_{50}) , with Perp=150\textbf{Perp}=150 . Rows vary the input kernel; columns vary the output kernel. The same observations and initial coordinates are used in every panel. Panel (a) is t-SNE and panel (b) is symmetric SNE. Axes show output coordinates after centering and orthogonal alignment, without a change of scale.
Figure 2: The same kernel combinations as in Figure 1 , for the balanced three-component Gaussian mixture in R50\mathbb{R}^{50} . There are 30003000 observations and Perp=150\textbf{Perp}=150 . Colors identify the generating component. All panels reuse the same sample and initialization.
Figure 3: MNIST: the same 30003000 images, represented by their 784784 pixel values, embedded in two dimensions under twelve kernel combinations. There are 300300 images per digit and Perp=150\textbf{Perp}=150 . Colors denote digit labels and are fixed across panels. The layout follows Figure 1 .
Figure 4: Fashion-MNIST: 30003000 images in R784\mathbb{R}^{784} , with 300300 images per clothing class and Perp=150\textbf{Perp}=150 . Rows, columns, initialization, and display conventions match Figure 3 . Class colors are shown in the legend.
Figure 5: Original SNE, Student-type α=1/2\alpha=1/2 output, and quartic rational output on the four main datasets. All panels use Gaussian input, n=3000n=3000 , and Perp=150\textbf{Perp}=150 . The first column uses ( 6.2 ); the others use the joint objective. Colors agree with the preceding figures. Full-view insets, where present, include remote points outside the central view.
Figure 6: Changing perplexity with the t-SNE kernels fixed. Each row uses the same 30003000 image vectors and initial coordinates as the corresponding main grid. The central views and any full-view insets retain the original output scales. Class colors follow Figures 3 and 4 .
More in this topic

Dimensionality Reduction

Dimensionality reduction maps high-dimensional data to compact representations for visualization, prediction, compression, and scientific modeling.

See all 145 papers

Explore similar work

CardsList
Apr 13, 2026

On the continuum limit of t-SNE for data visualization

This work is concerned with the continuum limit of a graph-based data visualization technique called the t-Distributed Stochastic Neighbor Embedding (t-SNE), which is widely used for visualizing data in a variety of applications, but is still poorly understood from a theoretical standpoint. The t-SNE algorithm produces visualizations by minimizing the Kullback-Leibler divergence between similarity matrices representing the high dimensional data and its low dimensional representation. We prove that as the number of data points n→∞n \to \infty, after a natural rescaling and in applicable parameter regimes, the Kullback-Leibler divergence is consistent as the number of data points n→∞n \to \infty and the similarity graph remains sparse with a continuum variational problem that involves a non-convex gradient regularization term and a penalty on the magnitude of the probability density function in the visualization space. These two terms represent the continuum limits of the attraction and repulsion forces in the t-SNE algorithm. Due to the lack of convexity in the continuum variational problem, the question of well-posedeness is only partially resolved. We show that when both dimensions are 11, the problem admits a unique smooth minimizer, along with an infinite number of discontinuous minimizers (interpreted in a relaxed sense). This aligns well with the empirically observed ability of t-SNE to separate data in seemingly arbitrary ways in the visualization. The energy is also very closely related to the famously ill-posed Perona-Malik equation, which is used for denoising and simplifying images. We present numerical results validating the continuum limit, provide some preliminary results about the delicate nature of the limiting energetic problem in higher dimensions, and highlight several problems for future work.

Jeff Calder, Zhonggan Huang, Ryan Murray +1

School of Mathematics, University of Minnesota · Department of Mathematics, University of Utah · Department of Mathematics, North Carolina State University

Data VisualizationDimensionality Reduction
AskPDF
Jul 16, 2026

Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limits

Stein variational gradient descent (SVGD) transports interacting particles toward a target distribution through deterministic kernelized dynamics. Singular Riesz kernels are attractive because they can provide quantitative population-level convergence, but at the finite-particle level the corresponding Stein energy has infinite self-interaction. We study periodic Riesz SVGD with self-interaction removed and prove a many-particle, long-time sampling theorem. Throughout the range in which the singular Stein energy is locally integrable, under a uniform bound on the initial relative entropy per particle, the time-averaged empirical-measure law converges weakly to the point mass δπδ_π at the target as the particle number and any diverging averaging horizon tend to infinity. We also show that the empirical-measure laws induced by invariant particle laws of finite relative entropy converge weakly to δπδ_π, without a uniform entropy bound. Below the logarithmic singularity threshold, we obtain an explicit algebraic finite-particle error bound. These results extend the joint-entropy approach for smooth-kernel SVGD to singular interactions.

Trevor Teolis, Maarten V. de Hoop

Rice University

Kernel MethodsStein Variational Gradient Descent
AskPDF
Jun 23, 2026

Sample complexity of unbalanced entropic OT

Optimal transport (OT) has become a central language for comparing probability measures, but exact balanced OT is often both too rigid for data with missing, created, or destroyed mass and subject to unfavorable high-dimensional sample complexity. Entropic regularization and unbalanced relaxations address these limitations in complementary ways. Entropy smooths the geometry, improves statistical behavior, and enables fast Sinkhorn-type algorithms, while unbalanced marginal penalties replace hard conservation constraints by divergence terms adapted to noisy empirical data. This paper studies the sample complexity of entropic unbalanced OT at the level of the optimal coupling, rather than only the scalar transport value. We develop a translation-invariant dual formulation, prove compactness and strong convexity properties for the intrinsic dual variables, and convert these geometric estimates into high-probability finite-sample bounds for empirical couplings. The results clarify why regularization is a practical necessity in machine learning applications: it softens the curse of dimensionality, reduces the number of samples needed for stable transport estimation, and keeps the resulting estimators compatible with scalable Sinkhorn-type solvers.

Francisco Andrade, Gabriel Peyré, Clarice Poon

INRIA (PreMeDICaL & HeKA) · INRIA · CNRS and ENS, PSL Universit´e +1

Sample ComplexityEntropy RegularizationOptimal Transport
AskPDF