Jumping up and down: Denoiser diffusion models for discrete ordinal data

Yair Shenfeld; Ricardo Baptista; Stefano Peluchetti

cs.LGOct 2, 2026

Jumping up and down: Denoiser diffusion models for discrete ordinal data

Authors: Yair Shenfeld, Ricardo Baptista, Stefano Peluchetti

Organizations: Division of Applied Mathematics Brown University Providence, RI, USA 02912 · Department of Statistical Sciences University of Toronto Toronto, ON, Canada M5G 1X6 · Sakana AI Tokyo, Japan

Abstract

Diffusion models are highly developed in continuous spaces for image and video domains. Recently, major advances have been made for discrete diffusion models for categorical data, specifically in the language domain. In contrast, diffusion models for discrete integer-valued data are less developed, despite the prevalence of this modality, ranging from images and music to gene counts. We introduce Jumping Up and Down (JUD)---a new family of denoiser-based diffusion models for discrete ordinal data. This is the first family of diffusion models for ordinal data which centers around training denoisers, which at the same time allows for bi-directional (up and down) perturbations of the data. The simplicity of the training objective, combined with the flexibility of bi-directional perturbations, leads us to obtain competitive results across different data modalities.

Figures & tables

Binomial–Poisson (ours)	Count Bridge ( Fishman et al., 2026 )

Binomial ( Shenfeld et al., 2026 )	Count Flow ( Wei & Pearson, 2026 )

Poisson	SEDD ( Lou et al., 2024 )

Figure 1: Sample paths for several noising processes in discrete diffusion models and the resulting histogram of generated samples (blue) relative to the true data distribution (black) Negative Binomial Mixture . We observe closest agreement in

W_{1}

distance to the data with the Binomial-Poisson process proposed in this work. See Appendix F for details.

Distribution	Shenfeld et al. (2026)		(Ours)		(Ours)		Fishman et al. (2026)		Wei & Pearson (2026)		Lou et al. (2024)
	Binomial-only		Binomial-Poisson		Poisson-Poisson		Count Bridges		count-FM		SEDD
Poisson	$0.31$ ${}\pm{}$	$0.05$	0.13 ${}\pm{}$	0.02	$0.35$ ${}\pm{}$	$0.02$	$0.22$ ${}\pm{}$	$0.02$	$0.28$ ${}\pm{}$	$0.06$	$0.46$ ${}\pm{}$	$0.08$
Poisson Mixture	$1.52$ ${}\pm{}$	$0.21$	1.39 ${}\pm{}$	0.30	$89.71$ ${}\pm{}$	$0.34$	$1.69$ ${}\pm{}$	$0.34$	$1.68$ ${}\pm{}$	$0.41$	$1.96$ ${}\pm{}$	$0.33$
Zero-Inflated Poisson	$0.20$ ${}\pm{}$	$0.03$	0.12 ${}\pm{}$	0.01	$0.32$ ${}\pm{}$	$0.02$	$0.16$ ${}\pm{}$	$0.03$	$0.33$ ${}\pm{}$	$0.06$	$0.53$ ${}\pm{}$	$0.08$
Negative Binomial Mixture	$1.68$ ${}\pm{}$	$0.23$	1.62 ${}\pm{}$	0.56	$17.73$ ${}\pm{}$	$0.41$	$1.80$ ${}\pm{}$	$0.74$	$2.45$ ${}\pm{}$	$0.46$	$2.35$ ${}\pm{}$	$0.33$
Beta Negative Binomial	$0.87$ ${}\pm{}$	$0.07$	0.65 ${}\pm{}$	0.06	$6.14$ ${}\pm{}$	$0.14$	$0.73$ ${}\pm{}$	$0.07$	$1.52$ ${}\pm{}$	$0.14$	$2.57$ ${}\pm{}$	$0.14$

Table 1: Wasserstein-1 distance across distributions and sampling methods (mean

\pm

std). Our process is denoted in bold and the best results are highlighted in orange.

Panel of Figure 2 — Figure 2: Target dataset and the generated samples using the Binomial-Poisson process for all eight synthetic distributions.

Table 4

Autoregressive models
Method	Accuracy (%) $\uparrow$
AR (sampled)	0.6
AR (greedy)	1.0
Discrete diffusion models
MDLM	30.2
Duo	58.4

Table 3: Hard Sudoku exact-board accuracy. Baselines are from Table 1 of Deschenaux & Gulcehre (2026) . Our scores are means (not best) over eight independent training runs.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

$c_{\textnormal{in}}(\sigma)$	$\frac{1}{\sqrt{v_{Y}(\sigma)}}$
$s_{\textnormal{in}}(\sigma)$	$-\frac{\mu_{Y}(\sigma)}{\sqrt{v_{Y}(\sigma)}}$
$c_{\textnormal{skip}}(\sigma)$	$\frac{\textnormal{cov}_{A,Y}(\sigma)}{v_{Y}(\sigma)}$
$c_{\textnormal{bias}}(\sigma)$	$\mu_{A}(\sigma)-c_{\textnormal{skip}}(\sigma)\mu_{Y}(\sigma)$
$C(\sigma)$	$v_{A}(\sigma)-\frac{\textnormal{cov}_{A,Y}^{2}(\sigma)}{v_{Y}(\sigma)}$
$c_{\textnormal{out}}(\sigma)$	$\sqrt{C(\sigma)}$
$w^{2}(\sigma)$	$\frac{1}{C(\sigma)}$

Appendix

Table 4: EDM scalings for target

A_{\sigma}

and input

Y_{\sigma}

Target $A_{t}$	$\mu_{A}(t)$	$v_{A}(t)$	$\textnormal{cov}_{A,Y}(t)=\operatorname{Cov}(A_{t}^{i},Y_{t}^{i})$
$\mathrm{B}_{t}$	$l_{t}\mu$	$V_{B}(t)$	$V_{B}(t)$
$\mathbf{x}_{\textnormal{data}}$	$\mu$	$v$	$l_{t}v$
$\mathrm{N}_{t}$	$\mathrm{u}_{t}$	$\mathrm{u}_{t}$	$\mathrm{u}_{t}$
$\mathrm{L}_{t}$	$(1-l_{t})\mu$	$V_{L}(t)$	$l_{t}(1-l_{t})(v-\mu)$

Appendix

Table 5: Moments of the regression targets for the Binomial-Poisson perturbation

X_{t}=\mathrm{B}_{t}+\mathrm{N}_{t}

Target $A_{t}$	$\mu_{A}(t)$	$v_{A}(t)$	$\textnormal{cov}_{A,Y}(t)=\operatorname{Cov}(A_{t}^{i},Y_{t}^{i})$
$\mathrm{U}_{t}$	$\mathrm{u}_{t}$	$\mathrm{u}_{t}$	$\mathrm{u}_{t}$
$\mathrm{D}_{t}$	$l_{t}$	$l_{t}$	$-l_{t}$

Appendix

Table 6: Moments of the regression targets for the Poisson-Poisson perturbation

Y_{t}=\mathbf{x}_{\textnormal{data}}-\mathrm{D}_{t}+\mathrm{U}_{t}

Method	Reference distribution
Binomial-Poisson	$\operatorname{Poisson}(20)$
Binomial ( Shenfeld et al., 2026 )	$\delta_{0}$
Poisson	$\operatorname{Poisson}(196)$
Count Bridge ( Fishman et al., 2026 )	$\operatorname{Skellam}(20,20)$
Count Flow ( Wei & Pearson, 2026 )	$\operatorname{Unif}\{0,\ldots,149\}$
SEDD ( Lou et al., 2024 )	$\operatorname{Unif}\{0,\ldots,149\}$

Appendix

Table 7: Reference distributions used by different methods.

Sampler	256	512	1024	2048
Euler	7.07	2.22	2.15	2.09
tau-leaping	3.27	2.29	2.11	2.07
tau-leaping + marginal-preserving rates	5.46	2.95	2.34	2.20

Appendix

Table 8: CIFAR-10 FID at different numbers of sampling steps (NFE).

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,169 papers

Binomial flows: Denoising and flow matching for discrete ordinal data
May 1, 2026Yair Shenfeld, Ricardo Baptista, Stefano PeluchettiFlow Matching Flow-Based Generative Modeling
CountsDiff: A Diffusion Model on the Natural Numbers for Generation and Imputation of Count-Based Data
Apr 4, 2026Renzo G. Soatto, Anders Hoel, Greycen Ren +5Diffusion Models Synthetic Data Generation
Massachusetts Institute of Technology (MIT), Cambridge, MA, USA · Laboratory for Information and Decision Systems (LIDS), MIT, Cambridge, MA, USA · Eric and Wendy Schmidt Center, Broad Institute, Cambridge, MA, USA +1
What Does a Discrete Diffusion Model Learn? One Reverse Process in Three Coordinates
Jul 6, 2026Rodrigo Casado Noguerales, Bernhard Schölkopf, Thomas Hofmann +1Markov Models Variational Inference
ETH Zurich · Max Planck Institute for Intelligent Systems, Tübingen · ELLIS Institute Tübingen

May 1, 2026J/K move · Enter open · S save

Binomial flows: Denoising and flow matching for discrete ordinal data

Yair Shenfeld, Ricardo Baptista, Stefano Peluchetti

Flow Matching Flow-Based Generative Modeling Diffusion Models Discrete Flow Matching

Open paper AskPDF arXiv

Flow-based generative modeling in continuous spaces exploit Tweedie's formula to express the denoiser (learned in training) as a score function (used in sampling). In contrast, this relation has been largely missing in the discrete setting where common approaches focus on learning discrete scores and rates. In this work we close this gap for discrete non-negative ordinal data by introducing Binomial flows. Our framework provides a simple recipe for training a discrete diffusion model which simultaneously denoises, samples, and estimates exact likelihoods. We verify our methodology on synthetic examples and obtain competitive results on real-world data sets.

Abstract

Diffusion Models

Explore similar work

Binomial flows: Denoising and flow matching for discrete ordinal data

CountsDiff: A Diffusion Model on the Natural Numbers for Generation and Imputation of Count-Based Data

What Does a Discrete Diffusion Model Learn? One Reverse Process in Three Coordinates