Discrete Diffusion with Sample-Efficient Estimators for Conditionals

Karthik Elamvazhuthi; Abhijith Jayakumar; Andrey Y. Lokhov

cs.LGFeb 23, 2026

Discrete Diffusion with Sample-Efficient Estimators for Conditionals

Authors: Karthik Elamvazhuthi, Abhijith Jayakumar, Andrey Y. Lokhov

Abstract

We study a discrete denoising diffusion framework that integrates a sample-efficient estimator of single-site conditionals with round-robin noising and denoising dynamics for generative modeling over discrete state spaces. Rather than approximating a discrete analog of a score function, our formulation treats single-site conditional probabilities as the fundamental objects that parameterize the reverse diffusion process. We employ a sample-efficient method known as Neural Interaction Screening Estimator (NeurISE) to estimate these conditionals in the diffusion dynamics. Controlled experiments on synthetic Ising models, MNIST, and scientific data sets produced by a D-Wave quantum annealer, synthetic Potts model and one dimensional quantum systems demonstrate the proposed approach. On the binary data sets, these experiments demonstrate that the proposed approach outperforms popular existing methods including ratio-based approaches, achieving improved performance in total variation, cross-correlations, and kernel density estimation metrics.

Figures & tables

Panel of Figure 1 — Figure 1: Trend of TV and Cross-correlation error as a function of training set size for Ising models, averaged over $5$ Ising models and $10$ trials per data set.

Panel of Figure 2 — Figure 2: Class-conditional MNIST samples. Each subfigure shows generated samples arranged with one row per digit, from 0 to 9.

Table 1: Comparison of generative models on MNIST and D-Wave datasets.

Panel of Figure 3 — Figure 3: Trends for NeuRISE Diffusion as a function of training set size, averaged over $10$ trials for each size. The test sample size was $10^{5}$ for each experiment. Error bars represent one standard deviation over trials. Left: TV for a non-binary Potts model. Right: cross-correlation error of NeuRISE Diffusion trained to learn the GHZ state.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Trend of TV and Training time for Ising models, averaged $10$ trials per data set. The test sample size was taken to be $10^{4}$ for each experiment. The harsh noise version of the problem performs competitively with situations where noise is soft.

Panel of Figure 5 — Figure 5: Trend of TV and Training time for Ising models, averaged $10$ trials per data set. The test sample size was taken to be $10^{4}$ for each experiment. Learning individual conditionals using separate neural networks achieves lower TV, than using one global neural network across all conditionals.

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,169 papers

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models
Oct 9, 2026Qiuyun ZhuConditional Diffusion Models Diffusion Sampling
Department of Statistics and Actuarial Science University of Iowa
Discrete Diffusion Models via Evolving Variational Autoregressive Networks
Sep 23, 2026Kewen Pan, Ying TangDiffusion Sampling Density Estimation
Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Physics, University of Electronic Science and Technology of China, Chengdu 611731, China · Key Laboratory of Quantum Physics and Photonic Quantum Information, Ministry of Education, University of Electronic Science and Technology of China, Chengdu 611731, China +1
Conditional Diffusion Sampling
May 5, 2026Francisco M. Castro-Macías, Pablo Morales-Álvarez, Saifuddin Syed +3Markov Chain Monte Carlo Sampling from Unnormalized Distributions
University of Granada · Work done while visiting the University of Cambridge. · University of British Columbia +2

Oct 9, 2026J/K move · Enter open · S save

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models

Qiuyun Zhu

Department of Statistics and Actuarial Science University of Iowa

Conditional Diffusion Models Diffusion Sampling Conditional Distribution Estimation

Open paper AskPDF arXiv

We establish finite-sample guarantees for conditional diffusion models learned from data and implemented with finitely many denoising steps. Our approach aligns the score-training objective with the sampling grid, allowing statistical estimation and numerical discretization errors to be analyzed together. Under a smooth conditional density model, we derive high-probability bounds on the generated distribution's error and quantify the required number of score evaluations. For every smoothness index

β>0

, a suitable grid yields averaged conditional Kullback--Leibler (KL) divergence

\widetilde O_p(n^{-2β/(d_x+d_y+2β)})

relative to the original target with or without early stopping. Both the full and early-stopped samplers attain average target total-variation (TV) error

\widetilde O_p(n^{-β/(d_x+d_y+2β)})

. Here

n

is the sample size,

d_x,d_y

are the data and conditioning dimensions, and errors are averaged over the conditioning variable. The total-variation bound matches the rate established in existing conditional distribution-estimation theory, up to logarithmic factors.

Abstract

Diffusion Models

Explore similar work

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models

Discrete Diffusion Models via Evolving Variational Autoregressive Networks

Conditional Diffusion Sampling