Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion

Renxiang Huang; Yuchen Liang; Yingbin Liang; Ness Shroff; Lifeng Lai

stat.MLOct 15, 2026

Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion

Authors: Renxiang Huang, Yuchen Liang, Yingbin Liang, Ness Shroff, Lifeng Lai

Organizations: Department of Electrical and Computer Engineering, University of California, Davis, Davis, CA, 95616, USA · Luddy School of Informatics, Computing, and Engineering, Indiana University Indianapolis, Indianapolis, IN, 46202, USA · Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH, 43210, USA

Abstract

Discrete diffusion models have become a powerful framework for modeling complex discrete distributions across a range of applications. While theoretical convergence guarantees depend on consistency between the target and training distributions, such a case rarely holds in practice, making score mismatch a central challenge in discrete diffusion sampling. This is particularly relevant to zero-shot conditional generation, where a model trained on the unconditional distribution is directly reused at inference time to sample from a conditional target distribution. In this paper, we provide the first theoretical analysis for score-mismatched discrete diffusion models and derive a general KL divergence upper bound that clearly identifies the additional bias caused by such a mismatch. Our analysis covers partial-observation constraints, including text infilling, sentence completion, and more general coordinatewise set constraints. Specifically, we analyze the classical Conditional SEDD sampler and show that it can incur a bias term that scales with the dimension, vocabulary size, and compatibility. Based on this analysis, we propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints. We show that the assignment probabilities can be expressed exactly by the unconditional scores with absorbing rate. Finally, numerical and real-data experiments demonstrate the effectiveness of the proposed RHC sampler.

Figures & tables

Fig. 1: Comparison of the Conditional SEDD and Oracle with fixed observation under absorbing rate in terms of the final KL divergence between the target and sampling conditional distributions.

Fig. 2: Comparison of the Conditional SEDD and Oracle with fixed observation under uniform rate in terms of the final KL divergence between the target and sampling conditional distributions.

Fig. 3: Comparison of Conditional SEDD with different numbers of free coordinates under the uniform rate in terms of the final KL divergence between the target and sampling conditional distributions. The different sizes of $\Omega$ are $1$ , $2$ , $3$ .

Fig. 4: Comparison of the RHC and Oracle in terms of the final KL divergence between the target and sampling conditional distributions.

Sampler	Transportation	Study	Game
Uniform Assignment	100.61 $\pm$ 1.72 (1610 s)	99.54 $\pm$ 1.44 (1611 s)	103.35 $\pm$ 1.52 (1605 s)
RHC	61.01 $\pm$ 1.53 (1660 s)	62.82 $\pm$ 1.08 (1664 s)	69.57 $\pm$ 0.75 (1659 s)
Sampler	Cooking	Project	Weather
Uniform Assignment	135.37 $\pm$ 0.76 (1601 s)	98.26 $\pm$ 2.66 (1606 s)	91.22 $\pm$ 1.31 (1600 s)
RHC	116.82 $\pm$ 2.06 (1655 s)	77.48 $\pm$ 1.35 (1660 s)	65.20 $\pm$ 1.27 (1650 s)

TABLE I: Summary of perplexity in transportation, study, game, cooking, project and weather. The results include the mean

\pm

standard deviation and total generation time of five independent trials.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Scenario	Position	Candidate Tokens
Transportation	7	plane, bus, taxi, bike, train, car
	15	fast, quick, rapid, swift, speedy, brisk
	22	anxious, nervous, worried, tense, uneasy, afraid
	27	but, however, yet, while
	36	safe, secure, protected, reliable, stable, harmless
	44	calm, relaxed, peaceful, relieved, quiet, comfortable

Appendix

TABLE II: Summary of scenarios.

Scenario	Generated Sentences
Transportation	We started driving together in a single car and quickly became hooked on sharing this fast driving fantasy. We were both nervous at this point, yet aware it’s important to be safe in driver jobs. We got more comfortable working on this scenario.
	We’re looking for a car . We’ll make it quick . I’m not worried about driver hours, but confident we’ll be comfortable and safe . I’m relieved to make it in.
	We’re waiting for the train , and it’ll be fast . It’s a tense , terrifying experience, but almost everyone is enjoying it and everything is safe . Our focus right now is staying calm and enjoy the train.
Game	The team then had a very difficult match . Our team manager made the fans nervous . The team reminded us of losing a great game, but this is it. The team rallied and scored our first goals. The team still advanced to the seventh round.
	The team took the field in the match . They played in a group with weak European play. They ended their losing streak in the match but they went through a difficult patch. They fought a very good campaign and the team won their four home matches.
	The team had competed in their early race . The team had suffered to a slow start, suffering the misfortune of losing in first race, but a few hours later, the team had improved to a higher level. They had qualified for their second race.

Appendix

TABLE III: Examples of generated sentences. The bold text shows the selected constrained position.

Scenario	Generated Sentences
Cooking	We had tossed green wasabi salad at sea. The combination of roasted pork and cilantro was very spicy and fragrant, but it varied a little bit in this salty -and-cheese dish. I enjoyed roasted pork and wasabi salad.
	We also share the grilled cheese pizza . This improved version of the grilled cheese pizza is fortified with the sweet and savory flavours while marinated with the plum and the salty onion, balanced by the juicy cheese and served with the tart and grape sticks.
	Weiner’s sandwich is a salad . It tastes like a ’ smoked ’ flavor of avocado, almost spicy , and sweet, but it does not have a ’ spicy salty ’ flavor of onions. It is also served with hot and fried meat rolls.
Project	The team first introduces a new system . The design is far more complex . The same system types are broken into primary domains, while differences in each case have to be tested separately. Once the system is ready , new features can be tested.
	The team managed to rebuild the platform and then solve a few more complex problems. The platform was very unstable from the beginning, but soon, its stability was increased and improved . The platform was once again in a stable condition and also with new challenges.
	The team working on the main project had sat for long periods with difficult problems. Work there had been delayed for several months, but after several months, the problems were fixed . The project was almost complete and was ready to be done before the holidays.

Appendix

TABLE IV: Examples of generated sentences. The bold text shows the selected constrained position.

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,163 papers

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models
Oct 9, 2026Qiuyun ZhuConditional Diffusion Models Diffusion Sampling
Department of Statistics and Actuarial Science University of Iowa
Mean-to-Score Discrete Diffusion: Posterior-Mean Denoisers for Score Entropy
Jul 23, 2026Jingyuan Li, Xiaoyi Jiang, Yixuan Jiang +4Constrained Generative Modeling Markov Models
Tsinghua University · Beijing Institute of Mathematical Sciences and Applications · Wuhan University +1
Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
Feb 16, 2026Daniil Dmitriev, Zhihan Huang, Yuting WeiSample Complexity Diffusion Model Sampling
Department of Statistics and Data Science, the Wharton School, University of Pennsylvania

Oct 9, 2026J/K move · Enter open · S save

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models

Qiuyun Zhu

Department of Statistics and Actuarial Science University of Iowa

Conditional Diffusion Models Diffusion Sampling Conditional Distribution Estimation

Open paper AskPDF arXiv

We establish finite-sample guarantees for conditional diffusion models learned from data and implemented with finitely many denoising steps. Our approach aligns the score-training objective with the sampling grid, allowing statistical estimation and numerical discretization errors to be analyzed together. Under a smooth conditional density model, we derive high-probability bounds on the generated distribution's error and quantify the required number of score evaluations. For every smoothness index

β>0

, a suitable grid yields averaged conditional Kullback--Leibler (KL) divergence

\widetilde O_p(n^{-2β/(d_x+d_y+2β)})

relative to the original target with or without early stopping. Both the full and early-stopped samplers attain average target total-variation (TV) error

\widetilde O_p(n^{-β/(d_x+d_y+2β)})

. Here

n

is the sample size,

d_x,d_y

are the data and conditioning dimensions, and errors are averaged over the conditioning variable. The total-variation bound matches the rate established in existing conditional distribution-estimation theory, up to logarithmic factors.

Abstract

Diffusion Models

Explore similar work

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models

Mean-to-Score Discrete Diffusion: Posterior-Mean Denoisers for Score Entropy

Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees