Discrete diffusion models typically assume the training and target distributions agree, but in zero-shot conditional generation that assumption fails: a model trained unconditionally is reused to sample from a conditional distribution. The authors provide the first theoretical analysis of this score-mismatch setting, deriving a general KL divergence upper bound that explicitly identifies the extra bias introduced by the mismatch. Their analysis covers partial-observation constraints such as text infilling, sentence completion, and coordinatewise set constraints.
Applying the bound to the classical Conditional SEDD sampler, they show it can incur a bias term that scales with the dimension, vocabulary size, and compatibility of the constraint. To address this, they propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints. RHC's assignment probabilities can be expressed exactly using the unconditional scores with an absorbing rate, avoiding the mismatch-induced bias of the earlier approach.
Numerical and real-data experiments demonstrate that RHC outperforms the baseline, confirming the theoretical predictions. The work is the first to rigorously characterize score mismatch in discrete diffusion and offers a practical fix for zero-shot conditional tasks.