Discrete diffusion models have become a standard tool for generating categorical data, but the field lacks a unified view of what these models actually learn. A new paper on arXiv points out that researchers train them using at least three different objectives: denoisers, bridge plug-ins, and concrete scores. Each of these comes with its own evidence lower bound (ELBO) derived under different conventions, which the authors say fragments the literature and makes it hard to compare results.
The paper's title suggests a way through the confusion: examining one reverse process in three coordinates. By looking at the same generative process from different mathematical perspectives, the authors aim to show how the various training objectives relate to a common underlying mechanism. This could help practitioners choose among objectives and understand why some work better in certain settings.
Because the abstract is truncated, the full derivation and conclusions are not available in the source. Still, the central claim is clear: the multiplicity of objectives is not just a cosmetic difference but a sign that the field lacks a shared conceptual foundation. The paper appears to be a step toward that foundation, though the details remain to be seen.