Speculative decoding accelerates large language model inference by having a small drafter propose tokens that a larger model verifies in parallel. Diffusion drafters take this further by generating multiple proposals at once, but the abstract notes that this comes with trade-offs in proposal capacity and mean acceptance rate, though the sentence is cut off in the available text.
The new paper introduces factorization-guided training and inference for these diffusion drafters. The approach appears to structure the drafting process to better handle the trade-off, although specific results are not included in the abstract.
As with any arXiv preprint, the work has not yet been peer-reviewed. The full paper would be needed to assess the method's effectiveness.