Speculative decoding has become a standard strategy for speeding up large language model inference: a lightweight draft model proposes tokens that a larger target model verifies in parallel. Two new arXiv preprints show the approach still has room to grow, though they push it in different directions.
The first, Draft-OPD, focuses on how draft models are built. The authors position their work against common drafting methods such as EAGLE3 and DFlash, and describe an on-policy distillation approach for training the draft model. The emphasis is on improving the draft model itself rather than changing the verification stage.
The second preprint takes a different angle: it combines multi-draft speculative sampling with watermarking via Poisson processes. Instead of targeting speed alone, it aims to add output provenance, so that text generated with speculative decoding can be traced while still leveraging parallel verification from multiple draft models.
Taken together, the two papers agree that speculative decoding is a key tool for efficient LLM deployment. They differ in focus: Draft-OPD improves the quality of the draft model, while the watermarking paper addresses the provenance gap that arises when sampling is spread across multiple drafts.