Speculative decoding accelerates autoregressive generation by having a small draft model propose tokens that a larger target model verifies. Researchers often measure the speedup by counting how many draft tokens are accepted per verification round. According to a new preprint on arXiv (2610.23818), that metric is only meaningful when verification is lossless—that is, when the target model's output distribution and expected completion length are preserved.
The paper's title, 'Acceptance Is Not Speed: Length Inflation in Training-Free Lossy Speculative Decoding,' signals the problem: in lossy verification, the draft model's proposals can change the output distribution and inflate the length of generated sequences. When that happens, a high acceptance rate may reflect longer outputs rather than faster generation. The authors argue that acceptance alone can therefore mislead practitioners about real speedups.
The abstract notes that lossless verification preserves expected completion length, which is why acceptance works as a proxy in that setting. The preprint appears to be a warning to the community: report length and latency alongside acceptance, or risk mistaking inflated outputs for efficiency gains.