Speculative decoding speeds up generation by using a small, cheap drafter to propose tokens, which are then verified in parallel against the larger target model. The abstract notes that this approach stops paying off at ultra-long context, because a drafter that attends over the full prefix inherits the same context-dependent cost as the target, erasing the benefit.

The paper's title states that long-context drafting does not require long-context drafters. That points to a design where the drafter does not attend to the entire prefix, avoiding the cost that makes speculative decoding unprofitable at extreme lengths.

The abstract is truncated, so the specific method and experimental results are not available in the source. The claim is clear, but the details of how the short-context drafter is trained or verified remain unspecified here.