Speculative decoding speeds up language model inference by drafting tokens with a cheap model and then verifying them with a larger model. Its effectiveness, as the abstract notes, depends on two quantities: how many drafted tokens are accepted and how much it costs to produce those drafts. These are usually treated as a trade-off, but the new paper argues they can be co-optimized.

JetSpec++ is presented as an algorithm-system co-design that targets both objectives together. The method's full name, "Foresight Parallel Drafting with Self Refinements," hints at a strategy that looks ahead and refines its own drafts, though the abstract does not provide further implementation details.

The source is limited to an abstract, so specifics of the co-design and experimental results are not available here. What is clear is the paper's central claim: acceptance rate and drafting cost need not be balanced against each other, but can be improved simultaneously.