Speculative decoding has become a standard technique for accelerating LLM generation: a small model drafts tokens and a larger model verifies them in parallel. A new arXiv paper, FOLD, targets the long-context setting, where the promise is that the cost of accessing the KV cache can be spread across many accepted tokens.
The paper's abstract argues that this amortization makes speculative decoding particularly promising for long inputs. It then introduces FOLD, described as 'position-free far memory with local order.' The abstract cuts off mid-sentence after 'Yet…', so the specific problem FOLD solves and the details of its mechanism are not available from the source.
Given the truncated abstract, we can only report the paper's framing and the name of the proposed method. The full text would be needed to evaluate how FOLD achieves its goals.