Speculative decoding has become a mainstream technique for accelerating large language model (LLM) inference. It works by using a lightweight draft model to propose candidate tokens, which are then verified in parallel by the larger target model. A new arXiv paper introduces DLite, a variant designed specifically for agentic AI, and its key distinction is being "history-free."

The abstract does not give implementation details, but the name suggests DLite avoids depending on the draft model's past token history. That could simplify the drafting process and reduce memory overhead, which matters in agentic workloads where models generate long, multi-step outputs and latency is critical.

This is a single-source announcement, so the results are preliminary and have not been peer-reviewed. The paper is available on arXiv under identifier 2610.23787.