Speculative decoding is a technique for speeding up large language model inference. A small, fast drafter model proposes several tokens, and the target model verifies them in a single parallel pass. This reduces the number of sequential steps needed, but the quality of the draft limits the speedup.
DominoTree, described in a recent arXiv paper, improves on this by having the drafter propose a tree of tokens rather than a single linear sequence. Because the target model can verify many alternative branches at once, the chance of accepting a longer draft increases. The paper describes this as conditional tree-structured drafting and introduces a mechanism called Domino to coordinate the process.
The abstract is truncated in the source, so the full algorithmic details are not available here. Still, the core idea is clear: by expanding the draft space from a line to a tree, DominoTree aims to make speculative decoding more efficient without changing the target model's outputs.