Standard autoregressive language models generate text one token at a time, even when several upcoming tokens are obvious. A new arXiv preprint (arXiv:2610.22788) proposes CADENCE (Context-Adaptive DEcoding), a method that learns both what to draft and how far to decode.
The key idea is that the decoding step size should not be fixed. Instead, the model can decide, based on context, how many tokens to generate in a single step. This could speed up generation by reducing the number of decoding steps, though the abstract does not include experimental results.
Because only the abstract is available, there are no details on benchmarks or comparisons with other methods. The stated contribution is the learned combination of content selection and decoding distance.