Interactive video generation usually relies on small bidirectional chunks to keep response latency low. The authors of this paper report, however, that larger chunks produce better video quality and make better use of GPU parallelism, allowing more latents to be sampled with only a modest increase in sampling time. That trade-off motivates their proposed strategy.
The strategy, called speculative sampling, combines the modeling power of large bidirectional windows with the low latency of small ones. It expands the modeling window by appending noise latents as padding, then caches the resulting predictions so they can be reused when the condition remains unchanged. Predictions are delivered incrementally, while the uncommitted future remains replaceable.
According to the authors' latency analysis, this removes the waiting-time lower bound imposed by chunk boundaries. Optimization focus shifts from reducing chunk size to accelerating per-chunk sampling, and bidirectional window size becomes a decision variable for jointly optimizing visual quality and sampling time rather than a necessary constraint on low-latency interaction.
The paper argues this opens a path to serving large bidirectional models in interactive streaming systems without complex streaming post-training. The work is currently available as a preprint on arXiv.