Autoregressive video generation models produce frames sequentially, and as the video grows, they must retain all previous key-value (KV) states. That accumulation drives up memory usage and per-step attention costs, making long videos increasingly expensive to generate.
FrameFold, described in a new arXiv paper, addresses this by consolidating the KV cache under a fixed budget. Instead of keeping every historical state, the method selectively merges or drops states to stay within the budget, which bounds both memory and computation.
The paper's key innovation is that consolidation is response-guided: the decisions about which KV states to keep are informed by the model's own generated output, rather than being static or heuristic. This allows the cache to adapt to the content being produced, potentially preserving important information while discarding redundant states.
The abstract is truncated, so details on the exact consolidation mechanism and experimental results are not available in the source. Still, the approach targets a central bottleneck in long-form video generation, where scaling to longer sequences is currently limited by memory and attention complexity.