Autoregressive video diffusion models generate frames sequentially, and as they do, the attention mechanism's key-value (KV) cache grows. This makes memory a key bottleneck for long video generation. A new paper on arXiv introduces CORAL-KV, a method designed to compactly represent this cache.
The approach uses cross-layer prediction and residual coding. It also draws on clustering-based quantization, which has recently emerged as an effective technique for compressing KV caches. By predicting cache contents across layers and coding only the residuals, CORAL-KV aims to reduce the memory overhead without discarding the information the model needs.
The significance is that shrinking the KV cache could allow autoregressive video models to generate longer sequences without a proportional increase in memory use. The abstract does not include experimental results or comparisons, so the practical gains remain to be detailed in the full paper.