The abstract of a new arXiv paper describes an "impossible triangle" in long-context large language models: designers must juggle KV cache size, training and prefill FLOPs, and model quality. The authors note that grouped-query attention (GQA) achieves strong quality but at the cost of a high KV cache footprint, leaving room for alternative mechanisms.
The paper proposes Virtual Head Attention (VHA) as a design that addresses this triangle. However, the abstract provided is truncated, so specific details of how VHA works, its reported trade-offs, or experimental results are not available in the source. The claim is limited to the mechanism's stated intent.
As with many preprints, the abstract frames the contribution as a conceptual and architectural response to a known bottleneck. Without the full text, it is not possible to verify the extent to which VHA actually resolves the trade-off or how it compares with other approaches beyond GQA.