Three new attacks on the KV cache bottleneck

All three papers agree on a central problem: the key-value cache is now a major constraint on LLM deployment. As sequence length and batch size grow, the cache consumes more memory and calls for more bandwidth. Each paper, however, attacks the bottleneck at a different layer of the stack.

MeshKV looks at hardware. It argues that prior compression and DRAM-placement schemes still route traffic through centralized memory paths, which become a choke point on tiled accelerators. Its answer is a network-on-chip KV cache fabric designed to handle irregular cache movement directly on the chip.

D-Quant and DeepSeek-V4.1-Flash stay closer to software. D-Quant introduces driftable entropy coding for KV cache quantization, aiming to ease both capacity and bandwidth demands. DeepSeek's work instead frames the problem around long-horizon agents, noting that prefill remains expensive even after earlier advances in long-context computation; the paper sets out to push compression further for that regime.

The three approaches are complementary rather than competing, though they differ in emphasis: MeshKV focuses on the accelerator's data movement, D-Quant on the encoding of values, and DeepSeek on workload-level efficiency for agent-heavy inference.