Key-value (KV) cache management is a growing bottleneck for scaling large language models to long contexts, as memory use grows linearly with sequence length. An arXiv paper introduces DropKV, a method designed to address this problem through a technique called decoupled residual-output perturbation.
The abstract notes that recent eviction methods have moved beyond simple attention-score-based approaches, and DropKV appears to continue that trend. The paper's title claims 'near-optimal KV-cache eviction,' suggesting the method offers a strong trade-off between memory savings and model performance.
The abstract is truncated, so further details of the method and evaluation are not available from the source. The paper is listed as arXiv:2610.22130v1.