Long-context serving runs out of key-value (KV) memory before the hardware reaches its useful throughput, according to the abstract. This bottleneck limits how much context can be processed efficiently.
The earlier HiSparse approach stores history in host memory and fetches only the selected entries, reducing the KV cache footprint. However, the abstract notes that it uses one cache budget at every load, and the sentence is cut off before explaining the consequence.
The new work, Adaptive DHiSparse, is described as a load-adaptive KV hierarchy. The title and the incomplete abstract suggest it varies the cache budget based on load, but the full details are not available in the provided text.