Retrieval-augmented generation (RAG) frequently reuses the same documents across many user requests, so caching the key-value (KV) states of those documents could save substantial computation. However, a cached document state is not simply reusable text: it is a causal state shaped by the specific context in which it was computed. This makes non-prefix reuse—where the new request does not start with the same tokens as the cached computation—difficult without changing the model's behavior.
The arXiv preprint introduces SAFE-KV, a method that repairs the causal suffix of a cached state to make it compatible with a new context. Rather than forcing a prefix match or discarding the cache, SAFE-KV adjusts the later portion of the cached representation so that the model's outputs remain consistent with what it would have produced without reuse.
The paper argues that this approach makes non-prefix KV-cache reuse feasible in RAG, potentially improving efficiency without sacrificing behavioral fidelity. As a preprint, the details of the repair mechanism and experimental validation are not fully described in the abstract, but the core idea is to treat cached states as causal objects that need careful repair rather than as interchangeable text blocks.