Cross-model key-value (KV) cache reuse is a technique that lets one model borrow prefix states already computed by a related model, reducing the prefill work needed for new queries. The approach promises faster inference, and prior work has largely optimized it for latency and output quality.
According to the arXiv abstract, however, this optimization focus leaves a gap: safety compatibility. When a receiver model reuses a cache generated by a sender model, the resulting behavior may diverge from the receiver's own safety alignment, potentially producing outputs that the receiver alone would not generate.
The paper, titled "To Cache or Not to Cache," investigates these incompatibilities and proposes ways to mitigate them. The abstract is truncated in the source, so the full details of the mitigation methods are not available, but the stated goal is to make cross-model cache reuse both efficient and safe.