Large language models need ways to acquire external knowledge beyond what they retain from pretraining. An arXiv paper titled "Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models" addresses this problem directly.
The abstract outlines two existing approaches with drawbacks. Continued pretraining risks catastrophic forgetting, while retrieval at inference time can add substantial overhead—though the sentence is cut off in the available text, so the exact nature of that overhead is not specified.
The paper's title indicates a proposed solution: a chunk-level parametric memory. Rather than relying on external retrieval, the model would store knowledge in parameters at the chunk level. The abstract does not provide further details on how this memory is trained or used.