Large language models perform better when given relevant context, but that improvement often vanishes when the context is removed. A new paper on arXiv, "Cortex: Consolidating Context into Parametric Memory," takes inspiration from neuroscience to address this. The authors frame the problem as Memory Consolidation (MC): internalizing the temporary gains from context so they persist in the model's weights.

Biological systems consolidate short-term experiences into long-term memory, and the paper applies a similar idea to LLMs. Rather than relying on external context at inference time, Cortex aims to update the model's parametric memory so that the benefits of context remain available later. The abstract does not provide experimental details, but the proposed direction is clear: make context-dependent improvements permanent.