The standard way to give an LLM context is to stuff it into the input window, but that gets expensive as context grows. A new arXiv preprint (2610.17686) proposes an alternative: updating the model's weights at test time to bake the context in.

The abstract argues that longer contexts let models condition on more information, but inference costs become prohibitive. Test-time updates are presented as an emerging alternative, though the provided abstract cuts off before explaining the mechanism or results.

The paper's title asks "What Makes It Context?" suggesting the authors are probing what distinguishes context encoded in weights from ordinary parameter updates. But without the full text, the answer remains unclear.