Streaming vision-language models (VLMs) must continuously integrate information from the past while staying responsive to current observations and user queries. A new arXiv paper, "Memory as Weights: Internalizing Long-Term History for Streaming Videos," addresses this challenge directly.
The abstract notes that existing memory mechanisms typically compress, retrieve, or cache. The paper's title suggests a different approach: internalizing long-term history into the model's weights, rather than keeping it in a separate memory store.
The available abstract does not provide implementation details, but the framing is significant. It proposes that memory could become a property of the model itself, potentially eliminating the need for explicit memory management during streaming inference.