Vision-language models (VLMs) typically understand video by encoding each frame into a set of visual tokens. For short clips this works well, but for hour-long videos, streaming feeds, or movies, the token sequence grows linearly and quickly overwhelms memory limits. That is the problem addressed in a new arXiv paper, "Elephant in the Fridge: Constant-Memory Frame Packing for Long Video Understanding."
The authors propose a frame-packing strategy that maintains constant memory usage regardless of how many frames are processed. Instead of letting the token sequence expand with each added frame, the method packs information into a fixed-size representation. This would allow VLMs to handle much longer videos without running out of memory or requiring excessive compute.
The paper is an announcement of the approach, and the abstract does not include experimental results or comparisons to existing methods. Still, the core idea directly targets a well-known bottleneck in long-video understanding, and if the packing method preserves relevant visual information, it could be a practical step toward practical hour-long video analysis with VLMs.