Long-video understanding with multimodal language models is held back by the cost of decoding dense RGB frames, the growth of visual tokens with frame count, and weak motion perception when only sparse keyframes are sampled. HIMMEL tackles all three by routing a small set of semantic anchor frames through the expensive host vision transformer, while a lightweight tri-stream adapter handles the denser inter-frame intervals using motion-vector maps, residual maps, and an I-frame context branch.

These motion tokens are aligned to the frozen visual backbone in a dedicated contrastive stage. The authors report that an InfoNCE objective outperforms MSE regression by 1.5 percentage points. On Video-MME, HIMMEL reaches 63.5% accuracy, beating a dense 32-frame baseline by 2.3 points while using 3.6 times fewer context tokens and cutting end-to-end latency from 2.75 seconds to 1.03 seconds per question. With a stronger Qwen3-VL-8B host, accuracy rises to 64.9% on a single NVIDIA H20 GPU.

Ablations across stream composition, motion-encoder family, fusion mode, alignment objective, anchor count, LoRA rank, and video duration confirm that the full tri-stream design drives the gains, and that the benefit increases with video length. The paper is accepted to NeurIPS 2026 Main.