Video multimodal large language models (MLLMs) have become adept at general and long-form video understanding, but their ability to catch brief, answer-critical visual evidence is less understood. A new benchmark called Moment-Video directly targets this gap by constructing questions around momentary actions or state transitions that occupy only a small fraction of a video. The benchmark includes 1,000 human-verified video-QA pairs spanning seven domains and 25 subcategories, with four task types: temporal occurrence, temporal counting, action description, and temporal reasoning.

When the authors evaluated 34 proprietary and open-source MLLMs, the results were stark. The best-performing model, Seed-2.0-Pro, achieved only 39.6% overall accuracy under the main settings, while most open-source models remained below 25%. These low scores suggest that broad video understanding benchmarks do not reliably indicate whether a model can identify and use temporally localized evidence.

Diagnostic analyses point to specific weaknesses. Denser frame sampling improved some models, but substantial errors remained even in an audited subset where sampled frames covered all annotated necessary event intervals. Performance also declined on longer videos, indicating additional temporal-localization challenges. The authors position Moment-Video as a diagnostic tool to guide the evaluation and development of video MLLMs that more reliably understand brief, answer-critical visual events.