On-screen text in videos can do two very different jobs: it can neutrally describe what is happening, or it can frame the footage to push a particular interpretation. According to a new arXiv preprint, existing video benchmarks do not distinguish between these roles. They test whether a model can read the text, not whether it should believe it.
The paper, which introduces a study called VisualTrust, argues that this conflation is a problem. A model that simply transcribes an overlaid claim may look capable even when it has no idea whether that claim is reliable. The authors propose that video reasoning should account for claim framing—treating on-screen text as something to be evaluated, not just consumed.
The abstract is brief and does not include full experimental details, so the specific methods and results are not yet available. But the core point is clear: as video models become more capable, they will need to separate factual description from persuasive framing, and benchmarks should reflect that challenge.