Speech recognition models have made impressive strides on general benchmarks, but they still trip over words that are acoustically ambiguous. In screen-based settings—where a user might refer to something visible on a display—visual context could provide the missing clue. However, existing evaluation suites do not capture this interaction well, according to a new arXiv preprint.

To address that gap, the paper introduces ScreenASR-Bench, a tiered benchmark designed specifically for speech recognition with visual context. The benchmark is structured to probe how models can leverage on-screen information to improve recognition of ambiguous words, rather than treating speech recognition as a purely audio-only task.

The significance lies in pushing ASR evaluation toward multimodal, real-world scenarios. If visual context proves consistently helpful, future models may need to integrate vision and language more tightly. The paper argues that current benchmarks are too limited to reveal these opportunities, making ScreenASR-Bench a step toward more realistic assessment.