Real-time interaction demands that an agent track a changing environment and respond within time limits. According to a new arXiv paper, current evaluations of real-time streaming models mostly measure conversational quality and streaming perception, leaving other aspects of interactive problem-solving unexamined.
To address this, the authors introduce Realtime-Bench, a benchmark designed to assess frontier models on real-time interactive problems. The abstract does not yet disclose results or the specific tasks included, but the stated aim is to test whether models can perceive evolving situations and act under temporal constraints.