Voice agents built from cascaded components must decide at every silence whether the user has finished speaking. Each related decision—whether the user will continue, whether an interruption is a backchannel, or whether a sound is speech at all—typically requires a separate detector or model call. The paper reports that open detectors on the eot-bench cut users off in more than a quarter of mid-turn pauses, and attributes this mismatch largely to training on segmented utterances rather than real conversational pauses.

VoxTurn is a 0.8B speech LLM that reframes these decisions as multiple-choice questions. It scores the hidden state of each option and uses a block-isolating attention mask, allowing it to answer any set of turn-taking questions in one forward pass, exactly as if each were asked alone. The model is trained on real conversational pauses with labels derived from their timing, and it is re-queried while a silence lasts.

In English, VoxTurn reaches 11.5% false cut-offs at a 300 ms latency budget, comparable to two commercial detectors. Its 14-language mean is 19.9%, the lowest among open models, at 8.3 ms per decision. In the Pipecat and LiveKit frameworks it cuts off 17% and 9% of turns respectively, and it ranks fifth of 24 in TurnBench end-of-turn detection without any TurnBench training data.