Full-duplex speech-to-speech models can listen and speak simultaneously, promising natural, low-latency voice agents. But that capability raises a question: when should the model actually take the floor? Three new arXiv papers address this from different angles, and together they highlight that the bottleneck is no longer raw audio generation but conversational judgment.
The first paper (arXiv:2609.19334) proposes a frontend-backend architecture for tool calls, separating the speech model's conversational layer from a backend that executes external functions. This lets the model handle tasks like booking or querying while maintaining full-duplex interaction. The second (arXiv:2609.19596) frames turn-taking as a decision problem: human listeners speak when addressed or when the speaker stops, but also self-select to correct or interject. The authors argue that models should speak when asked, not whenever they have something to say, and propose a mechanism to learn that timing. The third (arXiv:2609.13445) identifies a specific failure: models like Moshi and PersonaPlex can begin speaking inappropriately during prolonged user silence. The paper offers a causal analysis and mitigation strategy for these spurious onsets.
All three agree that full-duplex models need better control over when they speak, but they differ in focus. The tool-call paper is about integrating external actions, while the other two are about turn-taking and silence handling. The turn-taking paper takes a high-level decision view; the silence paper drills into a concrete failure mode. Together, they suggest that the next step for full-duplex assistants is not just faster speech, but smarter conversational policy.