Tool-using voice agents typically wait until a user stops speaking before they start calling tools, even when the actionable part of the request arrives earlier. That leaves the user staring at a spinner after their final word.
The new arXiv preprint, 'Heartbeat Thinking,' argues that speculative approaches can begin work during speech, but existing strategies are constrained. Chunk-bounded methods only act at fixed segment boundaries, while interrupt-on-update policies restart or revise work whenever new audio arrives—neither fully removes the end-of-speech delay.
The paper proposes 'Heartbeat Thinking' as a way to continuously reason and execute tools while the user is still talking. By aligning speculative work with the rhythm of incoming speech, the approach aims to have results ready by the time the utterance ends. The preprint is preliminary, and the abstract does not include experimental results, so the practical gains remain to be seen.