A cascaded voice-agent prototype for whatupwolf.com/lab.
The cascade. Four systems in a line: microphone → speech recognition →
Claude → speech synthesis. Each one has to finish before the next can start, so the latencies
add rather than overlap. That sum is what a person experiences as “how long until it answers”,
and it is what this page puts a number on.
- Speech-end → transcript. Deciding you have stopped, and finalising the text.
In push-to-talk this starts the instant you release the button — an explicit floor handoff, so
the moment is exact. In continuous mode it starts at the recogniser's own guess, and includes
the settle window this page waits before believing the guess.
- Transcript → first token. Request construction, network, and time-to-first-token
from the model. Streaming is on, so this is the first token, not the whole reply.
- First token → first audio. Buffering enough text to be worth saying, then the
synthesiser starting. This page flushes at the first sentence boundary; waiting for the full
reply would be simpler and much worse.
- Stop talking → start hearing. The sum, which is the only number the person
in the room can feel.
This is half-duplex, and that is the honest label. One party holds the floor at a
time. There is no overlap, no backchannel, no “mm-hm” while you are still talking, and the model
never hears your tone — only text. A native speech-to-speech model does not work this way; it
listens and speaks over the same continuous stream. Barge-in here is a crude approximation:
detected speech cancels playback mid-sentence. It is not the same thing.
Two turn-taking modes, on purpose. Push-to-talk removes the endpointing problem
by making you solve it. Continuous mode guesses, which is what a real product must do — and the
guess is where a good chunk of the perceived latency lives. Run the same sentence in both and
compare the first row.
Echo. In continuous mode the microphone can hear the synthesised voice. This
page ignores recogniser output for a short guard window after playback starts, but there is no
acoustic echo cancellation on that path, so a loud speaker can still make the loop talk to itself.
Headphones fix it. That, too, is part of the cascade's cost.
Why it matters. The τ-Voice benchmark (arXiv:2603.13686) ran matched agent tasks in text and
in voice: about 85% task completion in text collapses to 31–51% on clean
audio, and 26–38% under realistic noise and accents — with 79–90% of the
failures attributed to the agent's own behaviour rather than to transcription errors. Latency is the
part you can feel in ten seconds with this page; the capability gap is the part you cannot.
Bring-your-own-key. Anthropic-only, streamed straight from the browser. The key
stays in localStorage and is sent only to api.anthropic.com. Read the
privacy note for what happens to the audio, which is a separate question.