FastTalk: 400 ms voice interaction

A spoken-dialogue system that starts answering within a fraction of a second and can be interrupted mid-sentence — by splitting every reply between a fast reflexive model and a slower context-aware one.

AAIC built the engine; the interface in these recordings is a technical demonstrator of it, not a production UI.

Fig. 1Two exchanges. First: a fast reply — the person asks, the bot answers almost at once, the fast-model opener leading and the context model streaming in behind it. Second: the bot begins a long answer, the person talks over it, and the bot cuts off within a fraction of a second and re-plans. The oscilloscope waveform moves only for the human voice; the event log names each pipeline stage as it fires.

The latency gap

In speech, a pause that would pass unnoticed in text reads as confusion or a dropped call. A conventional voice pipeline runs its stages in series — endpointing, transcription, generation, synthesis — and that sum of latencies is what the listener hears as silence. FastTalk treats the gap as the central design problem and attacks it at three points: fast endpointing via Silero VAD (tens of milliseconds), streaming TTS that plays the first audio before the reply is fully generated, and the split described below.

Simple and hard

Not every word of a spoken answer requires the full context of the conversation. The opening of a reply is often reflexive — a discourse marker, an acknowledgement — and carries little information. The substance follows. FastTalk formalises this distinction and assigns the two parts to different models:

“Well, I don’t know—frankly, I find the question kind of beautiful.”

Easy part — small fast model — plays immediately on turn end Hard part — context-aware model — streams in while the opener plays

Because the opener is generic and cheap, it is available in roughly the time it takes to detect the end of the turn. That opener then buys the slower model the length of a spoken phrase to compose an answer that actually depends on what was said. The two are stitched into one continuous utterance; what a listener hears is a single voice that started replying immediately.

Interruption

Speed alone does not make a conversation feel natural; the ability to break in does. The same voice-activity detector that handles endpointing also registers speech that overlaps the bot. When a barge-in is detected, the current reply is cut off — audio stops, streaming generation is torn down — and the system returns to listening. The interruption is treated as the next turn, not as noise, and the bot re-plans around it. Without this, the exchange reverts to the slow half-duplex rhythm of a walkie-talkie.

The demonstrator is AAIC’s own work. FastTalk is in production use; the application is not shown here.