Armenian AI Company

MeFirst: the many shapes of a good interruption

An interruption engine for voice agents. People overlap each other constantly — and most of the time it helps. This is a tour of the ways a second voice can come in well, and how an engine decides where and how to do it.

Fig. 1Ten good interruptions. Reactions, continuers, joint completions, a detour and back — each labelled by the feeling it carries. Each agent is one waveform, drawn from the real audio, with its words beneath.

The need

A voice agent that only speaks into a clear silence feels slow and lifeless. But the fix is not simply to jump in sooner. In real conversation people talk over each other all the time — a murmured mm-hm, a gasp at good news, the end of a sentence supplied by the other person — and almost all of it is welcome. Overlap is not a failure of turn-taking; it is a systematic part of how turn-taking works (Schegloff, 2000).

So the goal is not to avoid interrupting. It is to interrupt the way a person does: in the right place, in the right manner, carrying the right feeling. MeFirst reads a turn as it is produced and decides, moment by moment, where a second voice can come in — and how.

The floor opens only at certain points

The founding account of conversation (Sacks, Schegloff and Jefferson, 1974) describes turns as built from units — a phrase, a clause, a whole sentence — each of which projects its own end. The clearest openings are those completion points, called transition-relevance places. A seam is marked by grammatical, intonational and pragmatic completion arriving together (Ford and Thompson, 1996), and the prosody is the deciding cue (Bögels and Torreira, 2015): a sentence that reads as finished on the page is not yet finished to the ear.

Timing is tight — the typical gap between turns sits near two hundred milliseconds (Stivers and colleagues, 2009), shorter than it takes to plan a reply — so people predict the seam and land on it (Levinson and Torreira, 2015). The openings are signalled by measurable cues — final intonation, phrase-final lengthening, a drop in pitch and loudness (Gravano and Hirschberg, 2009). MeFirst reads the same cues. But the interesting cases are the ones that do not wait for a seam at all: the overlaps that land inside a turn and still help. What follows is a tour of those.

Reacting

The most human interruptions are feelings arriving on time. A reaction held back to the next silence is a reaction that has missed its moment.

Fig. 2An empathy dip. A soft oh no… laid over the bad news as it lands. B meets the feeling without taking the turn away.
Fig. 3A joy burst. The good news barely lands before B erupts over it. The overlap is the delight — waiting would kill it.
Fig. 4Shared laughter. B’s laugh breaks in right on the punchline. Shared laughter overlaps by nature; it is the point, not a fault.

Listening out loud

Some overlaps take nothing at all. They are the sounds of attention — brief, quiet, placed just after a stressed word — and they keep the speaker going. This is the “back channel” (Yngve, 1970): short messages a listener sends without ever taking the turn.

Fig. 5A warm continuer. Two soft tokens over the speaker, after the accents. B signals it is listening and hands the floor straight back.
Fig. 6Thinking it through together. A gropes for the word with fillers — um… mmm — and B offers it just as A lands on it. The gap is filled jointly.

Building the turn together

Here the two voices genuinely talk at once — converging on the same words at the same instant. This is co-construction: one speaker completes a sentence the other has begun, so the turn is built by both rather than handed from one to the other (Lerner, 1991).

Fig. 7Finishing the thought. B says the word A is reaching for at the very instant A says it. The two land the ending together.
Fig. 8Word-search rescue. A stalls on a name; B supplies it and A seizes it in overlap. The word is found by both at once.
Fig. 9Telling it together. Both know the story, so B jumps onto the next beat as A reaches it — they narrate the punchline together.

Agreeing, and going off-script

Two more shapes: an agreement too strong to wait, and a quick loop off the topic and back onto it.

Fig. 10Emphatic agreement. B piles onto A’s conclusion instead of waiting for it — agreement so strong it cannot hold back to the seam.
Fig. 11A detour, and back. B slips in a side question; A answers it in a beat — anyway — and picks the story back up. A loop out and back.

A verdict for every point in the turn

Underneath all of these is one running task. As the language model produces a turn, token by token, MeFirst assigns each point one of three verdicts:

  • Hold — mid-unit. Coming in here would collide badly.
  • Support — a spot, usually after a stressed word, where a brief overlap (a continuer, a reaction, a supplied word) lands well without taking the turn.
  • Take — a seam. Grammar and intonation complete; the floor is open.

Those verdicts drive the mix of the two voices, and the manner of the incoming one — loud and fast for a joy burst, low and slow for an empathy dip. The decision is remade at every point, because the answer changes at every point.

Fig. 12The running verdict. One turn streaming out, each point tagged hold, support or take as the words arrive — the labels that gate the second voice.

Why the model has to be small

The verdict is only useful if it is ready before the point it judges. The whole value of the engine is in beating that two-hundred-millisecond gap, on every token, without pause — which rules out putting a second large model in the loop. The decision has to be made in the hot path, in a fixed and tiny budget.

This is the same shape as Lemon. A large model is used once, offline, as a teacher: it marks completion and overlap points on a great deal of speech. A small classifier learns from it, and only the small classifier ships. In the running system it reads the stream and returns a verdict in well under the budget, and in the same time on every token, because the path does not branch off to anything slow. The expensive model trains the cheap one; the cheap one decides.

The demonstrator uses two synthetic voices reading scripted lines, because the overlaps are easiest to hear when two clear voices meet. The engine’s task is the same wherever the speech comes from: for each point in what is being said, decide whether a second voice can come in — and if so, how. MeFirst comes in when a person would, the way a person would.

References