Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 16 of 22

Turn detection

0 of 22 complete

Why it matters

Conversation has no explicit end-of-utterance marker. A pause can mean the user finished, is thinking, or is about to continue a list. A fixed silence timer guesses, and the guess is wrong for anyone who speaks slowly or pauses for effect. Getting it wrong is expensive in both directions: too eager and the system talks over the user, too patient and it feels unresponsive.

Mental model

Turn detection is knowing when a sentence is finished by the way a speaker trails off rather than by the grammar alone. The metaphor breaks because a model sees only energy and text, not the breath or the glance that tells a listener the turn is over.

How it works

The same pause, two verdictsInteractive

The interactive waveform loads with JavaScript. The text version below describes the same frame.

Figure. A synthetic 3.90-second request in two parts with a 700 ms thinking pause. A 500 ms timer replies at 1.60 s and overlaps the user for 500 ms; a 1.2 s timer that restarts on the resumed speech replies at 3.00 s, 400 ms after the turn; a semantic endpoint replies at 2.72 s, 120 ms after speech ends. Timings are authored and no detector was run.

What to notice. The pause is the user thinking, so a short timer cuts in and a long one arrives late: one silence signal cannot be both fast and safe. The figure shows the two costs side by side, plus the clause evidence a semantic detector uses — “book me a” is unfinished, “room for two” is complete. Measure reply delay and false cut-in rate on the same axis.

Text version

A synthetic 3.90-second request with two user spans, a transcript strip (“book me a” then “room for two”), and a hatched 700 ms pause. Alternative reply lanes start at 1.60 seconds for a 500 ms timer, 3.00 seconds for a 1.2 second timer that restarts on the resumed speech, and 2.72 seconds for a semantic endpoint. Steps show the unfinished clause and the pause, the 500 ms timer replying mid-thought with a 500 ms overlap, the restarted 1.2 s timer replying 400 ms late, and the semantic reply 120 ms after speech ends. A slider scrubs the playhead.

  1. Start from a silence threshold and measure the distribution of pause lengths in real recordings, not in a script.
  2. Add a semantic signal, such as whether the transcript ends in a completed clause or a filler.
  3. Combine signals with an explicit rule, and log which signal fired for each decision.
  4. Tune separately for languages and speaking styles, since pause norms differ.
  5. Handle barge-in with its own detector, so an interruption stops playback quickly even if the turn is not over.
  6. Evaluate with time-to-response and false-cut-in rate together, because improving one degrades the other.

Trade-offs

ChoiceGood forCostFails when
Short silence thresholdSnappy responsesCuts off slow speakersUsers think before continuing
Long silence thresholdFewer interruptionsFeels sluggishThe product is a fast assistant
Semantic end-of-turn modelHandles thinking pausesLatency and training dataThe transcript is unreliable
Push-to-talkDeterministicNot hands-freeThe product is a phone call
Trade-off mapInteractive

Good for

Snappy responses

Cost

Cuts off slow speakers

Fails when

Users think before continuing

  1. Users think before continuing
  2. The product is a fast assistant
  3. The transcript is unreliable
  4. The product is a phone call

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Short silence threshold is good for Snappy responses, costs Cuts off slow speakers, and fails when Users think before continuing. Long silence threshold is good for Fewer interruptions, costs Feels sluggish, and fails when The product is a fast assistant. Semantic end-of-turn model is good for Handles thinking pauses, costs Latency and training data, and fails when The transcript is unreliable. Push-to-talk is good for Deterministic, costs Not hands-free, and fails when The product is a phone call

Failure modes

  • False cut-in. The system responds during a pause and the user talks over it. Debug: log pause durations before every cut-in and compare with the threshold distribution.
  • Threshold tuned on scripted audio. Real users pause longer and every turn times out. Debug: replay production recordings and plot the pause distribution against the configured threshold.
  • Transcript lag. The semantic detector decides on text that arrives after the audio, so the decision is late. Debug: measure the delay between audio end and transcript availability per stage.

In production

Track false cut-in rate and median response delay on the same dashboard, since they trade against each other. Re-tune when the user population or the capture device changes.

Common misconceptions

People think: A fixed silence timeout is good enough.

Actually: It fails on exactly the users who pause to think, which includes most people in a difficult conversation.

People think: Turn detection is a speech recognition problem.

Actually: Recognition gives words; turn end also needs timing and semantics.

Interview angle

Treat turn end as a decision under uncertainty with a cost on each side, and describe how you would measure both. A strong answer combines timing with a semantic signal. Weak answers propose a silence threshold and never mention false cut-ins or how the value was chosen.

Sources