# Concepts (/concepts)

## The pieces [#the-pieces]

<PipelineDiagram />

| Piece           | What it does                                                                                                      | Lives in            |
| --------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------- |
| **Endpoint**    | Speaks a wire protocol (OpenAI Realtime, raw PCM, your own) and turns it into harness calls.                      | `cuelo.endpoint`    |
| **Harness**     | Opens one `Session` per call, builds a fresh `Pipeline` for it, records events and audio.                         | `cuelo.runtime`     |
| **STT**         | Streaming speech-to-text. Audio goes in continuously and interim and final transcripts come out.                  | `cuelo.stt`         |
| **Controllers** | Turn-taking models. They listen to user audio, transcripts and (duplex ones) the agent's audio, and emit signals. | `cuelo.controllers` |
| **Policy**      | Turns signals into actions: respond, stop talking, keep talking, play a backchannel.                              | `cuelo.policies`    |
| **LLM**         | The talker. It streams the reply text and can call tools or consult the thinker.                                  | `cuelo.llms`        |
| **TTS**         | Streams audio for the reply as text arrives.                                                                      | `cuelo.tts`         |
| **Store**       | SQLite, Postgres or MySQL: sessions, turns, events, summaries. Recordings go to disk.                             | `cuelo.database`    |

Each piece is a **slot** with a small Python interface. Providers implement it,
and you can swap any of them in config or in code. See
[Python API](/reference/python).

## The life of a turn [#the-life-of-a-turn]

1. **Audio in.** The client streams PCM16 at 24 kHz. The harness resamples it for
   each consumer (16 kHz for most STT and controllers) and fans it out.
2. **Transcribe.** The STT streams interim results (`stt.partial`) and finals
   (`stt.final`).
3. **Decide.** Every 20 to 80 ms each controller scores the audio. When the
   end-of-turn model is confident the user is done, it emits `turn_end`. The
   policy then asks the STT to **finalize**, which flushes the transcript for
   the turn right away instead of waiting for the provider's own endpointing.
4. **Think.** The LLM gets the system prompt, the conversation and the tools.
   Text streams back as `llm.delta`. Tool calls run (Python, HTTP or forwarded
   to the client) and the model continues with their results.
5. **Speak.** Text streams into the TTS. Streaming providers take tokens as they
   arrive. Non-streaming ones get one sentence at a time. Agent audio streams
   back to the client.
6. **Listen while speaking.** While the agent talks, the barge-in detector tells
   a real interruption ("wait, no") from a backchannel ("yeah"). On an
   interruption the harness cancels LLM and TTS, emits `agent.interrupted`,
   and the client drops any audio it hasn't played.
7. **Backchannel.** While the user talks, the backchannel model can fire. The
   agent then plays a short "mhm" from a cache of pre-synthesized phrases in its
   own voice (`assets/phrases/`), so a backchannel costs no TTS round trip.

Every step emits an [event](/observe/events). Events are stored, streamed
to the dashboard and passed to your `on_event` hook.

## Where latency goes [#where-latency-goes]

The dashboard splits each agent turn into stages. The number that matters is
**TTFB**: from the moment the user stopped speaking to the first agent audio.

<TurnTimeline />

| Stage      | Meaning                                                  | Rough range           |
| ---------- | -------------------------------------------------------- | --------------------- |
| `vad_tail` | Silence the end-of-turn model needed before it was sure. | 100–200 ms            |
| `finalize` | STT final transcript after the finalize request.         | 50–150 ms             |
| `llm_ttft` | LLM time to first token.                                 | 150–400 ms (Cerebras) |
| `tts_ttfa` | TTS time to first audio for the first sentence.          | 100–250 ms            |
| `ttfb`     | All of the above, from user stop to first agent audio.   | 500–900 ms            |

The ranges are a rough guide. They depend on providers, region and prompt
length, so measure your own in the dashboard. Tool calls and the thinker add
their own time, shown as `tool`. Use a filler
("let me check") so the caller isn't left in silence. See
[Thinker](/build/thinker).

## Concurrency [#concurrency]

The engine is asyncio. Controllers are synchronous model code, and the harness
runs each one on its own worker thread. One process handles many calls. The
heavy part is the turn-taking models: the MaAI stack costs about 23 ms of CPU
per 80 ms step per call. Run `cuelo machine check` to measure your hardware,
and see [Deploy](/ship/deploy) for workers.