# Models (/reference/models) LLMs and speech run at providers. The turn-taking models run **locally**, on your CPU, inside the engine. This catalog (`schema/models.json`) lists every model Cuelo uses or has evaluated, with its license, size and cost. ```bash cuelo models # integrated models cuelo models --all # include ones still being evaluated cuelo models --role turn # vad, turn, text_turn, barge, agent_backchannel, duplex, component cuelo models info turn-1-mini cuelo models pull --all # download every integrated and experimental model (sha256-verified) cuelo models pull maai-turn ``` Weights download on first use into `$CUELO_MODELS` (default `~/.cache/cuelo/models`), pinned to a revision and checked by sha256. **Status:** - **integrated**: a Cuelo controller uses it. - **experimental**: runs, but its controller isn't a default. - **evaluating**: being tested. - **evaluated**: tested and not integrated. - **unsupported**: can't run locally, or its license blocks it. **Roles:** - `vad`: speech presence. - `turn`: end of turn from audio. - `text_turn`: end of turn from the transcript. - `barge`: interruption versus user backchannel. - `agent_backchannel`: when the agent says "mhm". - `duplex`: models that hear both sides. - `component`: shared parts, such as encoders. Check a model's license before you ship it. Most are permissive (MIT, Apache, BSD). The Mimi encoder used by MaAI is CC-BY-4.0 and requires attribution. ## Integrated [#integrated] ### silero-vad [#silero-vad] Speech presence, 20 ms frames. Feeds every controller; not end-of-turn on its own. | | | | ---------- | ----------------------------------------------------------------- | | Role | `vad` | | Config | `always on (cuelo[silero])` | | Source | [snakers4/silero-vad](https://huggingface.co/snakers4/silero-vad) | | Languages | any | | Parameters | 2M | | CPU | ~0.1 ms / 20 ms | | License | MIT | | Download | bundled with the package | ### smart-turn-v3 [#smart-turn-v3] Semantic end of turn from audio: judges at each pause whether the speaker sounds done. | | | | ---------- | --------------------------------------------------------------------------- | | Role | `turn` | | Config | `turns.controller: smart-turn` | | Source | [pipecat-ai/smart-turn-v3](https://huggingface.co/pipecat-ai/smart-turn-v3) | | Languages | 23 languages incl. English, Hindi | | Parameters | 8M | | CPU | ~40 ms per check | | License | BSD-2-Clause | | Download | 9 MB | ### turn-1-mini [#turn-1-mini] Streaming end of turn from audio, a score every 20 ms. Fastest endpointing in our tests. | | | | ---------- | --------------------------------------------------------------- | | Role | `turn` | | Config | `turns.controller: turn-1-mini` | | Source | [p99lab/turn-1-mini](https://huggingface.co/p99lab/turn-1-mini) | | Languages | English | | Parameters | 5.4M | | CPU | ~1.6 ms / 160 ms step | | License | Apache-2.0 (credit: turn-1-mini by p99lab) | | Download | 22 MB | ### backchannel-detector [#backchannel-detector] While the agent talks: user backchannel ("yeah") vs real interruption. | | | | ---------- | ----------------------------------------------------------------------------------------- | | Role | `barge` | | Config | `turns.barge_in_detector: backchannel-detector` | | Source | [pipecat-ai/backchannel-detector](https://huggingface.co/pipecat-ai/backchannel-detector) | | Languages | English | | Parameters | 0.7M | | CPU | ~0.3 ms / 40 ms | | License | BSD-2-Clause | | Download | 4 MB | ### maai-turn [#maai-turn] Voice Activity Projection from user + agent audio: end of turn and real interruptions vs backchannels. | | | | --------- | ------------------------------------------------------------------------- | | Role | `turn` | | Config | `turns.controller: maai (pip extra cuelo[maai], PyTorch)` | | Source | [maai-kyoto/vap_en_kyoto](https://huggingface.co/maai-kyoto/vap_en_kyoto) | | Languages | English | | CPU | ~15 ms / 80 ms step (PyTorch) | | License | MIT (code and weights; Kyoto English model) | | Requires | ['maai-mimi-encoder'] | | Download | 15 MB | Holds all 3 real mid-sentence cuts; complete turns end 380-980 ms after speech; interrupts caught 80-160 ms after the user starts, backchannels let through. The non-Kyoto English VAP weights are CC-BY-NC-ND (not used). It also ends the turn in 0.6 s pauses between complete sentences of a longer story; pair it with maai-backchannel so the agent says "mhm" there. ### maai-backchannel [#maai-backchannel] When the agent should say "mhm": VAP-BC timing from both audio channels. | | | | ---------- | ------------------------------------------------------------------- | | Role | `agent_backchannel` | | Config | `turns.agent_backchannel: maai (pip extra cuelo[maai], PyTorch)` | | Source | [maai-kyoto/vap_bc_en](https://huggingface.co/maai-kyoto/vap_bc_en) | | Languages | English | | Parameters | ~100M incl. encoder | | CPU | ~15 ms / 80 ms step (PyTorch) | | License | MIT (code and weights) | | Requires | ['maai-mimi-encoder'] | | Download | 403 MB | Kit monologue at p>=0.5: 4/4 pauses hit, 1 extra trigger (after the turn ended). Vendored MaAI matches upstream within 5e-4. ### maai-mimi-encoder [#maai-mimi-encoder] Streaming Mimi audio encoder (ONNX) shared by the MaAI models. | | | | ---------- | ----------------------------------------------------------------------------------------- | | Role | `component` | | Config | `pulled in by maai-backchannel / maai-turn` | | Source | [maai-kyoto/continuous-mimi-onnx](https://huggingface.co/maai-kyoto/continuous-mimi-onnx) | | Parameters | ~80M | | License | CC-BY-4.0 (Kyutai Mimi; attribution) | | Download | 156 MB | ## Experimental [#experimental] ### backchannel-cue [#backchannel-cue] Picks which reaction the agent says (61 forms: mhm, oh no, nice, ...) from the live transcript. | | | | ---------- | ------------------------------------------------------------------------------- | | Role | `agent_backchannel` | | Source | [KoljaB/backchannel-cue-32m](https://huggingface.co/KoljaB/backchannel-cue-32m) | | Languages | English | | Parameters | 32M | | CPU | 3-11 ms per call | | License | MIT (teacher labels partly from GPT models; check before shipping) | | Download | 129 MB | Phrase choice is good; its own gate fires too often (10/10 mid-sentence at the card's 0.09). Use it for wording, with a timing model deciding when. ## Evaluating [#evaluating] ### anyreach-semantic-turn [#anyreach-semantic-turn] Text-only: start speaking / keep listening / start listening / keep speaking. | | | | ---------- | ------------------------------------------------------------------------------------------- | | Role | `text_turn` | | Source | [anyreach-ai/semantic-turn-taking](https://huggingface.co/anyreach-ai/semantic-turn-taking) | | Parameters | 0.5B | | License | Apache-2.0 | License-clean text detector; weights (473 MB) not yet downloaded here. ## Evaluated [#evaluated] ### dualturn-endpointing [#dualturn-endpointing] Dual-channel audio end of turn with future speech-activity signals (Mimi features). | | | | ------- | ------------------------------------------------------------------------------------------- | | Role | `duplex` | | Source | [anyreach-ai/dualturn-endpointing](https://huggingface.co/anyreach-ai/dualturn-endpointing) | | CPU | ~26 ms / 80 ms step | | License | Apache-2.0 | Fast on complete turns (240-280 ms) but also ends turns on real mid-sentence cuts (460-680 ms); no barge/backchannel head. Not integrated: turn-1-mini / maai-turn are more selective. ### dualturn-0.5b [#dualturn-05b] Dual-channel floor control: end of turn, hold, backchannel, yield. | | | | ---------- | ------------------------------------------------------------------------------------------------------- | | Role | `duplex` | | Source | [anyreach-ai/dualturn-qwen2.5-mimi-0.5B](https://huggingface.co/anyreach-ai/dualturn-qwen2.5-mimi-0.5B) | | Parameters | 0.5B | | License | Apache-2.0 | Not evaluated end to end: 6.6 GB weights; heavier than the controllers above. ## Unsupported [#unsupported] ### tyger [#tyger] Two-stream controller: listen / hold / speak / backchannel, interruption and endpoint scores. | | | | ---------- | ------------------------------------------------------------------------------ | | Role | `duplex` | | Source | [nineninesix/tyger](https://huggingface.co/nineninesix/tyger) | | Parameters | 652M | | CPU | 775 ms per 320 ms step on CPU (2.4x slower than realtime); ~95 ms on Apple MPS | | License | OpenMDW-1.1 | Can't keep up on CPU. Plausible on GPU/MPS later; OpenMDW-1.1 allows commercial use with notices. ### easy-turn [#easy-turn] Audio + linguistic turn state: complete, incomplete, backchannel, wait. | | | | ------- | --------------------------------------------------------------- | | Role | `turn` | | Source | [ASLP-lab/Easy-Turn](https://huggingface.co/ASLP-lab/Easy-Turn) | | CPU | 0.7-1.7 s per decision on CPU | | License | Apache-2.0 | Not a streaming controller, trained mostly on Mandarin, no agent channel. ### livekit-turn-detector [#livekit-turn-detector] End of turn from the STT transcript. | | | | --------- | --------------------------------------------------------------------- | | Role | `text_turn` | | Source | [livekit/turn-detector](https://huggingface.co/livekit/turn-detector) | | Languages | multilingual | | License | LiveKit Model License: LiveKit Agents only | License forbids standalone use or use with other frameworks; also deprecated by LiveKit. ## Adding a model [#adding-a-model] To integrate a new turn-taking model, add a catalog entry and a controller. See [Add a turn-taking model](/contributing/controllers).