Add a turn-taking model

Turn-taking models run locally inside the engine. Integrating one means three things:

1. a catalog entry in schema/models.json that pins the weights,
2. a controller in engine/src/cuelo/controllers/ that runs the model and emits signals,
3. optionally, config wiring, so turns: in config.cuelo can select it.

1. Catalog entryCopy Anchor Link

schema/models.json is the shared list of every model Cuelo knows about. The CLI (cuelo models), the engine (weights.py) and the Models page all read it.

{
  "name": "acme-turn",
  "role": "turn",
  "description": "Streaming end of turn from audio, a score every 40 ms.",
  "status": "evaluating",
  "config": "turns.controller: acme-turn",
  "repo": "acme/acme-turn",
  "revision": "3f9c2e…full commit sha…",
  "license": "Apache-2.0",
  "languages": "English, Spanish",
  "params": "8M",
  "cpu": "~2 ms / 40 ms step",
  "files": [{ "path": "onnx/acme-turn.onnx", "size": 32104448, "sha256": "…" }]
}

• Pin a commit, not a branch. revision must be a full Hugging Face commit sha, and every file needs its exact size and sha256. Cuelo refuses a file that doesn't match, so a changed upstream file can't be loaded silently.
• role is one of vad, turn, text_turn, barge, agent_backchannel, duplex or component.
• status starts at evaluating. It becomes integrated once a controller ships and you have numbers from live calls.
• requires lists other catalog entries a model depends on. The MaAI models require maai-mimi-encoder, for example.
• Check the license. Weights are downloaded, never bundled, but users still need to know what they're running. Note attribution requirements in license.

Run make schema to copy it into the engine and CLI.

Get the size and hash with:

curl -sL https://huggingface.co/acme/acme-turn/resolve/<sha>/onnx/acme-turn.onnx -o m.onnx
stat -f%z m.onnx      # Linux: stat -c%s
shasum -a 256 m.onnx

2. The controllerCopy Anchor Link

A controller is synchronous code that consumes audio and transcripts and returns ControlEvents. The harness runs each controller on its own worker thread, feeds it in order, and resamples audio to its sample_rate.

"""Acme Turn: streaming end of turn from audio.

Scores every 40 ms; the turn ends when P(end) crosses `threshold` after at
least `min_silence_ms` of quiet, or after `max_silence_ms` regardless.

Model: https://huggingface.co/acme/acme-turn (Apache-2.0)
"""

from typing import ClassVar

import numpy as np

from cuelo.controllers.interface import ControlEvent, Controller, Signal
from cuelo.controllers.vad import VAD, SpeechTracker, default_vad
from cuelo.runtime.audio import Audio

STEP = 640  # 40 ms at 16 kHz


class AcmeTurnModel:
    """The ONNX model alone: audio in, probabilities out. Easy to fake in tests."""

    def __init__(self) -> None:
        from cuelo.controllers import weights

        self.session = weights.session(weights.weights("acme-turn"))   # downloads + verifies once
        self.reset()

    def reset(self) -> None:
        self._pending = np.zeros(0, np.float32)

    def push(self, audio: np.ndarray) -> np.ndarray:
        self._pending = np.concatenate([self._pending, audio])
        out = []
        while self._pending.size >= STEP:
            step, self._pending = self._pending[:STEP], self._pending[STEP:]
            (p,) = self.session.run(["p_end"], {"audio": step[None]})
            out.append(float(p.reshape(-1)[0]))
        return np.asarray(out, np.float32)


class AcmeTurnController(Controller):
    emits: ClassVar[frozenset[Signal]] = frozenset({Signal.TURN_END})
    sample_rate = 16_000

    def __init__(
        self,
        vad: VAD | None = None,
        *,
        model: AcmeTurnModel | None = None,
        threshold: float = 0.5,
        min_silence_ms: float = 80,
        max_silence_ms: float = 1500,
    ) -> None:
        super().__init__()
        self.model = model or AcmeTurnModel()
        self.tracker = SpeechTracker(vad or default_vad(), self.sample_rate, gap_ms=max_silence_ms)
        self.threshold = threshold
        self.min_silence_s = min_silence_ms / 1000
        self.t = 0.0           # seconds of user audio consumed
        self.fired = False

    def push_user(self, audio: Audio) -> list[ControlEvent]:
        samples = np.frombuffer(audio.pcm, np.int16).astype(np.float32) / 32768
        self.t += audio.duration
        ...  # update the speech tracker, run the model, and when the user is
        ...  # quiet long enough and P(end) >= threshold, return once:
        # return [ControlEvent(Signal.TURN_END, t=self.t, confidence=p, speech_end_t=end)]
        return []

    def reset(self) -> None:
        self.model.reset()
        self.t, self.fired = 0.0, False

The rules:

• Time is audio time. t is seconds of user audio consumed, never wall clock, so the same audio always gives the same events. That makes tests and replays deterministic.
• Declare emits. The harness fills gaps with other controllers. A controller that only does end of turn leaves interruptions to the barge-in detector.
• Duplex models set duplex = True and get push_agent(audio) with the agent's audio as it plays. Non-duplex controllers can read self.agent_speaking.
• Transcript models (text_turn) implement push_transcript().
• Fire once per turn. Return TURN_END once, then re-arm when the user speaks again. Set speech_end_t (when the user actually stopped) so latency metrics are right.
• Keep the model separate from the controller (AcmeTurnModel above), so tests can pass a fake model and don't need the weights.
• Mind the CPU. It runs on every audio frame of every call. Measure with cuelo-bench --controllers your.module:factory and put the per-step cost in the catalog's cpu field. Share heavy encoders across controllers in a call, the way MaaiCall shares one Mimi encoder pair between the MaAI controllers.

Register it in the schema's x-cuelo-providers.controller so tooling can find it:

"acme-turn": { "class": "cuelo.controllers.acme_turn.AcmeTurnController", "extra": "silero", "env": null }

Use silero as the extra for ONNX models (onnxruntime and numpy), and maai for PyTorch.

3. Config wiring (optional)Copy Anchor Link

At this point anyone can use the controller from src/turns.py:

from cuelo.controllers.acme_turn import AcmeTurnController
from cuelo.controllers.interface import Signal

def controllers(defaults):
    return [AcmeTurnController(), *[c for c in defaults if Signal.TURN_END not in c.emits]]

To make it a first-class choice (turns.controller: acme-turn):

1. Add acme-turn to the turns.controller enum in schema/config.cuelo.schema.json, with a sentence in its description. Then run make schema.
2. Map its config to constructor arguments in cli/internal/gen/plan.go (the switch t.Controller block). Pass min_silence, max_silence and threshold the same way turn-1-mini does.
3. Add a case to the default agent's CUELO_TURNS in engine/src/cuelo/agents/default.py.
4. Update the Go generator test expectations (cli/internal/gen/gen_test.go).

Tests and evaluationCopy Anchor Link

• Unit tests with a fake model: feed synthetic speech and silence and assert when TURN_END fires. See engine/tests/test_model_controllers.py.
• A weights test that loads the real model behind a skip when the weights aren't cached. See test_model_weights.py.
• Live calls. Before proposing integrated, run real conversations with cuelo start and compare against turn-1-mini in the dashboard's latency tab: vad_tail, premature turn ends and flagged turns. Include the numbers in the PR.