Measure the speech model before picking it

tebis lets you drive coding agents running in tmux from Telegram. In April I added voice, so a spoken message reaches the agent as text.

The first decision was where speech recognition runs. whisper-cli was out because of its “300 ms model-load per call”, so tebis loads Whisper in-process. The plan expected the binary to grow by about 20 MB. It grew from 4.25 MB to 4.56 MB.

The plan also had cloud providers, Groq and OpenAI, behind an abstraction. They were dropped the same day they were planned. “Tebis ships one STT backend”, and it runs locally.

Then I measured. With base.en on an M4 Pro, q5_1 quantization took the model from 147 MB to 57 MB, the load time from 7.21 s to 2.30 s, and cold inference from 289 ms to 181 ms. Warm inference stayed at 45 to 46 ms. I couldn’t measure a quality cost. On silent audio the full-precision model wrote “you” and the quantized one wrote nothing.

The measurements also argued for a bigger model. The default became small.en, chosen for accented speech: 181 MB, 500 ms cold and 141 ms warm on the M4, and about 1.5 to 2 s on a Ryzen 5825U CPU. In the commit’s words, “Quality > size at this scale.” distil-whisper, large-v3-turbo and qwen3-asr were weighed and not chosen.

How it works now is in docs/voice.md.