arbeiten-mit-ki ki-werkzeuge workflows coding-cli

When whisper.cpp Swallows the Start of a Sentence — and the Glossary Is to Blame

Martin Rau · 7. Juni 2026 · 5 Min. Lesezeit

With some bugs you spend days looking in the wrong place, and in the end it turns out the very feature meant to solve the problem was the one causing it. This was one of those. Our local speech recognition in the boostN CLI runs on whisper.cpp, and to help the model pick up our domain terms — Claude, RAG, MCP, boostN, DEVI Runner and so on — we feed it a list of keywords. That list is passed to whisper as an initial_prompt. Essentially, it’s a little glossary that runs ahead of the actual transcription.

And that very glossary kept eating half my sentence at the start. The funny part: it happened precisely on the sentences the glossary was meant for in the first place.

The symptom: sometimes great, sometimes chopped off

The transcription ran fine for a long time. But then there were recordings where a whole chunk at the front was missing. Not a single word — the entire start of the sentence. The output would begin mid-thought, somewhere in the middle.

A concrete example from the log: a 24-second recording into which I’d spoken a whole list of terms. What came out was just 11 words — and only starting from “RAG Update, Promptflow Studio, Superbase, Repository, Wave, Tickets …”. The entire front part was simply gone.

To give a sense of scale: across 128 real transcriptions my normal throughput averages 22.6 words per second, ranging from about 2.4 to 82.2 depending on how fast I speak. That one 24-second recording managed a meager 5.2 words per second. That’s about a quarter of normal — and the reason wasn’t slow speaking, it was simply that half the statement was missing.

Why that number mattered

The raw transcription time was a perfectly normal 2.12 seconds. So it wasn’t a performance or timeout problem. Nothing was slow — content was missing. That distinction turned the whole investigation in the right direction.

The wrong suspect: the microphone

My first thought went — as it probably does for most people — to the input signal. The classic suspicion: the start of a word is too quiet, you’re still drawing breath, the mic doesn’t capture the first syllables cleanly. I was even ready to “shave off the first few words” on purpose, because I was convinced the audio at the start was no good.

A second suspect was a particular flag in our call, --suppress-nst, which suppresses non-speech tokens. Maybe, the thinking went, it was swallowing too much?

Both sounded plausible. Both were wrong.

The A/B test: same recording, three variants

Instead of guessing further, I ran the exact same recording through the small model three times — only changing the flags:

(A) With glossary: start missing, output from ”… RAG Update, Promptflow Studio …”, 11 out of an estimated 40-plus words.
(B) With glossary, but without --suppress-nst: identical, the start is still missing. That ruled out that flag as the cause.
(C) Without glossary: the sentence complete — “Once more, boostN – Clouder, Notes, Hierarchy, Rack, Big Project, Orchestrator, Execution Pipeline, Commitment, LLM, MCP, Vision Board, … Wave, Tickets, Hierarchy and Sonnet.”

The comparison of (A) and (C) is the proof: it wasn’t the audio. The same audio gives a chopped sentence with the glossary and a complete one without it. So the trigger was the glossary itself.

The real cause: whisper thinks the start is a repeat

What’s happening technically? whisper.cpp receives the glossary as lead-in text. If the spoken sentence then itself begins with words that appear in that lead-in, the model treats the start as a repetition of the prompt — and skips it. It essentially thinks: “we just had that.” On the small model this effect is especially pronounced, because it works with lower confidence and goes wrong more easily.

And that’s the real irony: the effect hits exactly the sentences that begin with domain terms — precisely the cases I had added the glossary for in the first place. The help becomes the saboteur.

This also matches what others describe about Whisper’s behavior: the initial_prompt mainly affects the first segment, and the way the tokenizer detects word boundaries via the preceding space can produce these dropouts at the start.

What to take away from it

For anyone running whisper.cpp with an initial_prompt themselves, a few points are useful:

Larger models (large-v3-turbo, large-v3) are robust and barely show the skip. If you can afford them, you’re on the safe side.
The prompt limit of around 224 tokens is the same for all model sizes. So a bigger model doesn’t allow more keywords — it just handles them more cleanly.
We now stagger our glossary by model size: 12 keywords on the small model, 20 on medium/turbo, 25 on large. That’s deliberately a risk staggering, not a capacity calculation — on the weak model we keep the glossary short so less can go wrong.

But the real lesson is broader: a hint meant to improve recognition can delete text on the weaker model. That’s so counterintuitive you won’t arrive at it on your own — you look at the input signal while the cause sits in the prompt. What finally made it visible was the sober comparison of the numbers: 22.6 words per second normally versus 5.2 on the chopped recording. Without those measurements I’d probably still be fiddling with the microphone.

Keep reading

How we improved speech recognition in general: Better Speech Recognition in the boostN CLI
Sharpened with data points and prompts: Speech-to-Text with 90 Data Points

Entdecke mehr

Blog

Themenuebersicht

Arbeiten mit KI Workflows KI-Werkzeuge Coding-CLI