Better speech recognition in boostN-CLI: Audio normalization + a flexible Whisper model

Voice input in boostN-CLI is not a gimmick — it is a real working channel: thoughts move faster than fingers, and half the idea evaporates somewhere between brain and keyboard. So it was annoying that Whisper kept dropping words on us: short sentences, technical terms, quieter passages. This post explains why that happened, what the two fixes were, and gives a bit of audio-engineering background for anyone who wants to know why those fixes are the clean fixes rather than quick hacks.

The problem in one sentence

Whisper only sees the audio signal you hand it — and if the signal is too quiet or the model too small, it simply lacks the cues to distinguish between similar-sounding words, mumbled syllables, or specialized vocabulary.

In our case two things piled on top of each other. The microphone slider on the Mac was already at maximum, but the recorded WAV level was still low — a textbook headroom effect: to make sure the loudest peak never clips, the average of the recording sits well below the digital ceiling. And the Whisper Small model is fast and tiny (244 MB), but it knows fewer acoustic variants than Medium or Large-v3. On clean, loud studio speech you do not notice. On spontaneous dictation, with room noise and mixed German/English vocabulary, you absolutely do.

Fix 1: Audio normalization with SoX

Right after recording and before handing the file to Whisper, the WAV now runs through SoX with the filter norm -1. That means: the loudest point in the recording gets pushed up to -1 dBFS — almost the full digital ceiling, with a minimal safety margin against clipping.

Why -1 dBFS and not 0?

dBFS (decibels relative to Full Scale) is the digital level scale. 0 dBFS is the absolute ceiling — anything that would go above it gets cut off (clipping), and clipping does not just sound bad, it destroys information. Whisper hates clipping because distorted vowels look like broken phonemes to the model.

With -1 dBFS we keep 1 dB of headroom — acoustically inaudible (1 dB is the perception threshold), but enough to absorb rounding errors further down the pipeline or minor resampling artifacts. Classic mastering move: as hot as possible, never clip.

Why norm and not just a fixed gain boost?

A fixed gain (say “always +3 dB”) treats every recording the same way. An already-loud recording would clip, a very quiet one would still be too quiet afterwards. norm instead works adaptively: SoX scans the file, finds the loudest peak, and calculates the gain so that this peak lands exactly on the target level. A short whispered take gets boosted hard, an already-full recording barely at all.

That way Whisper sees every clip with a similar healthy level — regardless of how close to the mic you spoke, how loud the room was, or what shape your voice was in that day.

Why no compression?

A common reflex at this point: “Then also compress, so the quiet parts get louder.” Sounds logical, but it is counterproductive here. Compression changes the relationship between quiet and loud passages — and that relationship is itself a useful feature for Whisper. Breath pauses, soft consonant tails, slight emphasis at the end of a sentence: those are cues the model actively uses. Flatten them with compression and the audio sounds more present, but the model gets less information, not more.

norm only changes amplitude — not frequency response, not dynamics. It is pure linear scaling: quiet and loud parts are raised in exactly the same ratio. Which is exactly what we wanted.

Fix 2: Model switcher in the settings menu

Until the last version the Whisper model was hardcoded to Small. Fast, tiny, ran smoothly on Apple Silicon — but on technical terms and short dictation it cost us noticeably more errors than necessary. The obvious fix “use a bigger model” we did not want to hard-swap; we wanted it pickable.

Now there is a choice under Settings → Voice → Whisper model:

• Small (~244 MB) — fast, fine for clean speech, easy on RAM
• Medium (~769 MB) — the new default suggestion, clearly better on technical vocabulary
• Large-v3 (~1.5 GB) — Whisper’s top model, best English + German, slightly more latency

The flow is tuned to zero friction

That mattered to us more than the feature itself. Switching a model should not feel like a system configuration; it should feel like a click:

1. Select model → Enter
2. CLI checks whether the model is already on disk
3. If not: confirm dialog, then auto-download via the bundled whisper.cpp script
4. Config gets persisted, the Whisper server is restarted automatically with the new model
5. Status message: “Model is now active — ready to use”

No app restart. No manually editing config files. No setting paths. No “go download the model from HuggingFace yourself and move it into the right folder.” All of that happens in the background. If you trust the CLI you can switch the model mid-workflow and be dictating again three seconds later.

Why that matters more than it sounds

Tools only get used honestly when the friction is low enough. A model switcher that needs a restart will in practice almost never be used — and at that point the whole feature is dead. So we deliberately took the time to build the full lifecycle (download, persistence, server restart, status feedback) to feel like a dropdown change, even though there is a 700 MB file moving in the background.

The result

With the Medium model and normalization enabled, subjectively ~50 % fewer errors on short sentences and technical terms. Mixed-language input in particular (a German sentence with English technical words like “webhook”, “inference”, “prompt caching”) is now recognized far more reliably.

The surprise: speed stayed practically the same

What surprised us: despite three times the model size, Medium is barely slower than Small on Apple Silicon with the Metal GPU. The reason lies in Whisper’s encoder-decoder architecture.

Whisper consists of an encoder that processes the entire audio clip once (relatively expensive, runs in parallel on the GPU), and a decoder that then generates the text token by token (much cheaper per token, but linear in the token count). For short dictation of 2–5 seconds, encoder overhead dominates — and that encoder scales very well on a Metal GPU. The extra model complexity barely shows in GPU processing because the GPU was not the bottleneck in the first place.

For longer recordings (multiple minutes) the decoder share grows and the difference between Small and Medium becomes noticeable. For our main use case — short dictation of commands, notes, prompt snippets — Medium is the new sensible default.

What we take away from this

Three things, beyond Whisper itself:

1. Audio quality is often the missing link. When an ML model “is not good enough,” look at the input signal before swapping the model. We could have gone straight to Large-v3 — and the actual root cause (too quiet a level) would have stayed hidden. Only the combination of normalization and a bigger model genuinely improved recognition.

2. Zero-friction UX is architecture, not polish. We could have shortened the model switcher to “edit config.toml.” If we had, the feature would technically be there but practically dead. UX friction is a technical design decision, not a cosmetic concern.

3. Small tools, well built. SoX and whisper.cpp are both classic Unix tools — focused, well-documented, no cloud dependency. That fits our line: local tools where possible, external services only when necessary.

Next step

Large-v3 is one click away in the same menu. We are currently testing whether the extra accuracy is worth it for our typical dictation — or whether Medium stays the sweet spot. Reports will follow when the data is in.

Entdecke mehr

Blog

Saving Tokens with Claude: 6 Principles That Make Experts Twice as Fast

How I turned my CLAUDE.md from a style guide into a token budget — 6 principles for lower cost, less waiting, and more honest reporting.

Glossar

AI Marketing

AI marketing is the use of artificial intelligence and machine learning to analyse marketing data, automate content and delivery, and optimise campaigns in real time. Its main areas are content, personalisation, analytics and generative engine optimization.

Lexikon

Automating AI Workflows

No-/low-code automation with Make, Zapier and n8n: building blocks, use cases for agencies and SMBs, pricing models and the costliest pitfalls.

Themenuebersicht

Arbeiten mit KIboostn-cli