Two models per dictation: how speech becomes writing
Hold the key, talk, and what appears at your cursor has been through two models, both on your Mac. The first turns audio into words. The second turns those words into writing. Keeping them separate is a decision, and so is the short list of things the second one is allowed to do.
Why not one model?
A speech recognizer is trained to report what was said. That is not the same job as writing it down. It gives you the "um", the false start, the word you said twice while you thought, and punctuation that ranges from approximate to absent depending on the recognizer.
And you get to choose which one. Dictera ships with four: Apple's system engine, two from Nvidia, one from Alibaba, each with its own language coverage. Asking every one of them to also punctuate consistently across 140+ languages is not a thing you can download. So the punctuation, the fillers and the small repairs are a second pass, done the same way whichever recognizer you picked.
What the second model is allowed to change
The cleanup pass runs a Gemma 4 model locally, and its instructions are a short list rather than a general brief. It may remove fillers, hesitation sounds, false starts and accidental repetitions, in any language. It may fix punctuation, capitalization and spacing by the conventions of the language actually being spoken. It may fix a clear grammatical error, using the smallest change that makes the sentence correct. It may fix an unambiguous misspelling, restore a word you spoke in one language and the recognizer wrote phonetically in another script, and correct a brand or product name where the intended name is obvious.
It may not rephrase, summarize, paraphrase, translate, change the style or tone, or swap a correct word for a synonym. It keeps your line breaks. When it is uncertain, it keeps your text. It runs with greedy decoding, so the same sentence gets the same edit every time.
The part that makes this hard
A dictation is usually a message to somebody else, and very often that somebody is an AI assistant. Which means the text handed to the copy-editor is itself a question or a command, arriving in exactly the slot where a small instruction-tuned model expects to find its own task.
"Could you translate this on Japanese please" should come back as "Could you translate this into Japanese, please?" The failure is that it comes back as the Japanese. The instructions spend a whole paragraph on this one point: a question comes back as a question, a request comes back as a request, corrected rather than carried out.
A prompt is advice, so there is also a check
Between the model returning and the text reaching your document, the result is compared against what went in, and anything no copy-edit could plausibly have produced is thrown away. Three things count: the passage came back in a different writing system, the passage grew far longer than the input, or it collapsed to a fraction of it. Each of those is a model answering, expanding or acknowledging instead of editing. When one fires, your original transcript is what gets inserted.
The check is blunt on purpose. It does not try to catch a rewrite that stayed in the same script — "translate this to German", answered in German. No cheap comparison separates that from a heavy but honest edit, and a check that guessed would start discarding real corrections. That case stays the instructions' job.
Blunt also has to mean careful about mixed language, because people dictate that way constantly. Only a wholesale flip counts as a translation. "я установил спотифай on my new laptop" coming back as "Я установил Spotify on my new laptop" is precisely the repair the model was asked for, even though the text tips toward the other script on a few letters.
What you get when it goes wrong
The raw transcript. If the cleanup model is not downloaded, if generation fails or times out, or if the check refuses the result, the recognizer's words are inserted unchanged. There is no state in which speaking produces nothing because the second model had a bad day.
If you have pointed cleanup at your own server instead of the built-in model, an unreachable server also gets you a message saying so. Raw text arriving silently would look like the model deciding your sentence was already perfect.
Which model gets the last word
Neither, as it happens. Your spoken commands are applied first, the cleanup model runs next, and your custom replacements run last, after the model — so a rule you wrote for a colleague's name or a product nobody spells right always wins over the model's view of it. History keeps the raw transcript beside the cleaned one whenever the pass changed anything, so you can see what each model contributed.
And cleanup is a switch. Turn it off in Quick Controls (⌥⌘C) and dictation runs with no language model in the loop at all: the recognizer's words, as it heard them, with spoken punctuation still doing its work. Both models run on this Mac either way, which is the reason the whole dictation pipeline keeps working with the Wi-Fi off — and the reason the list of hosts the app contacts is as short as it is.
