Clean up dictation with a local LLM (Ollama) on Mac

Gabble cleans up dictation in two layers. The first is built in and runs on the device: it removes ums and repeated words and fixes capitals and spacing. The second is optional: a language model rewrites the transcript for a purpose, such as an email or a chat message. This guide sets up that second layer with Ollama, so it runs on your Mac too.

Do you need an LLM at all?

No. The default Clean mode does not use one. If all you want is a transcript without the "um" and the "I I think", you already have it.

An LLM is worth adding when you want the text reshaped, not just tidied:

You want	Mode	Needs an LLM
Exactly what you said	Verbatim	No
Ums and repeats removed, capitals and spacing fixed	Clean	No
Grammar and punctuation fixed, your wording kept	Polished	Yes
A clear, structured email body	Email	Yes
A short, casual chat message	Message	Yes
Bullet points from a stream of thought	Notes	Yes
Prompts and commit messages with identifiers spelled as in code	Developer	Yes

How do you set up Ollama with Gabble?

You install Ollama, download one model, and select Ollama in Gabble.

Download Ollama for macOS from ollama.com and open it. That starts a local server.
Open Terminal and pull the default model:
```
ollama pull llama3.2
```
Confirm it is installed:
```
ollama list
```
In Gabble, open the Modes tab and find AI cleanup.
Choose Ollama as the provider. The defaults are the server http://localhost:11434 and the model llama3.2. Leave them unless you changed Ollama's setup.
Choose an AI mode, hold Right Option, dictate a few sentences, and release.

The text will take a little longer to appear than in Clean mode, because the transcript makes a second trip through the language model before it is pasted. How much longer depends on your Mac and the model.

What exactly is sent to Ollama?

Only text is sent, and only to your own machine. The order is:

Your speech is transcribed on the Mac. The audio is held in memory, never written to disk and never uploaded.
On-device cleanup runs.
The transcript text is sent to the Ollama server at localhost, together with the mode's prompt and your dictionary terms.
The rewritten text comes back and is pasted into the app in front.

localhost means the request never leaves your Mac. You can turn Wi-Fi off and it still works.

What does a mode prompt do?

A mode prompt is the instruction the model gets along with your transcript. It is the whole difference between Email mode and Message mode: same speech, same model, different instruction.

Take Polished. Its prompt tells the model to fix grammar, punctuation and capitalisation, to remove filler words, false starts and self-corrections while keeping the corrected version, and to keep your wording, tone and meaning without adding content or making it more formal.

Email mode's prompt asks for a clear email body in short paragraphs. It says to keep a greeting and sign-off only if you said them, and to keep every fact, name and number without inventing details.

On top of every mode prompt, Gabble adds a fixed guardrail. It tells the model that the transcript is something to clean up, not a request to answer, and to reply with the rewritten text only, in the same language you spoke. That is why dictating "what is the capital of France" pastes the question and not "Paris".

Small local models do not always follow instructions perfectly. If a result looks off, your original is not lost: the History tab keeps your dictations and is searchable.

How do you write your own mode?

You create a custom mode in the Modes tab and give it your own prompt. Good prompts for this job are short and specific.

A few rules that help:

Say what the output is. "Rewrite the dictation as a standup update with three headings: Yesterday, Today, Blockers."
Say what must not change. "Keep every name, number and date."
Say what to leave out. "No greeting, no sign-off."

Then attach the mode to an app with a per-app rule, also in the Modes tab. The defaults already map Mail to Email, Slack and Messages to Message, Xcode, VS Code and Terminal to Developer, and Notes to Notes. You can change any of them.

Can you rewrite existing text with the local model?

Yes, with command mode. Select text in any app, hold Shift together with the dictation key, and say an instruction such as "make this shorter". The selection is replaced with the result. With Ollama as the provider, the selected text is processed on your Mac.

Should you use Ollama or an API key?

Use Ollama if you want everything to stay on the Mac, and an API key if you would rather not run a model locally.

	Ollama on this Mac	Your own API key
Where the transcript goes	`localhost`	Anthropic, or an OpenAI-compatible API
Works offline	Yes	No
Cost	Free to run	Billed by the provider for usage
Setup	Install Ollama and pull a model	Paste a key, stored in the macOS Keychain
Uses your Mac's memory and disk	Yes	No

In both cases only the transcript text is sent, never audio.

What if nothing changes after you turn it on?

Check that the provider is reachable and that the mode is an AI mode. When the AI step fails, Gabble pastes the on-device cleaned text so the dictation is not lost, which can look like the LLM is being ignored.

Run ollama list in Terminal. If it errors, Ollama is not running.
Make sure the model name in Gabble matches a model in that list.
Check which mode is active. Clean and Verbatim never use the LLM.

For the wider picture of keeping dictation on your own machine, see offline dictation on Mac and why your dictation app shouldn't need the cloud. If you do not have Gabble yet, download it here.

Frequently asked questions

Can I use Ollama to clean up dictation on a Mac?

Yes. Gabble can send the transcript to an Ollama server on the same Mac and paste the rewritten text. The default server is http://localhost:11434 and the default model is llama3.2.

Does AI cleanup with Ollama work offline?

Yes, once the Ollama model and the speech model are downloaded. The transcript goes to a server on your own Mac, not to the internet.

Is my audio sent to the LLM?

No. Speech is transcribed on the Mac first. Only the transcript text is sent to the AI provider, whether that is Ollama or an API key.

What happens if Ollama is not running?

You do not lose the dictation. If the AI step fails, Gabble pastes the on-device cleaned text instead. Start Ollama and dictate again to get the rewritten version.

Do I need an LLM to remove ums from dictation?

No. Gabble's default Clean mode removes ums and repeated words and fixes capitals and spacing on the device, with no AI involved.

Which Ollama model should I use for dictation cleanup?

Start with llama3.2, which is Gabble's default. If you try another model, compare the output on your own dictation before switching.