On-device meeting transcription: how it works and trade-offs

On-device meeting transcription means the speech-to-text model runs on your own computer or phone, so the audio of the meeting is never uploaded to a transcription service. You gain privacy and offline use, and you pay for it in hardware load and, on modest machines, some accuracy.

Meeting audio is some of the most sensitive material an organisation produces. People speak more loosely than they write. A recording holds voices, names, half-formed plans, and remarks about staff and clients. Where that audio goes is worth a few minutes of thought.

How speech-to-text works

Modern speech recognition is done by neural networks. The broad steps are the same whether the model runs in a data centre or on a laptop.

Capture. The microphone signal is sampled into digital audio.
Feature extraction. The audio is cut into short overlapping frames and converted into a representation of which frequencies are present over time, commonly a spectrogram.
Recognition. A neural network maps that representation to text. Current models are typically trained end to end on very large amounts of recorded speech paired with transcripts, and many handle several languages.
Formatting. Punctuation, capital letters and timestamps are added, either by the same model or by a later step.

Two extra steps matter for meetings:

Voice activity detection finds the stretches where someone is speaking, so silence is skipped.
Speaker diarisation works out who spoke when. It labels segments as speaker one, speaker two and so on, usually by comparing voice characteristics. Attaching real names needs further information, such as separate audio channels for each participant.

What "on-device" changes

In a cloud service, the audio is streamed or uploaded to the provider, recognised there, and the text is sent back. In an on-device setup, the model file sits on your machine and all four steps happen there. The audio stays in memory or on your own disk.

This has become practical because open speech models are now available in a range of sizes, and because ordinary computers and phones increasingly include hardware that runs neural networks efficiently. Smaller versions of these models, often compressed by storing their numbers at lower precision, run in real time on a recent laptop.

The benefits

The audio does not leave. There is no third party holding the recording, no retention period to check, and no question of whether it is used for training.
It works without a connection. Useful on site visits, in transit, and in buildings with poor coverage.
No per-minute charge. The cost is your own hardware and electricity.
Simpler consent conversations. Telling participants "the audio is processed on my computer and not uploaded" is easier to explain than a chain of processors.

The trade-offs

Accuracy. Larger models are generally more accurate, especially with accents, crosstalk, poor microphones and specialist vocabulary. A cloud service can run a very large model. Your laptop may need a smaller one. On clear audio the gap can be small. On a noisy conference line it shows.

Hardware load. Transcription is heavy computation. On a machine without a suitable graphics processor or neural accelerator, it may run slower than real time, spin up the fans and drain the battery. It also competes with the video call itself for resources.

Live versus after the fact. Live captions need the model to keep pace with speech, which favours small models. Transcribing the recording after the meeting can use a larger, slower model and usually gives better text. Many setups do a quick live pass and a better final pass.

Diarisation is harder locally. Working out who said what is a separate model with its own cost, and it is weaker when everyone shares one microphone in a room.

Features that need scale. Custom vocabulary, many languages, and live translation are areas where hosted services have often invested more. Check what you need before choosing.

You are responsible for the files. A transcript on your disk is in your custody. Disk encryption, backups and a retention habit are your job.

The summary is a separate step

This point is often missed. Transcription turns audio into text. Summaries, action items and follow-up emails come from a language model that reads the transcript. That second step may run locally or may be sent to a hosted model.

So a tool can honestly say "transcription happens on your device" and still send the full text of the meeting to an AI provider to be summarised. Text is less sensitive than audio in one respect, since it carries no voice. But it still contains everything that was said. Ask about both steps. The guide on what gets sent to an AI provider explains what such a request contains.

Consent and recording rules

Rules on recording conversations differ by country and, in some places, by region within a country. Some require every participant to agree; others require only one. Workplace and sector rules may add more. On-device processing does not remove the need for consent. Tell people at the start that the meeting is being transcribed, say where the transcript will be kept, and check the rules that apply to you with your own adviser. This guide is not legal advice.

Practical tips for better transcripts

Use a headset or a decent microphone. Input quality matters more than model size.
Ask people to avoid talking over one another when decisions are being made.
Where possible, capture each participant's audio as a separate channel. It makes speaker labels far more reliable.
Keep a short list of names and terms to correct after the fact.
Decide how long transcripts are kept, and delete raw audio once the transcript has been checked.
Treat the transcript as a draft. Read the summary before it is sent to anyone.

For what to do with the transcript afterwards, see meeting prep briefs and follow-ups. For the wider design idea, see local-first AI explained.

Where Prism fits

Prism Desktop includes audio huddles shared by people and agents. Speech-to-text runs on your own device, and the app produces summaries, prep briefs and follow-ups. When a hosted model is used, only the prompt goes to the AI provider the person chose. Early access is by waitlist: see Prism Desktop or join the waitlist.