Burnin
From speech to burned-in captions
Burnin's whole premise is that captioning can happen entirely on your phone. This page is the pipeline, in full, including the parts that are still approximate.
The four stages
A video goes through four steps between import and export, all of them on-device: transcription turns speech into text, segmentation turns that text into caption-length chunks, gap detection catches spans that likely got dropped, and diarization works out who is speaking so captions can be labeled by speaker.
Apple's on-device SpeechTranscriber turns the audio into text, on the phone.
The transcript is broken into readable caption-length chunks.
Spans that likely got dropped are found, then confirmed before a retry.
Who spoke when, so captions get a speaker-change prefix.
Transcription: Apple's SpeechTranscriber, on the phone
Burnin uses Apple's on-device speech transcription framework rather than sending audio to a server. The audio never leaves the phone at any point in this stage.
- Per-language models. Each language Apple supports has its own model. The first time you transcribe in a language, Burnin downloads that language's model.
- Download once, then offline forever. After a language pack is on the phone, transcribing in that language needs no connection again, on this video or the next one.
- Automatic detection, with a manual picker. Burnin tries to detect the spoken language on its own. A picker is there for when detection is not what you want, or a video mixes languages in a way automatic detection cannot resolve cleanly.
- Expected proper nouns, typed in first. Before transcription starts, you can type names it should expect, a person or a place it might otherwise mishear. Those words are passed straight to the speech recognizer as a bias, so a name like "Kylian Mbappé" is more likely to come out right the first time.
Segmentation: turning a transcript into captions
A raw transcript is one long stream of words with timing attached to each one. Segmentation groups that stream into caption-length chunks, breaking at points that keep each caption readable at a glance rather than breaking purely on a fixed word count or a fixed duration.
Gap detection: retried only when confirmed
Transcription can drop a span of real speech, usually where audio is quiet, overlapping, or otherwise hard for the model to parse. Burnin looks for spans in the video's timeline where a caption is conspicuously missing and treats them as candidates for a retry.
A candidate gap is not retried automatically. Burnin runs its diarization pass independently over that span, and only re-transcribes it when diarization confirms that speech is actually present there. A silent stretch, a music-only stretch, or a stretch of ambient noise stays uncaptioned rather than getting an invented line.
This is a heuristic, not a guarantee. It reduces the chance that a real line of dialogue goes uncaptioned, but it cannot promise that every dropped span gets caught, and a gap that is too short or too quiet for diarization to confirm stays as it was.
Diarization: who spoke, and when
Speaker diarization runs through FluidAudio, an on-device model that segments the audio by speaker without needing to know in advance who is in the video or how many speakers there are.
- Downloaded once. The diarization model is around 100MB and downloads the first time it is needed, then runs fully offline afterward, on this video and every one after it.
- Feeds the "- " prefix. Wherever diarization detects the speaker changed, the caption starting there gets a "- " prefix, the same convention subtitle formats use for a new speaker, so a two-person conversation reads correctly without manual labeling.
- Also used to confirm gaps. The same diarization pass that produces speaker labels is what gap detection asks "is there really speech here" before it retries a span.
What is still approximate
Automatic transcription makes mistakes, on any device, from any vendor. Segmentation can break a caption at an awkward point. Gap detection reduces dropped speech but cannot promise none is ever missed, especially on quiet or overlapping audio. Diarization can misattribute a speaker change on crosstalk or a quick back-and-forth. The in-app editor exists because of all of this: every caption is meant to be checked and adjusted before you export, not accepted as final.
Why on-device rather than a server
Sending video to a server for transcription would mean an account to tie the upload to, a connection every time you caption something, and a copy of your footage sitting somewhere that is not your phone. Running all four stages locally means none of that has to exist: no account, no upload, and captioning that keeps working on a flight or with the data off, once the models are on the device.