Karaoke captions highlight the wrong word

The highlight is not late. It is on a different word. That is the tell that separates a styling bug from a timing bug, and most people spend the next twenty minutes swapping presets.

Karaoke-style captions (full line visible, active word recolored as it is spoken) consume a start time per word. If those times were interpolated inside a speech segment rather than measured against the audio, the animation will look drunk on any sentence that is not perfectly even. Line-level subtitles hide the same error because the whole chunk arrives together. Word-timed looks do not.

The highlight is honest; the timestamps are not

Open the caption file. You will see a start and end on every word. That is not proof the engine heard the word at that instant. In the Whisper-class pipeline that still sits under a lot of auto-captioning, the recognizer emits a segment: a stretch of audio with one start, one end, and a string of words. Word positions inside that stretch are then spaced out. Six words across 1.70 seconds become six equal slots. "The" and "management" get the same duration. A pause in the middle of the segment pulls later words early, because the interpolator has no silence to respect.

Karaoke highlighting then plays those slots as if they were measurements. The viewer sees the accent jump to "price" while the speaker is still on "sale". Restyling the font does nothing. The animation is correctly following bad times.

This is why the same transcript can look fine as a two-line block and broken as a word pop. Word-timed looks consume a start time per word. On Versely, that is the DYNAMIC tier of add_veed_captions (glass, whisper, glide, fusion and the rest). BASIC presets are subtitle looks: the whole line appears together. DYNAMIC follows the transcription pass's word-clock; it is not a phoneme aligner. The caption styles catalog also documents two word-timed families, Snap and Reveal, as a visual taxonomy.

A useful diagnostic, once, on a clip you already captioned:

What you see	What it usually is	What it is not
Accent sits on the wrong word, especially mid-sentence	Interpolated word times inside a segment	A bad font, color, or stroke
Whole line arrives early or late, words inside it agree	Segment boundary is wrong, or the file's frame rate is wrong	Karaoke-specific
First two minutes fine, then every cue early	Accumulating clock error (variable frame rate, 30-second window rounding)	A single bad word time
One proper noun always late	Recognition merged it with the next word	Alignment of the rest of the line

The first row is this article. The third row is a different job: transcode to constant frame rate before you caption. Do not mix the fixes.

How interpolation hides inside a "word timestamp"

Speech-to-text has two clocks. The segment clock is measured. The word clock, in a lot of production pipelines, is inferred. The JSON will still say "start": 12.84. That number was not read off the waveform for that word. It was assigned.

Forced alignment is the opposite job. Forced alignment already has the transcript and only has to place it. A phoneme aligner (wav2vec2 is the usual one; WhisperX is the package that runs it after a VAD pass) maps each word to the audio evidence for that word. Short words get short spans. Pauses stay pauses. Plosives land on the closure, not on the vowel that follows.

The practical difference on a 12-word sentence is hundreds of milliseconds of local error that never shows up as a bad WER. Recognition can be letter-perfect and karaoke still wrong. That is the trap. Proofreading the words does not proof the times.

WhisperX's useful extra is not the recognizer. It is the order: voice-activity detection first, so segments cut at silences, then the phoneme aligner. Long regions get split at the quietest point rather than at a 30-second wall. If you skip VAD and align a two-minute monologue as one blob, you are back to interpolation with extra steps.

Do this on a clean vocal. Music under speech smears both recognition and alignment. If the voiceover already lives in a mixed file, run voice isolation and align the isolated track. Captioning a finished mix is how you get a karaoke accent that pulses with the kick drum.

Run a phoneme aligner before you pick a style

The order that actually works:

1. Lock the audio. Isolated vocal, no bed, no overlapping speaker. If you still need to mix music later, mix it after the captions are timed.
2. Get a corrected transcript. Alignment will treat your typos as spoken words. Pull a first pass from the clip, then fix names, product spellings, and numbers by hand.
3. Align, do not re-guess. Run a phoneme aligner (WhisperX or any wav2vec2 aligner) against that transcript and the same audio file. You want a start and end per word that came from the waveform.
4. Then choose a word-timed style. A DYNAMIC VEED preset only after step 3. If you skip alignment, pick a BASIC preset instead. Line-level looks will not advertise the interpolation the way a word highlight will.
5. Spot-check three hard lines. A stacked plosive ("big product pack"), a pause in the middle of a sentence, and a fast list. If those three highlight on the spoken word, the rest of the clip will.

You cannot style your way around step 3. Switching glass for whisper changes the treatment. It does not move the clock.

If the clip is a one-off hero and you only care about two punchlines, you can skip a full karaoke pass and burn those two lines with timed text overlays, using the aligner's start and end. That tool is not a subtitle engine. You supply the copy and the times. It is the right escape hatch for a claim, a price, or a CTA that has to land on a specific word, and the wrong tool for a whole talking-head transcript.

What to do inside Versely once the times are real

Versely's spoken-word captions go through add_veed_captions: speech-to-text on the video's own audio, then a styled burn-in. Presets sit in two billing tiers: 21 BASIC looks at the standard credit rate, and 9 DYNAMIC looks at 2×. Transcription and timing are the same across every preset. The tier buys the treatment, not accuracy. DYNAMIC is the word-level animated path (glass, whisper, glide, fusion, terminal, handwritten, backdrop and their siblings). BASIC is the line-level path (simple, plain, beans, corpo and the rest).

Use a DYNAMIC preset when the aligner has already done its job, or when the read is slow and even enough that interpolation is hard to see (scripted TTS with clean pauses). Use a BASIC preset when the speaker is unscripted, fast, or pausing in odd places.

preview_caption_style renders the opening seconds of your clip in a candidate style. It is a cheap decision aid, not a full-length caption job, and it is the right place to catch a karaoke miss: pick a sentence with mixed word lengths, preview a DYNAMIC look such as glass, and watch whether the accent tracks the voice. If it does not, stop. Do not roll the full render and hope the rest of the file is better. The rest of the file was interpolated the same way.

A request that keeps the agent on the right path:

"Transcribe this clip, isolate the vocal if there is music under it, and show me two caption looks on the first five seconds: one DYNAMIC preset (glass) and one BASIC preset (simple). If the glass highlight sits on the wrong word, do not render the full video in a DYNAMIC preset. Use simple, and keep the transcript so I can force-align it before we retry a word-timed style."

That is a style decision gated on a timing check. Most karaoke failures I see are the other way around: style first, timing never inspected.

One more boundary, because it gets mixed in: captions burned into pixels are finished. If you karaoke-caption the English master and then dub, every market inherits English timing. Dub first, caption each dubbed track from its own audio. Karaoke does not change that rule. It only makes the wrong-language cue more obvious, because now a German word is highlighted on an English beat.

FAQ

Why does the same transcript look fine as a block and broken as karaoke?

Block subtitles only need the segment to be roughly right. Karaoke needs every word's start. Interpolated word times can be hundreds of milliseconds off inside a segment that still looks acceptable as a two-line caption. The style did not get worse. The requirement got stricter.

Can I fix karaoke by shifting the whole caption track a few frames?

Only if every word is late or early by the same amount. Interpolation error is local: one word long, the next short, a pause swallowed. A global nudge moves the mess. Align the words, or drop back to a BASIC preset.

Does Versely run a phoneme aligner for me?

add_veed_captions transcribes the video's audio and times the captions to the speech. It is not a wav2vec2 alignment step you can toggle. If you need measured word times, run a phoneme aligner on a corrected transcript yourself, then either use a BASIC preset (so interpolation is hidden) or burn specific hero lines from those times with timed overlays.

Should I caption from the mixed video or from the voice track?

From the voice track. Music in the same 250–500Hz band as speech smears word boundaries. Isolate, transcribe, align, burn, then mix the bed back. Karaoke is the style that will punish you first if you skip isolation.