Captions run early after the first two minutes
Segment timestamps and 30-second window rounding accumulate. Use VAD-first segmentation, forced alignment, and a frame-rate check before you generate cues.
Captions that sit right for the first two minutes and then run half a second early are not a late-stage rendering bug. Segment-level timestamps and 30-second window rounding accumulate, and a wrong frame-rate assumption makes the slip grow with the file. The first ninety seconds look fine because the error is still inside a frame or two. By minute three the cue is landing on the previous word. QC that only watches the open will ship it.
Do not nudge the whole track by 400 milliseconds. That corrects one moment and misses the rest. Split on actual silences, force-align a corrected transcript, and convert seconds to frames using the container's real frame rate, not the rate you assumed because the file came from a phone.
Why the first two minutes look fine
Two different caption systems get collapsed in conversation, and only one of them fails this way.
Word-level engines return a start and end for each word. Chunking is then a reading-speed problem, not a clock problem. That is the version in timed captions from speech. When that pipeline drifts, the usual cause is a style that leaves a cue up longer than the words it belongs to.
Whisper-lineage transcription emits segment-level timestamps and interpolates word positions inside the segment. The interpolation is a guess. Whisper also works in ~30-second windows; rounding at each boundary is a small error that adds across a long file. Early captions look locked. Later captions run early.
Cartesia Ink-Whisper, which Versely uses for speech-to-text, is a Whisper variant. Treat its word times as interpolated unless you have aligned them. transcribe_audio returns plain text only; the caption burn path is where the times show up. A 15-second hook often never runs long enough for the accumulation to show. A 12-minute talking-head will.
Do not debug this by previewing the first five seconds. preview_caption_style trims the head of the file on purpose, which is correct for judging a look and useless for judging a clock that slips at minute two. Watch 0:30, 2:00, 5:00 and the last spoken line.
Three drifts that add up
They stack. Fixing only one leaves the other two visible on a long enough file.
1. Interpolated word times inside a segment. The segment says this clause ran from 184.0 s to 191.2 s. The engine then sprinkles the words across that span by duration, not by the actual audio. A pause or a stretched vowel inside the clause, and the later words land early. Hundreds of milliseconds is a typical miss.
2. 30-second window rounding. Each window's timestamps are quantised. The next window starts from a rounded edge, not from the true sample. Over four windows you are two minutes in, and the rounding has had four chances to bias the same direction. This is why "it was fine, then it wasn't" tracks file length rather than a particular word.
3. The wrong frame rate, applied when cues become frames. Seconds are honest only if you convert them at the rate the picture actually plays. Phone footage is often 30. Screen recordings are often 60. Versely's default video frame rate is 25. A 24 fps generate dropped onto a 25 fps timeline runs about 4 percent fast: 40 milliseconds of slip per second, 2.4 seconds over a minute. That mismatch has its own write-up in 24 fps clips on a 25 fps timeline. Variable frame rate is a different clock: if r_frame_rate and avg_frame_rate disagree, conform to a constant rate before you caption. The milder constant-rate miss is assuming 30 on a 29.97 file. The slip is linear. It is small at 0:20 and obvious at 8:00.
A fourth, separate cause: captions burned from language A, then played against a dub in language B. That is an order-of-operations error. Transcribe each dubbed track on its own, as in keeping captions in sync with a dub. If the file is still the original language and cues run early at minute three, you are in this article.
A VAD-first, forced-alignment pipeline
Stop asking the recogniser for clocks. Ask it for words, then place those words on the audio you actually have.
Step 1. Transcribe for text, not for timing. Run speech-to-text and correct the transcript. Alignment will treat every typo as a spoken word. Names, numbers, brand terms. The one wrong name is the one that will be highlighted.
Step 2. Cut on silences, not on 30-second ticks. Voice activity detection (VAD) finds the actual pauses. Split there. If a region is still longer than 30 seconds, split it at the quietest point inside the region (the Cut & Merge rule used by WhisperX). You are destroying the windowing artefact at the source: no long segment for interpolation to wander inside, no rounded 30-second edge to inherit.
Step 3. Force-align the corrected transcript. Forced alignment already has the text; the job is only to place each word on the waveform. A wav2vec2 phoneme aligner maps each word to its true audio position. That is the step WhisperX is known for. Word-by-word pop styles need that clock. Sentence-level subtitles can live without it.
Step 4. Build cues from aligned words. Chunk by pause, then by reading speed (about three words a second, in the 160–180 words-per-minute broadcast band), then by line length. Limits that still hold on social: about 42 characters per line, at most two lines, a cue held between one and six seconds. Do not let a cue span a VAD silence. That is how a caption hangs on screen after the speaker has stopped.
Step 5. Spot-check at four points. 0:30 (control). 2:00 (where this bug announces itself). 5:00. The last spoken line. If 0:30 is locked and 5:00 is early, you skipped alignment or the frame-rate check. If all four are off by the same amount, that is a constant offset: nudge once. The growing error is the one this pipeline exists for.
You do not need this on a 20-second Reel. You do need it on anything watched past two minutes with captions on, and on karaoke-style burn-in longer than a minute. For short clips, the caption generator is the finishing path. For a long cut, transcribe, align, then burn. A WhisperX deep-dive walks the VAD and wav2vec2 stages if you are implementing it yourself.
Check the real frame rate before you generate cues
Seconds are not frames. Cues become frames the moment they hit a timeline or a burn-in renderer. Read the container, not the camera-roll label and not the project default. ffprobe (or any inspector that prints r_frame_rate and avg_frame_rate) on the file you will actually caption. If those two rates disagree, you have variable frame rate. Conform to a constant rate before you generate cues.
Then match three clocks:
| Clock | Where it lives | What to verify |
|---|---|---|
| Audio samples | The WAV/M4A you aligned to | Uncompressed WAV for the aligner; do not align an MP3 and burn against a different encode |
| Video frame rate | The container | 24, 25, 30, 29.97, 60, as the file really is |
| Timeline | The editor | Versely sequences default to 25 fps |
If you align against 48 kHz WAV and burn onto a 25 fps timeline, convert with 25, not 30. If the picture is 24 fps on a 25 fps sequence, captions will hold against the audio and the picture will slip under them unless you conform the picture first. Audio is the master clock for speech captions. A constant offset after a correct rate conversion is usually an encoder delay: nudge once. A growing error is still an alignment problem.
For a long edit in Versely, iterate on picture with the editor's 480p preview pass (it is free and carries a short per-user cooldown). Burn captions on the cut you will ship; a single charge applies to the final export. Preview will not catch two-minute drift if you only render the first seconds of the EDL. Check the same four timestamps on the burned file. Burned-in captions are pixels. If they are early, the pixels are wrong.
FAQ
Why do the captions run early instead of late?
Interpolated words inside a segment get packed toward the start of the span when the engine overestimates how long later words take, and window rounding can bias the next window's origin. Early is the common read. Late happens too. The diagnostic is the same: the error grows with file length.
Do I have to use WhisperX?
You have to do the two things WhisperX is a known implementation of: split on silences (VAD, with long regions split at the quietest point) and force-align a corrected transcript with a phoneme aligner. Other aligners are fine. Raw Whisper-lineage word times on a 12-minute file are not.
Why does frame rate matter if the timestamps are in seconds?
Because the burn-in and the timeline address frames. Convert seconds at the wrong rate and a correct alignment becomes a linear slip. Phone and screen-recorded files are the usual source of a wrong assumption. Read the container.
My Reels are under a minute. Do I need this pipeline?
No. Accumulation needs length. For short-form, styled auto-captions and a pass over the transcript for names are enough. Use this pipeline when a viewer can still be watching past two minutes, or when a word-by-word style makes a 400-millisecond miss look broken.