The change is about what the model can attend to. A convolutional network reasons locally by design, building understanding outward from neighbouring pixels. A transformer treats the input as a sequence of patches and lets any patch attend to any other, so a decision in one corner of the frame can be informed directly by something happening in the opposite corner.
For video that is not a nicety. Coherence over time is a long-range relationship — frame one has to constrain frame ninety — and an architecture that can attend across the whole sequence is a far better fit than one built to look at neighbourhoods.
It also brings the scaling behaviour that made language models what they are. Performance improves predictably with size and data in a way that earlier image architectures did not, which is a large part of why the generation gap between model releases has been so visible.
In practice
- Transformer internals, diffusion training objective — the two are not alternatives.
- Long-range attention is why these models hold a scene together across many frames.
- Compute cost grows quickly with sequence length, which is one reason clips are short.
The mistake to avoid
Assuming a transformer-based generator works like a chatbot. It denoises frames; it does not predict the next word, and it has no conversation state.
Go deeper
Understanding AI Models in 2026: How Diffusion, Transformers, and Flow Matching Power Modern Creative Tools
A senior practitioner's mental model for how today's AI actually works. Transformers, diffusion, flow matching, DiTs, and state-space models — mapped to the 2026 tools you already use.
Related terms
Diffusion model
A diffusion model generates by starting from random noise and removing a little of it at a time until a picture or clip is left behind.
Flow matching
Flow matching trains a model to follow a direct path from noise to data, rather than learning to reverse a long chain of noise-adding steps.
Autoregressive model
An autoregressive model generates one piece at a time, each piece conditioned on everything produced before it, rather than refining a whole output at once.
Latent space
Latent space is the compressed representation a model actually works in — a much smaller version of the image or clip that keeps meaning while discarding raw pixel count.
LoRA
A LoRA is a small add-on file that adjusts a large model's behaviour — teaching it a specific character, product or style — without retraining or replacing the model itself.
The all-in-one AI studio for creators. 60+ models for video, image, voice, music and lipsync in a single app.