Training runs the process backwards. The model is shown real images with increasing amounts of noise added and learns to predict what was removed at each stage. Having learned that, it can be handed pure noise and asked to undo a corruption that never happened — and what emerges is a new image consistent with everything it learned.
This explains several behaviours that otherwise look arbitrary. Generation is iterative, so there is a step count. Every run starts from a different noise field, so there is a seed. Prompt conditioning is applied at each step, so there is a guidance strength deciding how hard the text pulls.
It also explains the characteristic weaknesses. The model is refining a whole frame towards plausibility, not assembling parts, so global structure resolves early and fine specifics — text, counts, hands — are decided late from local context, which is exactly where diffusion output is least reliable.
In practice
- Noise in, picture out, over a fixed number of refinement steps.
- Seed, step count and guidance all exist because the process is iterative and stochastic.
- Most image models and many video models in general use are diffusion-based or a close relative.
The mistake to avoid
Reading the process as "the model retrieves something it memorised". It is predicting a denoising direction, which is why it can produce combinations that were never in the training data — and why it is bad at exact reproductions.
Go deeper
How AI Video Generation Actually Works: The Technical Guide to Diffusion Video Models in 2026
A deep, practitioner-level walkthrough of how modern AI video models generate frames — from latent diffusion and DiTs to flow matching, spatio-temporal attention, and the inference pipeline behind Sora 2, VEO 3.1, and Kling 3.0.
Related terms
Sampling steps
Sampling steps is how many passes a model takes to turn its starting noise into a finished output — more passes, more refinement, more time.
CFG scale
CFG scale controls how strictly a model obeys your prompt, trading obedience against the model's own sense of what a natural image looks like.
Latent space
Latent space is the compressed representation a model actually works in — a much smaller version of the image or clip that keeps meaning while discarding raw pixel count.
Flow matching
Flow matching trains a model to follow a direct path from noise to data, rather than learning to reverse a long chain of noise-adding steps.
Diffusion transformer
A diffusion transformer is a diffusion model whose internals are a transformer — the same architecture behind large language models — instead of the convolutional network earlier image models used.
The all-in-one AI studio for creators. 60+ models for video, image, voice, music and lipsync in a single app.