A 20-prompt suite for testing any video model
Twenty short prompts, each isolating one known failure axis, so a new model's weaknesses show up in an hour instead of halfway through a client project.
Every video model has a demo reel, and every demo reel is a list of the prompts that worked. The useful information is the complement of that set — the shot types the model quietly can't do — and you'll find it eventually either way. The only question is whether you find it in an hour of deliberate testing or on the afternoon a client asks for a close-up of hands.
This is a fixed battery of twenty prompts. Each isolates one known failure axis, and each is deliberately boring in every dimension except the one it tests.
The design rule that makes it work
Make everything except the target trivially easy.
That's the whole method. If a hands prompt also has a moving camera, a crowd and dramatic lighting, a bad result tells you nothing — you can't attribute the failure. So: static camera unless the axis is camera movement. Plain background unless the axis is crowds. Even daylight unless the axis is lighting. Short sentences, one subject, one action.
Four rules follow:
- Keep prompts short. A long, richly specified prompt confounds instruction-following with the failure you're hunting. If the model can't render hands, you want to see that, not wonder whether it dropped a clause.
- Don't negative-prompt the failure away. Suppressing artefacts with a negative prompt is a production technique and it invalidates a diagnostic. Run the suite naked, then use negative prompts in real work knowing what they compensate for.
- Run short. These failures surface in the first few seconds. Testing at maximum duration buys almost no extra signal and costs a lot more credits.
- Hold every other variable fixed. Same duration, aspect ratio and resolution across all twenty prompts and every model. Versely's default is 25 fps, so that stays constant unless you override it. Lock the seed within a model, so rerunning prompt 7 is the same test twice rather than a fresh roll — but don't expect it to align anything between models. Seeds don't transfer: seed 42 on one model has nothing in common with seed 42 on another.
The twenty prompts
| # | Axis | Prompt | What you're looking for |
|---|---|---|---|
| 1 | Hands | Close-up of hands tying a shoelace on a wooden floor, even daylight, static camera | Digit count under self-occlusion; fingers that reform between frames |
| 2 | Hands | A hand picks up a single playing card from a table and turns it over, plain background, static camera | Thin-object grip; the card bending, doubling or passing through the fingers |
| 3 | Fast pan | Whip pan from a bookshelf to a window in a bright room, nothing else moves | Geometry surviving speed; blur that smears rather than resolves |
| 4 | Fast pan | Camera tracks left alongside a person walking down an empty corridor, matched speed | Parallax; whether the corridor regenerates new architecture as it scrolls |
| 5 | Dialogue sync | A woman in a grey jumper looks at camera and says "The package arrives on Thursday", static medium shot | Phoneme-level lipsync on one short line; drift by the final word |
| 6 | Dialogue sync | Two people at a small table alternate one line each, the second speaking only after the first stops | Speaker attribution; both mouths moving on one voice |
| 7 | Reflections | A person walks past a full-length mirror in an empty hallway, static camera | Whether the reflection tracks the subject or runs its own animation |
| 8 | Reflections | A glass of water on a dark table beside a lit window, slow push in | Refraction stability; specular highlights that crawl or flicker |
| 9 | Crowds | A busy railway platform at rush hour, static wide shot, no principal subject | Background figures dissolving, merging or gaining limbs over the clip |
| 10 | Crowds | A single person walks against a crowd moving the other way, medium shot | Crossing motion; identity surviving being occluded by strangers |
| 11 | In-frame text | A closed shopfront with a sign reading OPEN 9 TO 5, daylight, static shot | Legibility held for the full duration, not just the first frame |
| 12 | In-frame text | A hand rotates a cardboard box so the printed label faces camera | Text surviving a transform — the case that breaks nearly everything |
| 13 | Contact physics | Milk pours from a jug into a clear glass on a kitchen counter, static camera | Fluid contact, level rising, liquid not passing through glass |
| 14 | Contact physics | One person hands a coffee cup to another, both hands briefly on the cup | Grip and handoff; the cup floating, duplicating or being held by nobody |
| 15 | Style hold | A cel-shaded illustrated fox trots across a flat-colour meadow, side-on, static camera | Non-photoreal style drifting toward photoreal as the clip runs |
| 16 | Style hold | A 1970s 16mm-grain street scene, slow dolly forward | Style plus camera move together; grain that swims independently of the image |
| 17 | Occlusion | A person walks behind a pillar and out the other side, static camera | Whether the same person comes out as went in |
| 18 | Occlusion | A cat walks under a table and out the far side, static wide shot | Partial occlusion of a deformable subject; limb count on re-emergence |
| 19 | Lighting | A person walks from bright sunlight into the shade of an awning, static camera | Exposure and skin tone responding to the light change coherently |
| 20 | Lighting | A desk lamp is switched on in a dim room, static camera | Causality — whether the room actually responds to the source |
Why these ten axes
They aren't arbitrary. Each is a place where the generation problem is structurally hard, which is why the same axes keep failing across model generations rather than getting solved once.
Hands and contact physics are the same problem twice: an articulated structure whose parts occlude each other, constrained by physics the model never explicitly learned. Contact — two objects agreeing about occupying adjacent space — is where that shows.
Fast pan tests whether the model has a coherent scene or just a plausible frame. Frames that each look right independently fall apart once the camera moves fast enough to demand they be views of one room.
Dialogue sync and in-frame text are fine-grained temporal precision problems. Lipsync fails by tens of milliseconds; text fails by a few pixels. Both are invisible in a still and obvious in motion, which is why they survive informal review.
Reflections and lighting test whether the model treats the scene as a physical space or a texture. A reflection animating independently of its subject is the clearest evidence for the second answer.
Crowds and occlusion are temporal consistency under load. One subject on a plain background is the easy case. Twenty subjects hiding each other decides whether a model can shoot a real location.
Style hold catches drift toward the training distribution's centre of mass. Stylised prompts often start correct and decay photoreal over a few seconds, which no single frame reveals.
Running the suite
Twenty prompts across three candidate models is sixty generations, so do the arithmetic before you start rather than after. Checking credits before generating covers the estimate and the balance in one pre-flight step, and there's no zero-cost way to run a generation, so the number is worth knowing up front.
Then run it as a fan-out rather than sixty separate requests. Generating a video from text takes one prompt across several named models in a single ask — same instruction, same settings, different model, and no opportunity for a typo to creep into attempt three. Something like:
"Run this prompt across three models, 5 seconds, 16:9, no audio: 'Close-up of hands tying a shoelace on a wooden floor, even daylight, static camera.'"
Then repeat down the list. Batch generation is the pattern; testing twenty creatives in a morning is the same mechanic pointed at ad variants.
Record pass/fail first, not a score. Twenty binary judgements finish in fifteen minutes and give you the map — which axes this model can't do. Reach for a finer scale only on the axes you actually shoot. A model that fails prompts 9 and 10 and nails the rest is entirely usable if you never shoot crowds, and the binary map lets you decide that in one pass.
Keep the results. Running this once produces a per-model profile that stays useful for months and tells you where to route a shot without re-testing. It also converts "this came back wrong" into "this is prompt 12's failure mode, which this model has" — the diagnostic tree for generations that come back wrong starts faster when you already know the weak axes.
FAQ
Should I rewrite the prompts for my own content?
No — that defeats the purpose. The value is that the twenty are fixed, so a March result is comparable to an August one and one model's profile is comparable to another's. Add your own prompts as a separate list; don't edit these.
What if a model has no audio?
Skip prompts 5 and 6 and note that you did. Score eighteen, and don't compare an eighteen-prompt result against a twenty-prompt one as though they measured the same thing.
Do I need all twenty on every new release?
All twenty the first time you evaluate a model. On a point release, run the axes that failed plus two that passed — the failures to see if they're fixed, the passes to catch regressions. Usually six to eight prompts and twenty minutes.
Does this suite work for image models?
Partly, and less than you'd hope. The two hand prompts and the shopfront sign point at problems that exist inside a single frame — digit count under self-occlusion, a thin-object grip, legible in-frame text — so they transfer once you strip the verbs out of them. Everything else on the list is testing something that only exists over time: sync, occlusion, contact, style drift, reflections tracking a moving subject. An image battery needs its own twenty built on the same principle — isolate one hard thing, make the rest easy.
Run it on the next model you're tempted to adopt on the strength of a demo reel. An hour and sixty short generations buys the list of shots you now know not to send it.