Concurrency limits are a model selection criterion
For batch work, how many generations you hold in flight decides wall-clock time more than per-clip speed. How to measure throughput and order a queue.
Two models. Model A returns a clip in ninety seconds, Model B takes four minutes. You need sixty clips today. Almost everyone picks A, and for a batch that size it is frequently the wrong call, because per-clip speed is not the quantity that decides when you finish. The quantity that decides when you finish is how many generations you can have running at once.
The arithmetic is unforgiving and it takes about ten seconds to check, which is why it is strange that so few model comparisons mention it at all.
The arithmetic
Wall-clock time for a batch is roughly:
ceil(N / C) × T
Where N is the number of generations, C is how many you can hold in flight, and T is the per-generation time. T is the number every model page and every benchmark talks about. C is the number that multiplies it.
Run the two models above through it for sixty clips:
| Per-clip time | In flight | Waves | Wall clock | |
|---|---|---|---|---|
| Model A | 90s | 2 | 30 | 45 min |
| Model B | 4 min | 12 | 5 | 20 min |
Model B is 2.7x slower per clip and finishes in under half the time. Nothing clever happened. The slower model was simply allowed to run wide.
This flips a lot of intuitions. It means "fastest model" is a single-clip claim and a batch claim only by accident. It means a turbo tier bought for speed can be the slower choice at volume if it happens to be more contended. And it means the honest unit for batch work is not seconds per clip, it is clips per minute at your actual concurrency, which is a number you have to measure rather than read.
Why nobody publishes C. It is not a model property. Per-clip time is roughly a function of the model, its size, the resolution and the length. Concurrency is a function of capacity, routing, how many other people are asking for the same model at the same moment, and whatever limits sit between you and the hardware. It moves by hour and by day.
Which leads to the practical consequence: C is measurable for your account, on your models, at the hours you actually work, and not otherwise knowable. Vendors cannot publish it honestly and mostly do not try. The fastest-model shortlist and the catalog's fast-tier flag tell you about T. You have to find C yourself.
Measuring your real throughput
This takes one afternoon and you do it once per model you intend to batch on.
- Fix everything except count. Same prompt, same duration, same resolution, same aspect ratio. Versely defaults to 25 fps, so leave that alone too. You are measuring the pipeline, not the prompt.
- Estimate the bill first. The cost estimator returns per-item and total credits against your current balance before anything dispatches, and there is no free allowance to soak up a measurement run. Every one of these test generations is charged. Read the estimate-before-you-dispatch walkthrough if you want the full pattern, and check credits before a long run so it does not stop halfway.
- Ramp. Dispatch 2 at once and time to last completion. Then 4. Then 8. Then 16.
- Find the knee. Total time will fall as you widen, then flatten, then get worse. The point where it flattens is your effective
Cfor that model at that hour. Past the knee you are queuing, not parallelising. - Record clips per minute at the knee, not seconds per clip. That is your throughput number and it is the one to compare across models.
Two things to note while you do this. First, run the ramp twice, once at your normal working hour and once well outside it. If the two differ a lot, contention is your dominant term and the ordering advice below matters more than model choice. Second, count failures. A model that returns 12 in flight but fails one in six has a lower effective throughput than one that returns 8 cleanly, and reroll rate is invisible in every published speed figure.
Ordering the queue
Once you know C, the order you dispatch in is worth real minutes.
Longest jobs first. Classic scheduling result and it holds here. If you dispatch your 4-second clips first and your 20-second clips last, the tail of the batch is one long job finishing alone while everything else sits idle. Front-load the long ones so the short ones fill the gaps behind them.
Warm the model before the batch. The first request to an idle model pays a startup cost the next ninety do not. Send one throwaway generation, wait for it, then open the taps. Doing this in the middle of a wide dispatch means the whole first wave waits on it.
One model per wave, not interleaved. Alternating between two models across a batch means you are paying startup behaviour on both and getting the concurrency headroom of neither. Finish the batch on one, then start the batch on the other.
Put the reroll budget at the end, not inline. Do not stop to inspect and regenerate as you go. Run the whole batch, review it as a set, then dispatch the rerolls as their own wave. Inline rerolls collapse your concurrency to one because you are the bottleneck.
Split by resolution tier. Higher output resolution costs more time and more credits, and the two do not scale the same way. Batch all your 720p work together and all your 1080p work together rather than mixing, so the timing you measured actually applies to the wave you are running.
Where this changes model selection
Three concrete cases.
Volume creative testing. Twenty to forty variants of a hook, judged on which survives. Per-clip quality barely matters because most of them get thrown away. Pick for throughput, use a cheaper tier, and treat the whole wave as one unit of work. This is the shape batch creative testing assumes.
Localisation fan-out. One master, N language variants. The variants are independent, so this is embarrassingly parallel and C is the only thing that matters. Dispatch them all, do not wait on each.
A single hero shot. N = 1, so C is irrelevant and everything you just read is noise. Pick the best model, run it three times at different seeds, choose. This is the case people generalise from, and it is the case where the generalisation is least valid.
The batch generation entry is the short version of why these split, and the value shortlist is a reasonable starting point for the first two, because throughput work and cost-per-usable-clip work tend to point at the same models.
Running it without babysitting
Long batches do not need you sitting there. Background tasks exist for multi-step workflows that run past thirty seconds, including bulk generation, and they run asynchronously while you keep working, with progress visible as it goes. Every one of those also creates a reusable saved workflow keyed on the task name, so a batch you run monthly becomes a thing you re-run rather than re-specify. Automating a multi-step content task is the entry point, and checking or resuming a stuck run is what you want bookmarked before a hundred-clip wave rather than during one.
Name the task for the intent rather than the run. "Instagram vlog with captions" survives; "vlog rerun v2" becomes clutter within a month.
FAQ
Does running wide cost more credits than running sequentially?
No. Credit cost is per generation and depends on the model, duration and resolution, not on how many are in flight. Sixty clips cost the same whether they take twenty minutes or forty-five. Concurrency buys wall-clock time and nothing else, which is exactly why it is worth optimising separately from cost.
How often should I re-measure?
When you change models, when you change the resolution or duration you routinely batch at, and after any noticeable slowdown. The knee moves with capacity, so a number you measured in spring is a rough prior by autumn, not a spec.
What if the model I need is slow and narrow?
Then split the work. Use the constrained model only for the shots that actually require it and run everything else on a wider one, then assemble in the editor. Most batches are not homogeneous, and treating them as though they were is what makes one model's limits the whole project's limits.
Is per-clip speed ever the right criterion?
Yes, whenever a human is waiting on the result. Interactive iteration, a client on a call, a single shot you are refining. The rule is that T matters when N is small and someone is watching, and C matters when N is large and nobody is. Most teams need both and pick their models accordingly, one for the catalog shortlist they iterate on and one they batch on.