Streaming agent replies instead of blocking

Versely exposes two chat endpoints on the same agent. POST /api/v1/agentic/chat waits until the turn is finished and returns one JSON object. POST /api/v1/agentic/chat/stream is server-sent events: status, tokens, tool cards, and a done event. Same auth, same credit gate, same 60 requests per minute per user. The choice is what your client is willing to handle while the user is staring at a spinner.

If you listen only for a final text event on the stream, you have built the blocking endpoint with extra ceremony. The live channel is text_delta. Treat text as reconciliation at the end.

What the stream actually emits

Headers are the usual SSE set: Content-Type: text/event-stream, Cache-Control: no-cache, Connection: keep-alive, plus X-Accel-Buffering: no so a proxy does not sit on the bytes. Every 15 seconds the server writes an SSE comment (:heartbeat) so load balancers do not time the socket out during a long tool.

Events worth implementing, in the order a normal turn hits them:

Event	When	What to do with it
status	Immediately, then as phases change	First payload always includes conversation_id. Phases include loading_context, thinking, planning, answering_now, executing_plan.
text_delta	Each token	delta is the increment; accumulated is the full reply so far. Render accumulated, do not concatenate delta yourself unless you enjoy off-by-one bugs.
reasoning_delta	If the model exposes chain-of-thought separately	Same shape as text. Optional UI.
tool_call	Start and end of a tool	phase: "start" | "end", stable call_id so one card can go running → done.
tool_progress	Heartbeat inside a long tool	Copy for the activity row so the UI does not look frozen.
sub_agent_status	Domain agent start/end	Optional: show which specialist is working.
generation_start / generation_dispatched	A generate tool actually queued work	Attach pending media to this turn.
thinking	Once, if the model returned a thinking blob	Collapsed "Thoughts" row.
text	Once, at the end	Full reply for clients that ignored deltas.
done	Terminal success	conversation_id, credits_used, generation_credits_used, has_pending_generations, optional incomplete if the turn hit the tool-iteration cap.
error	Terminal failure	Stop the spinner. Do not wait for done.
refund	A generation in this turn errored	Refresh the credit balance. Log reason; do not show the raw provider string.

Plan mode adds plan, plan_update, plan_step_start, plan_step_complete. The web editor adds editor_actions (the server cannot apply them; the browser must). You can ignore both if your client is a headless script.

The first status event is a contract, not a nicety. On a new chat the client has no conversation_id yet. Without that early emit, Answer now cannot target the turn. If you need that mid-stream interrupt, you must stream, and you must persist that id the moment it arrives.

Disconnect aborts. Closing the socket cancels the LLM call and further tool dispatch on this turn, and the server persists whatever tools already ran so the next message does not re-dispatch them.

What a blocking client has to handle

POST /api/v1/agentic/chat is the same turn with the socket held until the end. The JSON you get back:

• response — final assistant text
• thinking — optional
• conversation_id
• function_calls — every tool that ran, with args and result
• generations — dispatch receipts from generate tools
• has_pending_generations — true if any of those receipts are still pending or queued
• credits_used — chat-token credits for the turn
• generation_credits_used — net generation credits this turn charged
• editor_actions — only if the web-editor agent proposed timeline edits; without this field a non-streaming client would hear "it's 9:16 now" and have nothing to apply
• incomplete — the turn ran out of tool steps; the text also asks the user to reply "continue"

Your client shows a spinner for the whole duration. A movie brief that spawns tools for a minute is a minute of nothing. You cannot Answer now. You cannot draw tool cards as they happen. You can parse one object and go home, which is exactly what a cron job wants.

has_pending_generations is the easy thing to miss on both endpoints. Generate tools return generation_started plus request_ids. The clip is not in response. If you treat HTTP 200 as "the video is ready," you will ship empty. Poll status, or wait for the conversation-task row to complete, the same way you would after a text-to-video or text-to-image ask in the app.

What a streaming client has to handle

Minimum viable player:

1. Open the SSE request with the same body you would send to /chat (message, user_id, optional conversation_id, chat_model, turn_id, attached_media).
2. On the first status, store conversation_id.
3. On text_delta, replace the assistant bubble with accumulated.
4. On tool_call start, insert a card keyed by call_id; on end, mark it done.
5. On generation_dispatched, show a pending tile for this turn_id.
6. On done, freeze the bubble, refresh credits, and if has_pending_generations or incomplete, keep the pending UI up.
7. On error or socket close, stop. Do not assume done will follow.

Do not wait for text to show words. It fires once, after the model is finished talking, which is the blocking experience again.

The official client keys off the event: line. Heartbeat comments have no event: and should be ignored. A type field on generation_start / generation_dispatched is the media kind (image, video, audio), not the event name.

Auth and credits are identical to blocking: bearer token (or the app session), credit middleware, 60/min. Chat tokens and generations both cost credits; there is no free chat allowance. Pricing is the tariff. Check the balance before a turn you know will fan out tools.

Picking deliberately

Stream when a person is watching. The agent app does this. Tool progress, Answer now, pending tiles, and "still working on the video" only exist if events arrive while the work is happening.

Block when nothing is watching. A script that sends "generate a 9:16 bottle shot, wait, print the request_id" should not parse SSE. Call /chat, read function_calls and generations, then poll. If even the agent loop is overhead, skip chat and dispatch a generate from the text-to-video tool or the matching image surface.

Stream, but ignore deltas is the failure mode. You pay for the long connection and still dump the answer in one block.

A turn that will spawn a background task is still worth streaming: the user sees spawn_background_task start and end, gets the task_id in the card, and can keep talking. The actual render collection moves to the task list. Streaming does not wait for those pixels; it waits for the decision to dispatch them.

# Person in a UI
POST /api/v1/agentic/chat/stream
# handle text_delta, tool_call, done

# Script that wants one object
POST /api/v1/agentic/chat
# read response + generations, then poll

Same agent, same models, same catalog behind the generate tools. The endpoint only changes the shape of the wait.

FAQ

Does streaming charge differently from blocking?

No. Both run the same turn, the same tools, the same credit metering. credits_used is chat tokens; generation_credits_used is the generate work. Streaming does not add a surcharge for the socket.

Why did my UI sit on a spinner and then dump the whole reply?

You subscribed to text (or to done) and not to text_delta. The server sends text once, at the end, for backward compatibility. Incremental rendering is text_delta.accumulated.

Can I Answer now on a blocking call?

No. Answer now is a mid-stream signal that needs a conversation_id from the first status event. Use /chat/stream if that control is part of the product.

What happens if the client drops mid-stream?

The server aborts further LLM tokens and further tool launches on that turn, and persists tools that already ran so the next turn does not replay them. Generations already at the provider keep going. A background task already spawned keeps going. The dropped socket is not a cancel of that work.