AI agent orchestration: who decides the next step

AI agent orchestration is the layer that decides what runs next: the order of steps, what happens when one fails, when a model is asked for judgment, and which action anything may take. What separates one architecture from another is not how many models you use, but who makes that decision — a model on every run, or code that a model wrote once.

What is AI agent orchestration?

Every automated process has an orchestrator, named or not. It answers four questions on every run:

• Order. Which step runs first, which waits, which can run in parallel.

• Retries and failure. What is tried again, what is skipped, what ends the run.

• Hand-offs. When work moves from code to a model, from one model to another, or to a person.

• Permission. Which step may write to an external system, and on whose say-so.

A working definition of orchestration is which agents run, in what order, and how the next step is decided, by a model or by your code. The taxonomy in the sources below draws the same line: workflows follow predefined code paths, agents let the model direct its own process. Both are agentic systems; the difference is where the orchestration lives.

That is why "AI orchestration", "LLM orchestration" and "agent orchestration" often name the same layer. AI orchestration is the broadest term for coordinating model calls, tools and data. LLM orchestration usually means chaining prompts and parsing their outputs. Agent orchestration adds the part people underestimate: deciding who may act.

Three architectures

The same process — read an inbox, decide what matters, file a ticket — can be built three ways. Picture three columns, each with the same four boxes: Trigger → Decide next step → Act → Record. Only the "Decide" box changes.

Fixed workflow

Who decides the next step: code. A model may have written it, and a step may call a model to classify or draft, but the branches are declared before the run starts. "Decide" is a code branch.

What to test: each branch with a clear input, an ambiguous one and a malformed one, plus the same input delivered twice. Because the route is written down, a test can assert which steps ran, in what order.

Cost driver: the number of runs. A bounded model call adds its own cost, and the count is known before the run starts.

Single agent

Who decides the next step: a model, inside a loop. It reads the task, picks a tool, reads the result and picks again until it stops or hits a cap. "Decide" is a model loop with a turn counter.

What to test: the stopping conditions — a tool error, an impossible task, the last allowed turn. Test the tool definitions as carefully as the prompt.

Cost driver: turns. Every turn is a model call whose input grows with the conversation, so cost per run varies and is known only when the run ends.

Multi-agent

Who decides the next step: a planner model that breaks the task up, hands pieces to worker models and merges what comes back. "Decide" splits into a planner and two workers.

What to test: the hand-offs. Does the planner pass enough context? Does a worker's failure reach the planner as a failure, or as a plausible answer? Can two workers write to the same record?

Cost driver: hand-offs. Each is at least one model call, and the planner's context grows with every result it reads. Multi-agent systems trade cost and latency for tasks whose subtasks cannot be predicted.

Multi-agent orchestration: when a second agent helps and when it just doubles the bill

A second agent earns its place in two situations: when one model call cannot hold the whole problem — a research task whose sources are unknown until the first search returns, a code change whose list of files depends on the request — and when the roles conflict, as a generator and an evaluator do, because the evaluator should not share the generator's assumptions.

It does not help when the split is cosmetic. If the "planner" always produces the same three subtasks, that plan is a fixed workflow wearing a model call as a hat, and each run pays to rediscover it. If a "specialist" exists only to call one API with arguments the planner already knows, it is a function, and a function call is free. Write down what the planner decided on the last twenty runs; if the answer was the same twenty times, the decision belongs in code.

Two further costs hide here. Errors compound: a worker's wrong answer becomes the planner's confident input. Permission blurs: when three models can each call a tool that writes, it is hard to say afterwards which one authorised the write. Both argue for few hand-offs and a code gate that validates what a model proposed before anything is written.

Orchestration as code

Whichever architecture you choose, the orchestration layer itself — order, retries, hand-offs, permission checks — should be reviewable code, not a prompt. Three reasons.

Run history you can read. A step declared in code has a name, and the runtime can record that it ran, under that name, in that run. On Whenever a step is an exported constant built with defineStep; calling it runs the operation through the runtime's step boundary, which records the invocation, and the run function's return value is kept with the run. Every run and step is logged for 7 days on Hobby and 30 days on Pro. A model-directed plan can be logged too, but then you are reading what the model said it did.

Idempotency you can prove. Orchestration code owns the duplicate check. A webhook delivery on Whenever carries a deliveryId, and two deliveries that share one produce a single run, so the same id is safe to pass downstream as an idempotency key. Trigger keys are stable identities declared in the manifest, so a workflow with several triggers tells them apart by ctx.trigger.key instead of guessing from the payload.

Replay and retries that mean what they say. On Whenever nothing retries a failed step for you and nothing replays a workflow: a step is a recorded boundary, not a resume point, and a failed step ends its run. RetryableError records that a failure was transient and how long a caller should wait; it does not schedule anything. So waiting, duplicate handling and "check the destination before writing again" are written into the workflow, where a reviewer can see them, rather than assumed from the platform.

The same principle governs the agent case. In Whenever's AI assistant example the model chooses which read-only tool to call, but the loop, its turn cap (MAX_TURNS = 8), the argument validation and the final Telegram send are all in the workflow's code; the platform keeps no conversation and never runs a tool itself, and model turns and tool calls count against the run's call budget. The model decides within the loop; the code decides that there is a loop, how long it may run, and what may leave it. Orchestration is code; a model is called only by a step the workflow declares.

A few useful answers

What is the difference between AI orchestration and workflow automation?

Workflow automation moves data between apps along a route drawn in advance. AI orchestration also coordinates model calls: when to ask a model, how to validate what it returns, and whether its answer may trigger an action. What keeps an automation with a model step from being an agent is that code, not the model, decides the route.

Is LLM orchestration the same as agent orchestration?

They overlap. LLM orchestration usually means chaining prompts and parsing results. Agent orchestration adds the questions a chain does not raise: which tools a model may call, how many turns it gets, and when control passes to another model or a person.

Do I need an orchestration platform?

You need an orchestration layer; whether it is a platform depends on what you would otherwise build. Scheduling, webhook endpoints, secret storage, run history and a hosted runtime take longest to get right and matter least to your process. If your orchestration is already a function with named steps, a platform that hosts, triggers and records it is a reasonable trade. If it lives only in a prompt, move the order and the permission checks into code first.

What is agentic architecture?

Agentic architecture is the structure of a system in which a model takes part in deciding what happens next. The usual taxonomy runs from a single augmented model call, through workflows such as prompt chaining, routing, parallelisation, orchestrator-workers and evaluator-optimizer, to autonomous agents that direct their own tool use in a loop. The choice is how much of the route is fixed in code and how much is left to the model on each run.

Sources & further reading

Sources reviewed November 13, 2026.

• Anthropic: Building effective agents — the workflow/agent distinction and the common patterns.

• OpenAI Agents SDK: Agent orchestration — a framework's own statement of the same question.

• Whenever docs: Workflow steps and context

• Whenever docs: Workflow triggers

• Whenever docs: Runtime limits and responsibilities

• Whenever docs: An AI assistant workflow

Choose your next step

• Deploy an agent task as a workflow from your coding agent

• Host an AI agent without a server to run

• Agentic workflows: an AI agent, a workflow, or both?

• Read the workflow contract