A workflow and an agent look identical in a demo. The difference shows up on the three-hundredth run. A workflow has steps you wrote, in an order you chose; an agent gets tools and a goal and picks its own path. Most of the work people buy an agent for has an order, so it should be a workflow.
What is the difference between an agent and a workflow?
The difference is where the control flow lives: in a workflow it lives in your code, and in an agent it lives in the model. Fetch the document, extract the fields, validate them, write the record, notify the owner: five named steps, which is a state machine with model calls inside the states. Your code decides that step two follows step one, and your code decides what happens when it doesn’t. Hand the same job to a loop and the model chooses its next tool call on every turn, so two near-identical inputs take different routes and both routes are defensible. It isn’t a question of quality. It is a question of where the decisions sit, and which of them you can test on its own without rerunning the whole job.
What named steps buy you: retries, readable traces and a forecastable bill
Named steps buy four things an open loop can’t give you. The first is a retry that means something: step three timed out, you run it again with the same inputs, and the first two steps still stand. That works only if the step is safe to repeat — idempotent, or holding an idempotency key the downstream system deduplicates on, because a request that timed out might already have completed.
The second is a trace you can read. Each step logs its name, its inputs, its output, its token count and its latency. Something went wrong on Tuesday; you open that run and find the step. We keep ours in Langfuse. An agent run hands you a transcript instead, and you can score spans in it, but the spans differ from run to run, so a regression lands on a step that existed in half your cases and not the other half. The named step is the thing you can point at.
The third is a bill you can forecast. Four model calls a job, 2,000 tokens in and 300 out on each, thinking off. On Claude Sonnet 5.5 that means thinking: {type: "between_tools"}, which the effort documentation says works at low, medium and high effort and rejects with a 400 at xhigh or max. Leave it on and thinking bills as output. At first-party list price, $2 per million in and $10 out, that is $0.004 plus $0.003: $0.007 a call, $0.028 a job, $28 a month for a thousand jobs. Bedrock and Vertex price separately.
The fourth is a test you can run in CI, because one job per step means one output to score: fifty real cases per step, rerun on every prompt edit and every model change, driven from GitHub Actions with promptfoo.
Costing an agent loop against a workflow: the Lamplighter example
On Lamplighter the loop costs 2.4 times the workflow on a good run and 35 times on a bad one, and neither number decides the build. Lamplighter is an invented document-retrieval system over a contracts archive that takes 1,000 contracts a month. It is not a client, not a past employer and not a composite of either; every parameter below was picked so the arithmetic can be checked rather than measured off anyone’s system.
Every turn resends the transcript, which is what prompt caching exists for. Cache reads on Sonnet 5.5 are $0.10 per million: the pricing page footnotes them at “0.05x the base input price”, not the usual tenth. Twelve turns averaging 8,000 in, 7,000 of them cached, and 300 out: $0.0007 plus $0.002 plus $0.003 is $0.0057 a turn, $0.068 a job. That is 2.4 times the workflow. And it flatters the loop: the first turn writes the prefix at $2.50 per million, 1.25 times the input rate.
Now the run where it takes a second guess. Thirty turns at 15,000 in and 300 out with the cache lapsed is $0.033 a turn and $0.99 a job, 35 times the workflow for the same output. Say one run in twelve goes that way. Across 1,000 jobs, 83 bad runs at $0.99 is $82.17 and 917 good at $0.068 is $62.36, so $144.53, or $0.145 a job.
Call the 95th-percentile job cost over the median the tail multiple. One run in twelve is 8.3%, more than 5%, so the 95th percentile sits in the bad band and the multiple is $0.99 ÷ $0.068 = 14.6. Above roughly 10 we cap the loop or write the steps out. I can’t predict that ratio in advance; we have only ever got it by leaving a loop running for a fortnight and counting, which is the other reason the traces are worth keeping.
Build costs run the other way. Five named steps with five prompts and five test sets is about nine engineer-days, the same job as a loop about three. At $600 a day that is $5,400 against $1,800, a gap of $3,600. The running gap is $0.145 minus $0.028, or $0.117 a job, which is $117 a month and $1,404 a year, so crossover sits at $3,600 ÷ $0.117 = 30,769 jobs. About 31 months at this volume.
Then add the term nobody puts in the quote. Per-step validation writes a wrong field on roughly 0.5% of Lamplighter’s jobs; the loop, with no schema gate between its own turns, does it on roughly 2%. A wrong renewal date found three months later costs somebody $120 to chase and undo, so that 1.5-point gap is worth 0.015 × 1,000 × $120 = $1,800 a month against $1,404 of tokens a year. Per job the gap becomes $0.117 plus $1.80 = $1.917, and crossover falls to $3,600 ÷ $1.917 = 1,878 jobs. Under two months.
Derate the error cost fourfold, to $30, and the per-job gap is $0.567, crossover 6,349 jobs, payback a little over six months. The decision survives. Halve inference prices and crossover moves to 1,937 jobs, a 3% shift, because tokens were never what decided it.
How you recover when step four fails on run 300 of 1,000
In the workflow the job stops at step four, and you have its name, its exact inputs and the outputs of steps one to three in storage, so you fix the bug, replay run 300 from step four, and pay for one step’s tokens. In the agent there is no step four. You have a transcript of thirty tool calls, in one of which it wrote the record twice, or filled a missing field with something plausible.
Re-running might not reproduce it, and the workflow isn’t deterministic either. You can’t pin a model step at all now. The API versioning page says that from version 2026-10-21 the temperature, top_p and top_k parameters “are removed” and that “a request that sends any of them returns a 400 error on every model”. What a named step gives you is its exact inputs, stored; run it fifty times and score the fifty outputs.
The failure that costs most is the one that doesn’t throw. A model asked to produce a field will produce a field, and if your schema can’t represent “not in the document”, you have required the guess. A nullable field in the schema you hand to output_config.format — "renewal_date": {"type": ["string", "null"]}, still listed in required — gives the absence somewhere to go. The structured outputs reference documents two keys there and no more.
When an open-ended loop is actually worth it
An open-ended agent loop is worth it when the order of the work genuinely depends on what the work finds. Debugging is the clearest case: read the stack trace, open a file, and that file tells you which file to open next. No branch diagram survives that. A loop also wins when the tail of cases is too wide to write down and a wrong answer is cheap to check: an agent in a terminal with an engineer reading its output is a different risk from the same loop running unattended at three in the morning.
Cap it anyway. A turn cap, a task_budget inside output_config (beta header task-budgets-2026-03-13, minimum 20,000 tokens) and a max_tokens that actually stops the request. The task budgets page is headed “Task budgets are advisory, not enforced”; it says “Claude might occasionally exceed the budget” and that the enforced limit on total output tokens is still max_tokens. Ship the budget without the ceiling and you have bought the $0.99 run.
Write the steps in code if you can write them on a whiteboard
If you can write the steps on a whiteboard in order, write them in code, and put the person at a threshold rather than at the end. Not “a human reviews everything”, which nobody is doing by week three. A threshold per step, a queue for what falls under it, and a hard stop in front of anything irreversible or outward-facing: an email to a customer, a refund, a merge to main.
The threshold needs a signal with a number on it. How many fields came back null is one, because strict: true holds the shape in most cases — refusals and truncation excepted — and what is left to measure is how much of that shape is empty. A retrieval score is another, and it needs deriving rather than picking: on Lamplighter’s extraction step we set an acceptable miss rate of 1 in 200, read the score distribution off a labelled set of 400 contracts, and the cut landed at 0.81. A model’s own confidence is not a candidate; asked for a number it produces one, and that number tracks its correctness poorly.
But if your volume is a few hundred jobs a month and a wrong field costs ten minutes to fix, don’t build the workflow at all.
Run the loop, read the traces for a fortnight, and come back when the tail multiple is over 10. What we do with applied AI starts at that division.