Structure makes frontier models more expensive and open-weights models cheaper.
We ran the same 104-task agent benchmark across two models and two conditions, then went back to the logs for the bill. Structure nearly tripled the frontier model's cost per task, and cut the open-weights model's: it costs the frontier model extra quality-gate loops, and it saves the smaller model from getting lost.
| Per task | Opus 4.6 | GLM-5.1 |
|---|---|---|
| Raw | $1.85 | $0.54* |
| + DeepWork workflows | $5.18 | $0.11–$0.19* |
Tyler Willis · April–June 2026 experiments · Published July 2026
What we measured
AgentIF-OneDay is 104 tasks a competent person could finish in a day: a financial report, a scraped spreadsheet, a small website, a trip plan. We have run it dozens of times across models and harnesses, and the numbers runs usually report, score and wall-clock, don't tell you what a run costs: a task that finishes in ten minutes can bill more than one that runs an hour, if the ten-minute task fanned out to a dozen sub-agents. So we went back to the logs for the two things cost is made of: tokens moved, and dollars charged.
Two of the four cells give us dollars directly. Claude Code writes a metadata.cost_usd field for every task, so the frontier runs are billed numbers, not estimates. The open-weights runs are harder. OpenRouter returned a usage object on every call with exact token counts, but the cost field came back zero. For GLM we have measured token spend and a price we recorded separately, not a receipt.
The bill
Both models improved; only the frontier paid for it.
| Condition | Per task | 104-task run | vs raw | Score |
|---|---|---|---|---|
| Claude Code | $1.85 | $192 | baseline | baseline |
| Claude Code + DeepWork | $5.18 | $469–$562 | ×2.8 | +5.6 pp · −36% failures |
| GLM-5.1 / Pi | $0.54* | $56* | baseline | baseline |
| GLM-5.1 / Pi + DeepWork | $0.11–$0.19* | $11–$20* | ×0.2–0.35 | +4.5 pp · −29% failures |
Where the money goes
The frontier tripling is execution, not authoring. Creating workflows fresh cost about the same as running pre-built ones ($4.51 versus $5.18 per task), which tells you the money is in the execute-validate-fix loops that quality gates add, not in writing the procedure down.
The average hides the shape. Cost on this benchmark is long-tailed: a handful of tasks eat the run. In one DeepWork run, taskif_90 cost $41.77, taskif_125 $37.16, and taskif_31 $33.89 while the median task cost a few dollars; a later run pushed taskif_125 to $44.67 on a single turn. The expensive tasks are the ones that fan out to sub-agents, and that is exactly where our own telemetry went blind. A task reporting one turn had quietly spent $22.31 on 821 Claude Haiku web searches that never appeared in the parent's usage. The pattern held beyond that one task: tasks reporting five turns or fewer averaged $10.05, while mid-turn tasks averaged $2.70. A short transcript reads as a small task, but it is often a sub-agent fan-out wearing one turn, and those tasks ran 3.7× more expensive.
None of that tail is forced spending. It suggests three levers: skip structure on tasks that don't need it, skip it on the tasks that explode, and cap it — a workflow that stops and raises a manual-review-required error after, say, $7 of spend would have cut taskif_90 from $41.77 to $7 and left the median task untouched. We haven't run the capped configuration, but on this run's own numbers, capping just the three worst tasks would have taken roughly $90 off a $469 run.
The logs support a sharper strategy than a flat cap. Of the 104 tasks, 42 improved under DeepWork, 23 got worse, and 39 didn't move, so a run or two of comparison data is enough to learn which tasks structure helps and to buy it only there. Replayed on this data, that portfolio scores 80.0, a +8.9-point lift over baseline against full DeepWork's +5.6, because it also skips the tasks structure hurt, and it costs $332 a run instead of $539 (both per-task means × 104): about $16 per additional point, where indiscriminate DeepWork pays about $62. That is an oracle number, picked on the same runs it is scored on, so a fresh run would land lower; the direction is the point. Because the workflows are editable files, the tuning is iterative: each run's telemetry shows where the next safeguard earns its cost and where structure is just overhead.
Open weights
GLM-5.1's raw run moved 158.0M tokens: 50.9M input, 2.0M output, and 105.1M read from cache. Priced at the public OpenRouter rate we had on file, $0.98 per million input and $3.08 per million output, the billable input and output alone come to about $56 for the whole 104-task run, roughly $0.54 a task, with cache reads billed on top at a rate we didn't capture. That is $0.54 against Opus's $1.85 raw, and the recorded blended rates put GLM about 6.4× cheaper per token.
On GLM, structure made the run cheaper, not costlier. The two workflow runs moved 93.2M and 95.3M tokens against the raw run's 158.0M, and their billable input plus output came to $11 and $20 rather than $56. The mechanism is visible in the same logs. Raw GLM drifts, summarizes what it would do instead of doing it, and times out: its raw run took 48,247 seconds (13.4 hours) and hit one timeout, while the workflow runs finished in about 21,000–23,000 seconds (six hours) with none. The tokens it saved were the tokens it had been spending on wandering.
token ledger
token spend · GLM-5.1 via OpenRouter # measured from run logsraw control 158.0M tokens 50.9M in · 2.0M out · 105.1M cache est. $56opus workflows 93.2M tokens 5.2M in · 2.0M out · 85.9M cache est. $11opus workflows rep2 95.3M tokens 14.3M in · 1.9M out · 79.1M cache est. $20# est. = in/out tokens × recorded public rate ($0.98/M in, $3.08/M out).# provider logged $0.00 for cost; cache reads bill extra, rate unrecorded.judge pass · Gemini 3.1 Pro via OpenRouter # billing export1,957 generations $196.88
The judge pass is scoring, not execution: grading has its own bill, and at $196.88 it cost more than some of the runs it graded.
Break-even
Whether structure is worth buying at all comes down to what a failure costs you. DeepWork cut outright task failures 36%, 13.0 to 8.3 zero-score tasks per run, so it pays for itself once a single prevented failure is worth more than about $74 in downstream remediation (roughly $346 of added run cost over 4.7 prevented failures), a bar many enterprise workflows clear before any of the cost levers above. On the open-weights model the calculus barely applies, because there structure lowered both the error rate and the bill at once.
None of these figures needed a pricing assumption we invented. The frontier dollars are billed, the open-weights dollars are measured tokens times a rate we wrote down, and the judge line is a provider export. The one number we still can't produce cleanly is a fully-loaded open-weights dollar-per-run, because the provider handed back a token count and a zero where the cost should have been. That is the next thing we instrument.