Early readout · AgentIF-OneDay

Capability can live in the harness, not the weights.

Workflows written by Claude Opus made GLM-5.1, an open-weights model, finish 104 real one-day agent tasks in under half the wall-clock time, with fewer failures and a modest score edge. Same model, same harness, two conditions: raw, and executing the borrowed workflows.

Tyler Willis · June–July 2026 experiments · Published July 2026

Setup

AgentIF-OneDay is 104 tasks a competent person could finish in a day: build a report, wrangle a spreadsheet, plan a trip, stand up a small website. Each task carries its own rubric. A single judge, Gemini 3.1 Pro, scores every criterion pass or fail, and the net is positive points minus penalties out of a possible 636.

Both runs used the same agent harness (Pi) and the same weights (z-ai/glm-5.1 via OpenRouter). The only variable was the process the model followed.

In the raw condition, the model got the task and went. In the other, it executed a set of DeepWork workflows (explicit steps and quality gates for that kind of task) authored by Opus, not by GLM-5.1. GLM never saw the judge's criteria, the reference answers, or its own prior scores. It just followed a stronger model's recipe for the work.

Results

Higher score, less than half the clock — twice.

ConditionNetNet %Zero-scoreCompletedWall-clockvs raw
Rawcontrol · n=1451 / 63670.9%12103 / 104 · 1 timeout48,247s · 13.4hbaseline
Opus-authored workflowsrep 1488 / 63676.7%7103 / 104 · 0 timeouts21,210s · 5.9h+37 net · 2.3× faster
Opus-authored workflowsrep 2471 / 63674.1%10104 / 104 · 0 timeouts23,243s · 6.5h+20 net · 2.1× faster
All runs: GLM-5.1 in Pi via OpenRouter. Scores trace to judge summaries in the run directories; zero-score counts come from the same combined score files.
Unsupervised.com — June 2026

The two workflow runs average 479.5/636: about 28.5 net points over raw, or 4.5 percentage points. Both finished in less than half the wall-clock time (2.1–2.3× faster), and neither timed out. Failures moved the same way both times: 12 zero-score tasks raw, 7 and 10 under workflows, a 29% mean reduction. The time result reproduced cleanly. The score result reproduced too, but smaller the second time.

The bill fell with the clock: billable input and output for the workflow runs came to about $11 and $20 against the raw run's $56, measured tokens at a recorded rate. The cost readout has the full ledger.

run ledger

$ ls benchmark_runs/ # net score · wall-clock, from judge summaries
 
full104_..._glm51_raw_control_20260611_002 451/636 48246.7s
full104_..._glm51_opus_v23_20260610_001 488/636 21210.3s
full104_..._glm51_opus_v23_rep2_20260611_001 471/636 23243.4s
 
# identical model, harness, and tasks; the only variable is who wrote the process.
Unsupervised.com — June 2026

Signal

The task-level picture is mixed: on 23 of 104 tasks the workflow condition beat raw in both replications, on 7 it lost in both, and 32 tasks moved in raw's favor at least once. The lift is a net effect, concentrated in a minority of tasks.

The gains cluster where you'd expect a process to help: tasks the raw model under-organizes, where it drifts, summarizes what it would do instead of doing it, or runs long and times out. The losses cluster where the scaffold gets in the model's way on work it already handles cleanly. That shape matches everything else we've measured: structure helps most exactly where the model helps itself least.

The mechanism: the workflow keeps the model on task, the raw run's timeout and drift disappear, and the tokens go to the work instead of the wandering. That is where the missing seven hours went.

Caveats

What we run next.

We trust the speed result. The score edge needs more evidence than two runs can give.

More raw controls. The 70.9% baseline is a single full-104 run. Repeat baselines tell us how much of that number is the model and how much is the day.

More workflow reps. The two workflow runs scored 488 and 471: a 17-point spread against a 28.5-point uplift, so today the true effect could run anywhere from marginal to roughly double the point estimate. More replications narrow that range.

A judge-variance study. Every criterion was scored once, by a single Gemini 3.1 Pro call we ran ourselves. Repeat-judging separates a few-point task delta from judge noise.

Ablations. The workflow condition also carried generic runner controls, and the workflows are task-specific and Opus-authored. Isolating those pieces turns “these workflows helped this model” into a statement about which part did the helping.

Self-authored workflows · running now. We're testing whether the author has to be a frontier model: GLM writing workflows for GLM to follow. If that holds, the whole loop runs on open weights.

More open-weights models · running now. The same protocol on the newly released GLM 5.2 and on MiniMax M3, to see whether the effect follows the approach or this one model.

None of this touches the speed. Two runs, both about 2.2× faster than raw, both with zero timeouts, is the finding we'd stake the post on.

Why it matters

Strip the caveats away and what's left still stands: a process a frontier model wrote, handed to a cheaper open-weights model, made that model finish the same work in less than half the time. As far as we can measure, it was no worse and probably a little better.

That's a claim about where agent quality comes from. When you want a fleet of agents to get better, the reflex is to buy bigger weights. But a chunk of what looks like model capability is really process, like planning before acting and checking outputs, and process is portable. It can be authored once by an expensive model and executed cheaply by a smaller one, across model generations, on infrastructure that doesn't care which weights are underneath.

There's a second mechanism we didn't use here at all: DeepWork can revise a workflow from the mistakes of its own runs. The idea is that consistency improves and token spend drops the more you use it. Measuring that loop with the same rigor is on the list above.

This is the same shape we keep finding: here, workflows cut GLM's zero-score failures by 29%; in a sibling experiment on this benchmark, Claude Code running under DeepWork cut its own by 36%, the sturdiest number this benchmark has given us. The durable value of borrowed process is dependability: an agent that finishes faster and fails less under the time and cost limits real work actually has. That part survives the next model release, and it is the part we're building on.