Architecture
Orchestrator-workers
Does splitting a task across parallel contexts pay for the coordination?
An orchestrator splits the task, workers run the parts in parallel contexts, and a final call merges their reports.
How it runs
1The orchestrator
no tools
Splits the task into two to four independent subtasks through a structured reply, or one worker with the whole task when it judges the task cannot honestly be split.
2The workers
full toolsetruns in parallel
One worker per subtask, started together in separate contexts with the full toolset. They cannot see each other, so an overlap or a dependency the split missed shows up in the score.
3The synthesis
no tools
Writes the final answer from the workers’ reports. It holds no tools, so it can only merge what the workers found.
Where it can fail quietly
If the split never happens the design is one worker with an orchestrator around it, and the score looks like a single agent’s.
What the structure buys
Orchestrator-workers against a single agent on the same tasks and repeats, on each model. The interval is on the paired difference, so it is as wide as the evidence is thin.
- Haiku 4.5=1 of 6
- Score
- 0.701
- 0.573–0.824
- Cost / run
- $0.171
- 2.0× single
- Ranked
- 50/50
not separable from single−0.070 [−0.152, −0.001]too few tasks carry the spread to show equivalence
- Sonnet 5.5=1 of 6
- Score
- 0.924
- 0.883–0.962
- Cost / run
- $0.216
- 1.8× single
- Ranked
- 50/50
not separable from single+0.025 [−0.005, +0.078]too few tasks carry the spread to show equivalence
- Opus 5.5=1 of 6
- Score
- 0.921
- 0.869–0.961
- Cost / run
- $0.459
- 1.8× single
- Ranked
- 50/50
not separable from single−0.019 [−0.051, +0.006]too few tasks carry the spread to show equivalence
Task by task against a single agent
A mean can hide whether a design helps everywhere a little or on one task a lot. Each bar is the difference from a single agent on one task; the best-served tasks are at the top.
Where the money goes
A run of Orchestrator-workers is 3 stages. This is what each one costs, so the price of the structure is separate from the price of the work.
- Haiku 4.5$0.171 per run · 50 runs
- Sonnet 5.5$0.216 per run · 50 runs
- Opus 5.5$0.459 per run · 50 runs
| Stage | Haiku 4.5 | Sonnet 5.5 | Opus 5.5 |
|---|---|---|---|
| The orchestrator | $0.01811% | $0.0147% | $0.0327% |
| The workers | $0.13981% | $0.16777% | $0.34776% |
| The synthesis | $0.0148% | $0.03516% | $0.08017% |
Did it run as designed?
Each check is recorded with every run. A run that fails one is excluded from every ranking and counted, never scored.
The design’s own checks
The task was decomposed
orchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
150/150 100%The fan-out happened
orchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
150/150 100%The synthesis wrote the answer
orchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
150/150 100%Each call was shown the right tools
orchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.
150/150 100%
The harness’s checksall 10 held in every run
No API failures
harness.apiNo model call failed at the API.
150/150 100%Builtin tools
harness.builtin-toolsA check recorded with the run.
150/150 100%Every call ended
harness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
150/150 100%No harness tool denied
harness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
150/150 100%Judge answered
harness.judgeAt least one grading call returned a usable verdict.
18/18 100%Tool servers connected
harness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
150/150 100%No foreign tool servers
harness.mcp-hermeticNo call loaded a tool server the harness did not give it.
150/150 100%No silent fallbacks
harness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
150/150 100%Run completed
harness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
150/150 100%Structured replies arrived
harness.structured-outputEvery call that asked for structured output and ran to the end got it.
150/150 100%
A run of it
Orchestrator-workers on Cancel a fare the customer wrongly believes is refundable to their card with Haiku 4.5, repeat 1. It scored 0.429.
3 calls · 1m 21s · $0.068
- tool call
- denied by the guard
- tool returned an error
- shown no tools (reasoning only)
- lost its tools (server failed)
- failed
150 ranked runs of 150 recorded.
Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.