Architecture
Propose-select
Does choosing among independent proposals beat committing to the first idea?
Three proposers investigate in parallel with read-only tools, a selector picks one plan, and an executor applies it.
How it runs
1The proposers
read-only toolsruns in parallel
Three proposers investigate at the same time with read-only tools, each starting from a different angle: the rules, the current state, or the outcome the person needs. Nothing is changed, and they never see one another.
2The selector
no tools
Reads the three proposals, picks the strongest and merges in what the others got right, through a structured reply. It holds no tools, so it chooses on the proposals alone.
3The executor
full toolset
Applies the chosen course of action with the full toolset. Until this call nothing in the world has changed.
Where it can fail quietly
A proposer that starts without its tools can only guess, and a selector that falls back to proposal 1 turns three investigations into one.
What the structure buys
Propose-select against a single agent on the same tasks and repeats, on each model. The interval is on the paired difference, so it is as wide as the evidence is thin.
- Haiku 4.5=1 of 6
- Score
- 0.809
- 0.711–0.901
- Cost / run
- $0.373
- 4.3× single
- Ranked
- 50/50
not separable from single+0.038 [−0.018, +0.094]
- Sonnet 5.5=1 of 6
- Score
- 0.939
- 0.903–0.972
- Cost / run
- $0.437
- 4.1× single
- Ranked
- 50/50
not separable from single+0.040 [+0.003, +0.095]too few tasks carry the spread to show equivalence
- Opus 5.5=1 of 6
- Score
- 0.958
- 0.925–0.985
- Cost / run
- $0.928
- 4.1× single
- Ranked
- 50/50
not separable from single+0.018 [0.000, +0.043]too few tasks carry the spread to show equivalence
Task by task against a single agent
A mean can hide whether a design helps everywhere a little or on one task a lot. Each bar is the difference from a single agent on one task; the best-served tasks are at the top.
Where the money goes
A run of Propose-select is 3 stages. This is what each one costs, so the price of the structure is separate from the price of the work.
- Haiku 4.5$0.373 per run · 50 runs
- Sonnet 5.5$0.437 per run · 50 runs
- Opus 5.5$0.928 per run · 50 runs
| Stage | Haiku 4.5 | Sonnet 5.5 | Opus 5.5 |
|---|---|---|---|
| The proposers | $0.24866% | $0.29066% | $0.61967% |
| The selector | $0.05916% | $0.05613% | $0.11913% |
| The executor | $0.06618% | $0.09121% | $0.19121% |
Did it run as designed?
Each check is recorded with every run. A run that fails one is excluded from every ranking and counted, never scored.
The design’s own checks
All three proposers ran
propose-select.proposalsThree proposers ran in parallel, each from its own angle, and none was starved while another ran.
150/150 100%The selector chose
propose-select.selectionThe selector picked a proposal by a valid number, read from its schema reply and not a default.
150/150 100%Each call was shown the right tools
propose-select.toolsEach proposer was shown the read-only tools, the selector none, and the executor the full toolset.
150/150 100%
The harness’s checksall 10 held in every run
No API failures
harness.apiNo model call failed at the API.
150/150 100%Builtin tools
harness.builtin-toolsA check recorded with the run.
150/150 100%Every call ended
harness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
150/150 100%No harness tool denied
harness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
150/150 100%Judge answered
harness.judgeAt least one grading call returned a usable verdict.
18/18 100%Tool servers connected
harness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
150/150 100%No foreign tool servers
harness.mcp-hermeticNo call loaded a tool server the harness did not give it.
150/150 100%No silent fallbacks
harness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
150/150 100%Run completed
harness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
150/150 100%Structured replies arrived
harness.structured-outputEvery call that asked for structured output and ran to the end got it.
150/150 100%
A run of it
Propose-select on Cancel a fare the customer wrongly believes is refundable to their card with Haiku 4.5, repeat 1. It passed.
5 calls · 2m 21s · $0.199
- tool call
- denied by the guard
- tool returned an error
- shown no tools (reasoning only)
- lost its tools (server failed)
- failed
150 ranked runs of 150 recorded.
Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.