RunSonnet 5.5repeat 1
Propose-select on async vs defer on <script>
claude-sonnet-5-5__propose-select__research-script-loading__r1
Score
1.000
- Cost
- $0.605
- + $0.076 judging
- Latency
- 1m 40s
- setup 0.0s, grading 5.2s
- Turns
- 44
- Tool calls
- 39
- Subagents
- 3
- Tokens in / out
- 327k / 26k
- 242,568 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- All three proposers ran: heldpropose-select.proposalsThree proposers ran in parallel, each from its own angle, and none was starved while another ran.
- The selector chose: heldpropose-select.selectionThe selector picked a proposal by a valid number, read from its schema reply and not a default.
- Each call was shown the right tools: heldpropose-select.toolsEach proposer was shown the read-only tools, the selector none, and the executor the full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Proposal 1 (policy-first) | 37.2s | $0.151 | 14 | 13 | claude-sonnet-5-5 |
| Proposal 2 (state-first) | 53.4s | $0.170 | 16 | 15 | claude-sonnet-5-5 |
| Proposal 3 (outcome-first) | 36.2s | $0.134 | 12 | 11 | claude-sonnet-5-5 |
| Select | 46.2s | $0.149 | 2 | 0 | claude-sonnet-5-5 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies propose-select --tasks research-script-loading --models claude-sonnet-5-5 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E2, E3. “`async` and `defer` both let the browser fetch an external script while it keeps parsing the HTML.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E3, E5. “Deferred scripts run after parsing and before the event, and the event waits for them.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E2, E8, E9. “An async script can therefore run before the event (fast download) or after it (slow download).”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E6, E7. “Deferred scripts run in document order, whichever download finishes first.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E11. “`async` and `defer` have no effect on them.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E4, E16. “The `defer` attribute has no effect on them, and MDN's modules guide says it is unnecessary.”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded","webdev-third-party-javascript","mdn-js-modules"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript","mdn-js-modules"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"mdn-script-element":0.8439,"whatwg-html-scripting":0.4479,"mdn-domcontentloaded":1,"webdev-third-party-javascript":0.9618,"webdev-optimize-resource-loading":0.0354,"mdn-js-modules":0.0058}
- cited count
- 4
- verified count
- 4
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 18
- verified quote count
- 18
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Per the spec, a classic external script with neither async nor defer is fetched and run immediately, blocking the parser.","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"An async classic script is fetched in parallel and runs as soon as it is available, possibly before parsing completes.","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"A defer classic script is fetched in parallel but evaluated once the page has finished parsing.","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts are deferred by default, and defer has no effect on them.","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts execute after parsing but before DOMContentLoaded, and they hold up that event.","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts keep document order.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts carry no ordering guarantee.","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded does not wait for async scripts.","quote":"It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"An async script may run after DOMContentLoaded has already fired, so code should check readyState.","quote":"If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"MDN describes the typical sequence: deferred and module scripts, then DOMContentLoaded, then async scripts. This is narrower than the spec's 'potentially before parsing completes'.","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async and defer have no effect on inline classic scripts.","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Inline classic scripts run immediately and block the parser.","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"If both async and defer are given, the element behaves as async only.","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The spec allows defer alongside async as a fallback for legacy browsers.","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"In the spec's algorithm, deferred classic scripts and module scripts share the list that runs when parsing finishes.","quote":"Otherwise, if el has a defer attribute or el's type is \"module\":","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules","claim":"Modules are deferred automatically, so defer is unnecessary on them.","quote":"There is no need to use the defer attribute (see <script> attributes) when loading a module script; modules are deferred automatically.","verified":true,"document":"mdn-js-modules"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Inline module scripts never execute synchronously with parsing, because a task is queued.","quote":"Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts use CORS for cross-origin fetching, unlike classic scripts.","quote":"Unlike classic scripts, module scripts require the use of the CORS protocol for cross-origin fetching.","verified":true,"document":"whatwg-html-scripting"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E3","E5"]},"async-timing":{"status":"met","support":["E2","E8","E9"]},"ordering":{"status":"met","support":["E6","E7"]},"inline-ignored":{"status":"met","support":["E11"]},"modules":{"status":"met","support":["E4","E16"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E3","E5"]},"async-timing":{"status":"met","support":["E2","E8","E9"]},"ordering":{"status":"met","support":["E6","E7"]},"inline-ignored":{"status":"met","support":["E11"]},"modules":{"status":"met","support":["E4","E16"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E3","E5"]},"async-timing":{"status":"met","support":["E2","E8","E9"]},"ordering":{"status":"met","support":["E6","E7"]},"inline-ignored":{"status":"met","support":["E11"]},"modules":{"status":"met","support":["E4","E16"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
Short answer. async and defer both let the browser fetch an external script while it keeps parsing the HTML. They differ in when the script runs and whether order is kept. An async script runs as soon as it has downloaded, in no guaranteed order, possibly before parsing finishes. A defer script runs after parsing finishes and before DOMContentLoaded, in document order.
Plain classic external script (no attribute). The spec says: "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete." (https://html.spec.whatwg.org/multipage/scripting.html)
async on a classic external script
- Parsing: The script is fetched in parallel with parsing and evaluated "as soon as it is available (potentially before parsing completes)". Download never blocks the parser, but execution interrupts it if the parser is still running. MDN adds that a running async script "blocks the page from rendering".
DOMContentLoaded: There is no fixed relationship. The spec puts async scripts in a "set of scripts that will execute as soon as possible". The event does not wait for them: MDN says it "doesn't wait for other things like images, subframes, and async scripts to finish loading". An async script can therefore run before the event (fast download) or after it (slow download). Code inside an async script should checkdocument.readyStateinstead of assuming the event is still pending. MDN's note that the event fires "before the execution of scripts with async" describes the typical sequence, not a guarantee. The spec's "potentially before parsing completes" is broader, so I treat the spec as authoritative.- Order: None is guaranteed. MDN says "You get no guarantee that scripts will run in any specific order." Each async script runs when its own download finishes, so async suits independent scripts such as analytics.
defer on a classic external script
- Parsing: The script is fetched in parallel and "evaluated when the page has finished parsing". In the spec's algorithm it is appended to the document's "list of scripts that will execute when the document has finished parsing".
DOMContentLoaded: Deferred scripts run after parsing and before the event, and the event waits for them.document.readyStateis"interactive"while they run.- Order: Deferred scripts run in document order, whichever download finishes first. Use defer when scripts depend on each other or on the DOM.
Both attributes together. MDN says the element acts as if only async were set. The spec allows defer alongside async so that legacy browsers that only know defer fall back to deferring instead of blocking.
Inline classic scripts. async and defer have no effect on them. MDN says the attributes must not be used without src, and that in that case they "would have no effect". An inline classic script runs immediately and blocks the parser. In the spec's algorithm, only a classic script with src, or a module script, takes the async or defer path.
type="module" scripts
- Default timing: Module scripts are deferred by default. The module and its dependencies are fetched in parallel with parsing and evaluated when parsing finishes. They share the deferred list, so they run in document order and before
DOMContentLoaded. Thedeferattribute has no effect on them, and MDN's modules guide says it is unnecessary. - With
async:asyncdoes apply to modules. The module and all its dependencies run as soon as they are available, possibly before parsing completes, in no guaranteed order. - Inline modules: Unlike inline classic scripts, inline modules also follow the defer or async behaviour.
asyncis allowed on them;deferis not. The spec queues a task, so an inline module never runs synchronously in the middle of parsing. - CORS: Cross-origin module fetches require CORS, unlike classic scripts.
{"evidence": [
{"claim": "Per the spec, a classic external script with neither async nor defer is fetched and run immediately, blocking the parser.", "quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "An async classic script is fetched in parallel and runs as soon as it is available, possibly before parsing completes.", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "A defer classic script is fetched in parallel but evaluated once the page has finished parsing.", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Module scripts are deferred by default, and defer has no effect on them.", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Deferred scripts execute after parsing but before DOMContentLoaded, and they hold up that event.", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Deferred scripts keep document order.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Async scripts carry no ordering guarantee.", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded does not wait for async scripts.", "quote": "It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "An async script may run after DOMContentLoaded has already fired, so code should check readyState.", "quote": "If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "MDN describes the typical sequence: deferred and module scripts, then DOMContentLoaded, then async scripts. This is narrower than the spec's 'potentially before parsing completes'.", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "async and defer have no effect on inline classic scripts.", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Inline classic scripts run immediately and block the parser.", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "If both async and defer are given, the element behaves as async only.", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "The spec allows defer alongside async as a fallback for legacy browsers.", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "In the spec's algorithm, deferred classic scripts and module scripts share the list that runs when parsing finishes.", "quote": "Otherwise, if el has a defer attribute or el's type is \"module\":", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Modules are deferred automatically, so defer is unnecessary on them.", "quote": "There is no need to use the defer attribute (see <script> attributes) when loading a module script; modules are deferred automatically.", "url": "https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules"},
{"claim": "Inline module scripts never execute synchronously with parsing, because a task is queued.", "quote": "Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Module scripts use CORS for cross-origin fetching, unlike classic scripts.", "quote": "Unlike classic scripts, module scripts require the use of the CORS protocol for cross-origin fetching.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.