RunOpus 5.5repeat 1
Orchestrator-workers on async vs defer on <script>
claude-opus-5-5__orchestrator-workers__research-script-loading__r1
Score
0.906
- Cost
- $0.777
- + $0.086 judging
- Latency
- 1m 43s
- setup 0.0s, grading 4.7s
- Turns
- 37
- Tool calls
- 31
- Subagents
- 3
- Tokens in / out
- 231k / 16k
- 168,076 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- The task was decomposed: heldorchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
- The fan-out happened: heldorchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
- The synthesis wrote the answer: heldorchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
- Each call was shown the right tools: heldorchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Decompose | 16.3s | $0.042 | 2 | 0 | claude-opus-5-5 |
| Worker 1: parsing-domcontentloaded | 46.0s | $0.245 | 13 | 12 | claude-opus-5-5 |
| Worker 2: execution-order | 33.6s | $0.169 | 7 | 6 | claude-opus-5-5 |
| Worker 3: module-scripts | 36.2s | $0.155 | 14 | 13 | claude-opus-5-5 |
| Synthesize | 40.7s | $0.167 | 1 | 0 | claude-opus-5-5 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies orchestrator-workers --tasks research-script-loading --models claude-opus-5-5 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E2, E3. “the script downloads in parallel with parsing and does not block it.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E7, E8. “deferred scripts run after parsing and before DOMContentLoaded fires.”
- partialStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.0.50weight 0.19
partial, backed by verified evidence E2, E12. “My conclusion that an async script can run either before or after DOMContentLoaded is an inference from that missing guarantee, not a quoted statement.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E9, E10. “there is no ordering guarantee. MDN says "You get no guarantee that scripts will run in any specific order."”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E14, E15. “`async` and `defer` have no effect on inline classic scripts.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E17, E18. “"The defer attribute has no effect on module scripts — they defer by default."”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["whatwg-html-scripting","mdn-script-element","webdev-optimize-resource-loading","webdev-third-party-javascript"]
- opened by role
- {"primary":["whatwg-html-scripting"],"secondary":["mdn-script-element","webdev-optimize-resource-loading","webdev-third-party-javascript"]}
- displayed share
- {"mdn-script-element":0.9706,"whatwg-html-scripting":0.2674,"mdn-js-modules":0.0334,"mdn-domcontentloaded":0.235,"webdev-optimize-resource-loading":0.4044,"webdev-third-party-javascript":0.9285}
- cited count
- 4
- verified count
- 4
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 20
- verified quote count
- 20
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"An external classic script with neither async nor defer is fetched and executed immediately, blocking parsing.","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"An async classic script is fetched in parallel and runs as soon as available, possibly before parsing completes.","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"A defer classic script (without async) is fetched in parallel and runs once parsing has finished.","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The spec keeps a per-document list of scripts to run when parsing finishes (used for defer and module scripts).","quote":"Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Async scripts go into an unordered set executed as soon as possible.","quote":"Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Specifying defer together with async lets legacy browsers fall back to defer rather than blocking.","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Deferred scripts run after parsing is completely finished but before DOMContentLoaded.","quote":"Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts delay DOMContentLoaded until they have loaded and run.","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Multiple deferred scripts execute in document order.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts give no ordering guarantee.","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts run at the first opportunity after downloading and are only guaranteed to run before the window load event.","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can interrupt DOM building if they finish downloading during parsing.","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"If both async and defer are present, the element behaves as async.","quote":"If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer has no effect on inline (no src) scripts.","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Inline classic scripts and attribute-less external classic scripts are executed immediately before parsing continues.","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Module scripts, including inline ones, are deferred automatically and do not block the parser.","quote":"async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer has no effect on module scripts because they defer by default.","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async on a module script fetches it and its dependencies in parallel and runs it as soon as available.","quote":"For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"One web.dev page describes deferred scripts as running at the same time as DOMContentLoaded, looser wording than other sources.","quote":"Scripts loaded with async are parsed and executed immediately once downloaded, while scripts loaded with defer are executed when HTML document parsing is finished—this occurs at the same time as the browser's DOMContentLoaded event.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Script-inserted scripts behave like async, while module scripts are deferred by default.","quote":"Note: Scripts loaded using the type=\"module\" attribute are deferred by default, while scripts loaded by injecting <script> markup into the DOM with JavaScript behave like async scripts.","verified":true,"document":"webdev-optimize-resource-loading"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E7","E8"]},"async-timing":{"status":"partial","support":["E2","E12"]},"ordering":{"status":"met","support":["E9","E10"]},"inline-ignored":{"status":"met","support":["E14","E15"]},"modules":{"status":"met","support":["E17","E18"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E7","E8"]},"async-timing":{"status":"partial","support":["E2","E12"]},"ordering":{"status":"met","support":["E9","E10"]},"inline-ignored":{"status":"partial","support":["E14","E15"]},"modules":{"status":"met","support":["E17","E18","E16"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E7","E8"]},"async-timing":{"status":"partial","support":["E2","E12","E11"]},"ordering":{"status":"met","support":["E9","E10"]},"inline-ignored":{"status":"met","support":["E14"]},"modules":{"status":"met","support":["E17","E18","E16"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
How async and defer work on <script>
The baseline: an external classic script with neither attribute
An external classic script with neither attribute blocks the parser. The WHATWG HTML standard says: "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete." (https://html.spec.whatwg.org/multipage/scripting.html). web.dev adds that rendering is blocked as well as parsing (https://web.dev/learn/performance/optimize-resource-loading).
defer on a classic external script
- Parsing: the script downloads in parallel with parsing and does not block it. It runs only after the document has finished parsing (https://html.spec.whatwg.org/multipage/scripting.html).
- Spec mechanism: a parser-inserted deferred script is added to the document's "list of scripts that will execute when the document has finished parsing." The same list also holds module scripts.
- DOMContentLoaded: deferred scripts run after parsing and before DOMContentLoaded fires. They also hold the event back until they have run. MDN and web.dev both say this (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script, https://web.dev/articles/efficiently-load-third-party-javascript).
- Caveat on the spec: the spec's parsing section ("the end"), which defines this step, returned HTTP 404 to my researcher. So the timing relative to DOMContentLoaded is supported by MDN and web.dev, not by the spec text itself.
- Order: several deferred scripts run in the order they appear in the document (MDN; web.dev).
async on a classic external script
- Parsing: the script downloads in parallel with parsing and runs as soon as it is available, "potentially before parsing completes" (https://html.spec.whatwg.org/multipage/scripting.html). If it finishes downloading while the parser is still working, running it interrupts DOM construction (https://web.dev/articles/efficiently-load-third-party-javascript).
- Spec mechanism: async scripts go into an unordered set of "scripts that will execute as soon as possible."
- Order: there is no ordering guarantee. MDN says "You get no guarantee that scripts will run in any specific order." web.dev says async scripts are likely to run out of document order.
- DOMContentLoaded: none of the sources ties async scripts to DOMContentLoaded. web.dev only guarantees that they run before the window's
loadevent. My conclusion that an async script can run either before or after DOMContentLoaded is an inference from that missing guarantee, not a quoted statement.
Both attributes on the same script
If both are present, async wins. MDN says the element acts as if only async were specified. The spec explains why you would write both: defer lets legacy browsers that support only defer fall back to deferring instead of blocking (https://html.spec.whatwg.org/multipage/scripting.html).
One researcher, who was not allowed to open the spec, reported this fallback point as unsourced. Another researcher did open the spec and quoted it, so it is sourced.
Inline classic scripts (no src)
async and defer have no effect on inline classic scripts. MDN warns against using either attribute there. Inline classic scripts run immediately and block the parser (MDN; https://web.dev/learn/performance/optimize-resource-loading).
type="module" scripts
- Deferred by default: module scripts are deferred automatically, and web.dev says this applies to inline module scripts too. They do not block the parser (https://web.dev/learn/performance/optimize-resource-loading).
deferdoes nothing: MDN says "The defer attribute has no effect on module scripts — they defer by default."asyncdoes change behaviour: withasync, a module "and all their dependencies" are fetched in parallel to parsing and run as soon as they are available (MDN).- Inline modules don't run synchronously: one researcher reported that the spec queues a task before running an inline module. They gave no quote for this, so I treat it as unverified.
Gaps and disagreements
Unconfirmed points. I could not confirm these with any opened source:
- that
asyncspecifically works on inline module scripts; - that a module included several times runs only once.
The javascript.info and v8.dev pages that were meant to cover these returned 404.
Wording differences.
- web.dev's two pages: https://web.dev/learn/performance/optimize-resource-loading says deferred scripts run "at the same time as" DOMContentLoaded. The other web.dev article and MDN's attribute reference say "before". "Before" is the more precise statement.
- MDN's Examples section: it says defer scripts "load" in page order, which means execute. It also says they run once "everything has finished loading", which conflicts with MDN's own attribute section. I relied on the attribute section.
Script-inserted scripts. web.dev notes that scripts injected into the DOM with JavaScript behave like async scripts.
{"evidence": [
{"claim": "An external classic script with neither async nor defer is fetched and executed immediately, blocking parsing.", "quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "An async classic script is fetched in parallel and runs as soon as available, possibly before parsing completes.", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "A defer classic script (without async) is fetched in parallel and runs once parsing has finished.", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "The spec keeps a per-document list of scripts to run when parsing finishes (used for defer and module scripts).", "quote": "Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Async scripts go into an unordered set executed as soon as possible.", "quote": "Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Specifying defer together with async lets legacy browsers fall back to defer rather than blocking.", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Deferred scripts run after parsing is completely finished but before DOMContentLoaded.", "quote": "Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "Deferred scripts delay DOMContentLoaded until they have loaded and run.", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Multiple deferred scripts execute in document order.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Async scripts give no ordering guarantee.", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Async scripts run at the first opportunity after downloading and are only guaranteed to run before the window load event.", "quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "Async scripts can interrupt DOM building if they finish downloading during parsing.", "quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "If both async and defer are present, the element behaves as async.", "quote": "If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer has no effect on inline (no src) scripts.", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Inline classic scripts and attribute-less external classic scripts are executed immediately before parsing continues.", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Module scripts, including inline ones, are deferred automatically and do not block the parser.", "quote": "async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.", "url": "https://web.dev/learn/performance/optimize-resource-loading"},
{"claim": "defer has no effect on module scripts because they defer by default.", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "async on a module script fetches it and its dependencies in parallel and runs it as soon as available.", "quote": "For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "One web.dev page describes deferred scripts as running at the same time as DOMContentLoaded, looser wording than other sources.", "quote": "Scripts loaded with async are parsed and executed immediately once downloaded, while scripts loaded with defer are executed when HTML document parsing is finished—this occurs at the same time as the browser's DOMContentLoaded event.", "url": "https://web.dev/learn/performance/optimize-resource-loading"},
{"claim": "Script-inserted scripts behave like async, while module scripts are deferred by default.", "quote": "Note: Scripts loaded using the type=\"module\" attribute are deferred by default, while scripts loaded by injecting <script> markup into the DOM with JavaScript behave like async scripts.", "url": "https://web.dev/learn/performance/optimize-resource-loading"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.