RunHaiku 4.5repeat 1
Orchestrator-workers on async vs defer on <script>
claude-haiku-4-5-20251001__orchestrator-workers__research-script-loading__r1
Score
1.000
- Cost
- $0.310
- + $0.089 judging
- Latency
- 1m 42s
- setup 0.0s, grading 4.6s
- Turns
- 55
- Tool calls
- 49
- Subagents
- 3
- Tokens in / out
- 502k / 18k
- 400,919 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- The task was decomposed: heldorchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
- The fan-out happened: heldorchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
- The synthesis wrote the answer: heldorchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
- Each call was shown the right tools: heldorchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Decompose | 18.3s | $0.012 | 2 | 0 | claude-haiku-4-5-20251001 |
| Worker 1: async-defer-classic | 35.2s | $0.070 | 13 | 12 | claude-haiku-4-5-20251001 |
| Worker 2: inline-scripts | 56.4s | $0.090 | 19 | 18 | claude-haiku-4-5-20251001 |
| Worker 3: module-scripts | 46.8s | $0.102 | 20 | 19 | claude-haiku-4-5-20251001 |
| Synthesize | 27.3s | $0.036 | 1 | 0 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies orchestrator-workers --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E2, E3. “The `async` attribute tells the browser to fetch the script in parallel with HTML parsing without blocking it.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E2, E6, E7. “`defer` scripts** execute *before* DOMContentLoaded fires. The event waits for all deferred scripts to complete.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E6, E7, E17. “the script executes **immediately and synchronously**, which blocks parsing at that moment.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E4, E5. “Multiple deferred scripts execute in the order they appear in the document”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E8, E9. “The `async` and `defer` attributes have **no effect** on inline scripts”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E11, E13, E14, E15. “The `defer` attribute has no effect on module scripts because they already defer by default.”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript","whatwg-html-scripting","webdev-optimize-resource-loading","mdn-js-modules"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript","webdev-optimize-resource-loading","mdn-js-modules"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"mdn-script-element":0.857,"webdev-third-party-javascript":1,"whatwg-html-scripting":0.2241,"webdev-optimize-resource-loading":0.9624,"mdn-domcontentloaded":1,"mdn-js-modules":0.5564}
- cited count
- 6
- verified count
- 6
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 17
- verified quote count
- 17
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The async attribute on external classic scripts causes the script to be fetched in parallel to parsing and evaluated as soon as it is available, potentially before parsing completes","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The defer attribute on external classic scripts causes the script to be fetched in parallel but evaluated after the page has finished parsing","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Without async or defer, classic scripts are fetched and evaluated immediately, blocking parsing","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts execute in the order they appear in the document","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts execute with no guaranteed execution order","quote":"scripts with an async attribute will execute as soon as the download is complete. This blocks the page and does not guarantee any specific execution order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded fires after deferred and module scripts have executed but does not wait for async scripts","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"...\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"document.readyState becomes interactive before defer/module scripts execute, and DOMContentLoaded fires after them but before async scripts","quote":"document.readyState is set to \"interactive\" after the completion of the HTML parser but before the execution of scripts with defer or type=\"module\". DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The async attribute has no effect on inline classic scripts","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The defer attribute has no effect on inline classic scripts","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Inline classic scripts execute immediately and block parsing","quote":"When using inline JavaScript, the parser is similarly blocked until the inline script is parsed and executed.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules","claim":"Module scripts are deferred by default without needing the defer attribute","quote":"There is no need to use the defer attribute (see <script> attributes) when loading a module script; modules are deferred automatically.","verified":true,"document":"mdn-js-modules"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Inline scripts with type=module are deferred automatically","quote":"async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The defer attribute has no effect on module scripts","quote":"The defer attribute has no effect on module scripts.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"For module scripts with async, the module and all dependencies are fetched in parallel and evaluated as soon as available","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"For module scripts without async, the module and dependencies are fetched in parallel but evaluated when the page has finished parsing","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules","claim":"Module scripts are only executed once even if referenced in multiple script tags","quote":"Modules are only executed once, even if they have been referenced in multiple <script> tags.","verified":true,"document":"mdn-js-modules"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can interrupt DOM building if they finish downloading while the parser is still working","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E2","E6","E7"]},"async-timing":{"status":"met","support":["E1","E6","E7","E17"]},"ordering":{"status":"met","support":["E4","E5"]},"inline-ignored":{"status":"met","support":["E8","E9"]},"modules":{"status":"met","support":["E11","E13","E14","E15"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E2","E6","E7"]},"async-timing":{"status":"met","support":["E1","E6","E17"]},"ordering":{"status":"met","support":["E4","E5"]},"inline-ignored":{"status":"met","support":["E8","E9"]},"modules":{"status":"met","support":["E11","E13","E14","E15"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E2","E6","E7"]},"async-timing":{"status":"met","support":["E1","E6","E17"]},"ordering":{"status":"met","support":["E4","E5"]},"inline-ignored":{"status":"met","support":["E8","E9"]},"modules":{"status":"met","support":["E11","E13","E14","E15"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
The HTML async and defer Attributes on Script Elements
The async and defer attributes are boolean attributes that control how and when JavaScript executes relative to HTML parsing. Their behavior depends critically on whether the script is a classic external script, an inline script, or a module script.
Classic External Scripts with src
What async Does
The async attribute tells the browser to fetch the script in parallel with HTML parsing without blocking it. However, once the download completes, the script executes immediately and synchronously, which blocks parsing at that moment. Multiple async scripts execute as soon as each finishes downloading, with no guaranteed order—they may execute out of sequence relative to their appearance in the HTML.
What defer Does
The defer attribute tells the browser to fetch the script in parallel with HTML parsing without blocking it, but to delay execution until after the HTML has been completely parsed. Multiple deferred scripts execute in the order they appear in the document, providing a guaranteed execution order that preserves dependencies.
Interaction with HTML Parsing
Without attributes (synchronous): The script is parser-blocking. The browser stops parsing, fetches and executes the script immediately, then resumes parsing.
With async: Parsing continues while the script downloads. However, when the download completes, execution blocks parsing until the script finishes. Because download time is unpredictable, this interruption can occur at any point during parsing.
With defer: Parsing continues uninterrupted. The script waits in the background until parsing is completely finished, then executes.
Execution Relative to DOMContentLoaded
The DOMContentLoaded event fires when HTML parsing is complete and all deferred scripts have executed. Critically:
deferscripts execute before DOMContentLoaded fires. The event waits for all deferred scripts to complete.asyncscripts execute at unpredictable times relative to DOMContentLoaded. They may execute before or after it. DOMContentLoaded does not wait for async scripts.
Execution Order Guarantees
defer: Multiple deferred scripts execute in the order they appear in the HTML, making them suitable for scripts with dependencies.
async: There is no guaranteed order. Multiple async scripts may execute in any sequence, making them suitable only for independent scripts.
Inline Classic Scripts (No src)
Inline scripts (script elements without a src attribute) execute immediately and synchronously during HTML parsing, blocking the parser until execution completes. The async and defer attributes have no effect on inline scripts—they are ignored by the browser. Inline scripts function as if they have no attributes, executing as part of the normal parsing flow before DOMContentLoaded fires.
Module Scripts (type="module")
Module scripts behave fundamentally differently from classic scripts.
Default Behavior
Module scripts are deferred by default without needing the defer attribute. Both external and inline module scripts:
- Are fetched in parallel to parsing (for external scripts)
- Execute after HTML parsing is completely finished, before DOMContentLoaded fires
- Execute in the order they appear in the document
- Are executed only once, even if referenced multiple times
The defer attribute has no effect on module scripts because they already defer by default.
With async
If async is specified on a module script, the module and all its dependencies are fetched in parallel and executed as soon as they become available—potentially before parsing completes. This means async module scripts can interrupt the parser, unlike the default module behavior.
Inline Module Scripts
Inline module scripts (<script type="module"> without src) also defer execution by default until after parsing completes. They execute in order with other module scripts and before DOMContentLoaded fires.
Evidence
{
"evidence": [
{
"claim": "The async attribute on external classic scripts causes the script to be fetched in parallel to parsing and evaluated as soon as it is available, potentially before parsing completes",
"quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "The defer attribute on external classic scripts causes the script to be fetched in parallel but evaluated after the page has finished parsing",
"quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "Without async or defer, classic scripts are fetched and evaluated immediately, blocking parsing",
"quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "Deferred scripts execute in the order they appear in the document",
"quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Async scripts execute with no guaranteed execution order",
"quote": "scripts with an async attribute will execute as soon as the download is complete. This blocks the page and does not guarantee any specific execution order.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "DOMContentLoaded fires after deferred and module scripts have executed but does not wait for async scripts",
"quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"...\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.",
"url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"
},
{
"claim": "document.readyState becomes interactive before defer/module scripts execute, and DOMContentLoaded fires after them but before async scripts",
"quote": "document.readyState is set to \"interactive\" after the completion of the HTML parser but before the execution of scripts with defer or type=\"module\". DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.",
"url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"
},
{
"claim": "The async attribute has no effect on inline classic scripts",
"quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "The defer attribute has no effect on inline classic scripts",
"quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Inline classic scripts execute immediately and block parsing",
"quote": "When using inline JavaScript, the parser is similarly blocked until the inline script is parsed and executed.",
"url": "https://web.dev/learn/performance/optimize-resource-loading"
},
{
"claim": "Module scripts are deferred by default without needing the defer attribute",
"quote": "There is no need to use the defer attribute (see <script> attributes) when loading a module script; modules are deferred automatically.",
"url": "https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules"
},
{
"claim": "Inline scripts with type=module are deferred automatically",
"quote": "async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.",
"url": "https://web.dev/learn/performance/optimize-resource-loading"
},
{
"claim": "The defer attribute has no effect on module scripts",
"quote": "The defer attribute has no effect on module scripts.",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "For module scripts with async, the module and all dependencies are fetched in parallel and evaluated as soon as available",
"quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "For module scripts without async, the module and dependencies are fetched in parallel but evaluated when the page has finished parsing",
"quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "Module scripts are only executed once even if referenced in multiple script tags",
"quote": "Modules are only executed once, even if they have been referenced in multiple <script> tags.",
"url": "https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules"
},
{
"claim": "Async scripts can interrupt DOM building if they finish downloading while the parser is still working",
"quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.",
"url": "https://web.dev/articles/efficiently-load-third-party-javascript"
}
]
}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.