RunHaiku 4.5repeat 2
Plan-execute on async vs defer on <script>
claude-haiku-4-5-20251001__plan-execute__research-script-loading__r2
Score
1.000
- Cost
- $0.120
- + $0.084 judging
- Latency
- 1m 25s
- setup 0.0s, grading 5.4s
- Turns
- 18
- Tool calls
- 14
- Subagents
- 0
- Tokens in / out
- 210k / 8.2k
- 171,542 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- The plan was a plan: heldplan-execute.planThe planner’s reply was read through its schema and held at least one step.
- Each call was shown the right tools: heldplan-execute.toolsThe planner was shown no tools and the executor the task’s full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Plan | 24.7s | $0.020 | 3 | 0 | claude-haiku-4-5-20251001 |
| Execute | 1m 1s | $0.100 | 15 | 14 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies plan-execute --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E4. “The `defer` attribute also allows the browser to fetch the script in parallel to parsing”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E4, E6. “Deferred scripts execute after HTML parsing completes but before the `DOMContentLoaded` event fires”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E3, E7. “async scripts execute independently whenever they finish downloading and may fire well before, during, or after the DOMContentLoaded event”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E8, E9, E10. “scripts with the `defer` attribute execute in the exact order they appear in the document”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E2, E5. “Inline scripts (where JavaScript code is directly embedded in the HTML without a `src` attribute) are unaffected by both `async` and `defer` attributes.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E11, E12. “The `defer` attribute has no effect on module scripts because they already defer automatically. However, the `async` attribute can be applied to module scripts”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","mdn-domcontentloaded","whatwg-html-scripting","webdev-third-party-javascript","webdev-optimize-resource-loading"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript","webdev-optimize-resource-loading"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"webdev-third-party-javascript":0.9618,"mdn-script-element":0.8439,"whatwg-html-scripting":0.2175,"webdev-optimize-resource-loading":0.9624,"mdn-domcontentloaded":1,"mdn-js-modules":0.0056}
- cited count
- 5
- verified count
- 5
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 15
- verified quote count
- 14
- verified quote rate
- 0.9333333333333333 (93%)
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async attribute on external classic scripts causes them to be fetched in parallel to parsing and evaluated as soon as available","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async attribute cannot be used on inline scripts","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"async scripts can interrupt DOM building and rendering","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML. It also means they can interrupt DOM building if they \n…[truncated 53 chars]","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"defer attribute causes scripts to be fetched in parallel but executed after document parsing completes","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer attribute cannot be used on inline scripts","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute before DOMContentLoaded event fires, and browser waits for them","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event. Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished e\n…[truncated 10 chars]","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded does not wait for async scripts","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async scripts have no guaranteed execution order","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute in the order they appear in the document","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"defer attribute execution order is preserved regardless of download timing","quote":"defer ensures that scripts run in the order they appear in the HTML and don't block the parser.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"module scripts defer by default","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"module scripts without async are fetched in parallel but executed after parsing completes","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async module scripts execute as soon as available without waiting for parsing","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":false,"document":"mdn-script-element","failure":"the quote does not occur in the page"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"inline scripts with type=module behave differently from classic inline scripts","quote":"async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"scripts without async, defer, or type=module are fetched and executed immediately","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E6"]},"async-timing":{"status":"met","support":["E1","E3","E7"]},"ordering":{"status":"met","support":["E8","E9","E10"]},"inline-ignored":{"status":"met","support":["E2","E5"]},"modules":{"status":"met","support":["E11","E12","E13"]}},{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E6"]},"async-timing":{"status":"met","support":["E1","E3","E7"]},"ordering":{"status":"met","support":["E8","E9","E10"]},"inline-ignored":{"status":"met","support":["E2","E5"]},"modules":{"status":"met","support":["E11","E12","E13"]}},{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E6"]},"async-timing":{"status":"met","support":["E1","E3","E7"]},"ordering":{"status":"met","support":["E8","E9","E10"]},"inline-ignored":{"status":"met","support":["E2","E5"]},"modules":{"status":"met","support":["E11","E12","E13"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
I now have comprehensive information from authoritative sources. Let me write the complete answer.
The async and defer Attributes on HTML <script> Elements
The async and defer attributes provide two different mechanisms for controlling when external scripts are fetched and executed relative to HTML parsing and the DOM ready state.
How async Interacts with HTML Parsing
When the async attribute is present on an external script, the browser fetches the script in parallel to parsing and evaluates it as soon as it becomes available, potentially before HTML parsing completes. This means the script download does not block HTML parsing. However, once the script download finishes, the script execution blocks page rendering while it runs. The async attribute cannot be used on inline scripts (where it would have no effect) and only applies to external scripts and module scripts.
How defer Interacts with HTML Parsing
The defer attribute also allows the browser to fetch the script in parallel to parsing, but crucially, deferred scripts are not executed until after the entire HTML document has been parsed. This means defer eliminates parser-blocking while still deferring execution until parsing completes. Like async, defer cannot be used on inline scripts and has no effect on them.
Execution Timing Relative to DOMContentLoaded
The timing relationship with the DOMContentLoaded event differs significantly between async and defer. Deferred scripts execute after HTML parsing completes but before the DOMContentLoaded event fires—the browser actually waits for all deferred scripts to finish executing before firing DOMContentLoaded. In contrast, async scripts execute independently whenever they finish downloading and may fire well before, during, or after the DOMContentLoaded event. The browser does not wait for async scripts before firing DOMContentLoaded.
Execution Order Guarantees with Multiple Scripts
When multiple scripts use the async attribute, there is no guaranteed execution order. Scripts execute as soon as they finish downloading, so their order depends on network timing and download duration, not document order. Conversely, scripts with the defer attribute execute in the exact order they appear in the document, preserving document order regardless of download speed variations. This makes defer suitable for scripts with interdependencies.
Inline Scripts and async/defer
Inline scripts (where JavaScript code is directly embedded in the HTML without a src attribute) are unaffected by both async and defer attributes. These attributes simply have no effect on inline scripts and should not be used on them. Inline scripts always execute immediately when encountered during parsing, blocking further parsing until they complete.
Module Scripts (type="module")
Module scripts have fundamentally different default behavior than classic scripts. Module scripts defer by default—they are fetched in parallel to parsing but executed only after the HTML has been parsed, similar to the defer behavior. The defer attribute has no effect on module scripts because they already defer automatically. However, the async attribute can be applied to module scripts to change their behavior: when async is present, module scripts execute as soon as they and their dependencies are available, without waiting for document parsing to complete, similar to async classic scripts. Module scripts respect execution order for their dependencies while still allowing parallelized fetching.
{"evidence": [
{"claim": "async attribute on external classic scripts causes them to be fetched in parallel to parsing and evaluated as soon as available", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async attribute cannot be used on inline scripts", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "async scripts can interrupt DOM building and rendering", "quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML. It also means they can interrupt DOM building if they finish downloading while the parser is still at work.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "defer attribute causes scripts to be fetched in parallel but executed after document parsing completes", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer attribute cannot be used on inline scripts", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer scripts execute before DOMContentLoaded event fires, and browser waits for them", "quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event. Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded does not wait for async scripts", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "async scripts have no guaranteed execution order", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer scripts execute in the order they appear in the document", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer attribute execution order is preserved regardless of download timing", "quote": "defer ensures that scripts run in the order they appear in the HTML and don't block the parser.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "module scripts defer by default", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "module scripts without async are fetched in parallel but executed after parsing completes", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async module scripts execute as soon as available without waiting for parsing", "quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "inline scripts with type=module behave differently from classic inline scripts", "quote": "async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.", "url": "https://web.dev/learn/performance/optimize-resource-loading"},
{"claim": "scripts without async, defer, or type=module are fetched and executed immediately", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.