RunHaiku 4.5repeat 1
Single on async vs defer on <script>
claude-haiku-4-5-20251001__single__research-script-loading__r1
Score
1.000
- Cost
- $0.075
- + $0.089 judging
- Latency
- 43.8s
- setup 0.0s, grading 5.2s
- Turns
- 14
- Tool calls
- 13
- Subagents
- 0
- Tokens in / out
- 98k / 4.3k
- 71,272 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Main | 43.8s | $0.075 | 14 | 13 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies single --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E2. “Without either attribute, external classic scripts block HTML parsing—the browser must download and execute the script before continuing to parse the rest of the HTML.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E4, E7. “Scripts with `defer` execute after HTML parsing completes but **before** the `DOMContentLoaded` event fires.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E3, E7. “Scripts with `async` execute as soon as they download, which may be before or after the `DOMContentLoaded` event, depending on timing.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E5, E6. “For `async` scripts, there are no guarantees about execution order.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E8. “The `async` and `defer` attributes have **no effect** on inline classic scripts.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E9, E10. “The defer attribute has no effect on module scripts — they defer by default.”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded","webdev-optimize-resource-loading","webdev-third-party-javascript"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-optimize-resource-loading","webdev-third-party-javascript"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"webdev-third-party-javascript":0.9618,"webdev-optimize-resource-loading":0.5195,"mdn-script-element":0.8439,"whatwg-html-scripting":0.2175,"mdn-domcontentloaded":1,"mdn-js-modules":0.0065}
- cited count
- 5
- verified count
- 5
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 10
- verified quote count
- 10
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async attribute causes external classic scripts to be fetched in parallel to parsing and evaluated as soon as available","quote":"if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"defer attribute causes scripts to be fetched in parallel but evaluated when page has finished parsing","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"async scripts can still interrupt HTML parsing","quote":"Scripts with async can still block HTML parsing","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute after HTML parsing but before DOMContentLoaded","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"async scripts execute before the window's load event and their order is unpredictable","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute in document order","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded waits for defer and module scripts but not async scripts","quote":"document.readyState is set to 'interactive' after the completion of the HTML parser but before the execution of scripts with defer or type='module'. DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async and defer attributes have no effect on inline classic scripts","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer attribute has no effect on module scripts as they defer by default","quote":"The defer attribute has no effect on module scripts — they defer by default","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"module scripts with async attribute execute as soon as available rather than after parsing","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies \n…[truncated 87 chars]","verified":true,"document":"whatwg-html-scripting"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E4","E7"]},"async-timing":{"status":"met","support":["E1","E3","E7"]},"ordering":{"status":"met","support":["E5","E6"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E4","E7","E2"]},"async-timing":{"status":"met","support":["E1","E3","E7"]},"ordering":{"status":"met","support":["E5","E6"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E4","E7"]},"async-timing":{"status":"met","support":["E1","E3","E5"]},"ordering":{"status":"met","support":["E5","E6"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
Now I have comprehensive information. Let me write the complete answer based on my research.
The async and defer Attributes on HTML <script> Elements
The async and defer attributes provide two different strategies for controlling when external JavaScript scripts are fetched and executed, helping to optimize page load performance.
What Each Attribute Does
The async attribute tells the browser to fetch the script in parallel with HTML parsing and execute it as soon as it becomes available. Once the script finishes downloading, the browser immediately executes it, which can interrupt HTML parsing if parsing is still ongoing. According to the HTML specification, "if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)." (https://html.spec.whatwg.org/multipage/scripting.html)
The defer attribute tells the browser to fetch the script in parallel with HTML parsing but delay execution until after the HTML document has been completely parsed. The HTML specification states: "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing." (https://html.spec.whatwg.org/multipage/scripting.html)
HTML Parsing Interaction
Without either attribute, external classic scripts block HTML parsing—the browser must download and execute the script before continuing to parse the rest of the HTML. According to web.dev: "A parser-blocking resource interrupts the HTML parser, such as a <script> element without async or defer attributes." (https://web.dev/learn/performance/optimize-resource-loading)
With async, the script is fetched in parallel while parsing continues, but execution can still interrupt parsing. As noted, "Scripts with async can still block HTML parsing." (https://web.dev/articles/efficiently-load-third-party-javascript)
With defer, the script is fetched in parallel and parsing is never interrupted. The browser completes parsing the entire HTML document before executing deferred scripts. According to web.dev: "defer ensures that scripts run in the order they appear in the HTML and don't block the parser." (https://web.dev/articles/efficiently-load-third-party-javascript)
Execution Timing Relative to DOMContentLoaded
The timing relationship with the DOMContentLoaded event is a key difference between these attributes:
Scripts with defer execute after HTML parsing completes but before the DOMContentLoaded event fires. According to MDN: "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script)
Scripts with async execute as soon as they download, which may be before or after the DOMContentLoaded event, depending on timing. According to web.dev: "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event." (https://web.dev/articles/efficiently-load-third-party-javascript) The MDN documentation on the DOMContentLoaded event clarifies: "document.readyState is set to 'interactive' after the completion of the HTML parser but before the execution of scripts with defer or type='module'. DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async." (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event)
Execution Order Guarantees
For defer scripts, execution order is guaranteed to match the document order. MDN states: "Scripts with the defer attribute will execute in the order in which they appear in the document." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script) This means if you have three deferred scripts in your HTML, they will execute in the order they appear, regardless of download times.
For async scripts, there are no guarantees about execution order. Web.dev explains: "This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML." (https://web.dev/articles/efficiently-load-third-party-javascript) If your scripts have dependencies on each other, this unpredictability can cause problems.
Inline Scripts vs. External Scripts
Inline scripts (scripts with content directly in the HTML rather than loaded from a src attribute) behave fundamentally differently from external scripts. The async and defer attributes have no effect on inline classic scripts. MDN notes: "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script)
Inline classic scripts always execute immediately and block HTML parsing until they complete.
Module Scripts
Module scripts (specified with type="module") have fundamentally different default behavior. MDN states: "The defer attribute has no effect on module scripts — they defer by default." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script) This means module scripts automatically behave like deferred scripts: they are fetched in parallel, and execution is delayed until after HTML parsing completes and before DOMContentLoaded fires.
For module scripts, the async attribute has a special effect: if you add async to a module script, it changes the behavior so the module executes as soon as it's available (like a non-deferred script), rather than waiting for parsing to complete. According to the HTML specification: "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing." (https://html.spec.whatwg.org/multipage/scripting.html)
Inline module scripts also defer by default but ignore the defer attribute. They wait for the HTML to finish parsing before executing, and can use the async attribute to change this behavior (causing them to execute as soon as they load).
{"evidence": [{"claim": "async attribute causes external classic scripts to be fetched in parallel to parsing and evaluated as soon as available", "quote": "if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}, {"claim": "defer attribute causes scripts to be fetched in parallel but evaluated when page has finished parsing", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}, {"claim": "async scripts can still interrupt HTML parsing", "quote": "Scripts with async can still block HTML parsing", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "defer scripts execute after HTML parsing but before DOMContentLoaded", "quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "async scripts execute before the window's load event and their order is unpredictable", "quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "defer scripts execute in document order", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "DOMContentLoaded waits for defer and module scripts but not async scripts", "quote": "document.readyState is set to 'interactive' after the completion of the HTML parser but before the execution of scripts with defer or type='module'. DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}, {"claim": "async and defer attributes have no effect on inline classic scripts", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "defer attribute has no effect on module scripts as they defer by default", "quote": "The defer attribute has no effect on module scripts — they defer by default", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "module scripts with async attribute execute as soon as available rather than after parsing", "quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.