RunHaiku 4.5repeat 2
Single on async vs defer on <script>
claude-haiku-4-5-20251001__single__research-script-loading__r2
Score
1.000
- Cost
- $0.079
- + $0.086 judging
- Latency
- 43.6s
- setup 0.0s, grading 5.0s
- Turns
- 13
- Tool calls
- 12
- Subagents
- 0
- Tokens in / out
- 90k / 4.0k
- 58,505 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Main | 43.6s | $0.079 | 13 | 12 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies single --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E11. “Both `async` and `defer` eliminate this parser-blocking behavior for the download phase.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E3, E7. “Deferred scripts execute just before the DOMContentLoaded event fires”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E5, E7. “async scripts can interrupt DOM building if they finish downloading while the parser is still processing HTML”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E4, E6. “Deferred scripts maintain execution order: if you have three deferred scripts in sequence in the HTML, they will execute in that same sequence. Async scripts offer no such guarantee”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E2. “Inline scripts (scripts without a `src` attribute) do not support the `async` or `defer` attributes; these attributes are ignored for inline classic scripts.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E8, E9, E10. “The `defer` attribute has no effect on module scripts since they already defer by default. If you add the `async` attribute to an external module script”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","mdn-domcontentloaded","whatwg-html-scripting","webdev-third-party-javascript","webdev-optimize-resource-loading"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript","webdev-optimize-resource-loading"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"webdev-third-party-javascript":0.9618,"whatwg-html-scripting":0.2175,"mdn-script-element":1,"webdev-optimize-resource-loading":0.9624,"mdn-domcontentloaded":1}
- cited count
- 5
- verified count
- 5
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 13
- verified quote count
- 13
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The async attribute causes external classic scripts to be fetched in parallel to parsing and evaluated as soon as they are available, potentially before parsing completes","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The async attribute must not be used on inline scripts as it would have no effect","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The defer attribute causes scripts to execute after the document has been parsed but before the DOMContentLoaded event","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts execute in the order they appear in the document","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can interrupt DOM building if they finish downloading while the parser is still at work","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts do not execute in a guaranteed order when there are multiple scripts","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded fires after deferred scripts execute but before async scripts execute","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Module scripts are deferred by default and the defer attribute has no effect on them","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts with async attribute fetch and execute like async classic scripts","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts without async execute in order after HTML parsing completes","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Module scripts are deferred automatically and scripts without async or defer are parser-blocking","quote":"async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Inline scripts without type=\"module\" block the parser until parsed and executed","quote":"When using inline JavaScript, the parser is similarly blocked until the inline script is parsed and executed.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"When both async and defer are specified, the element acts as if only async is specified","quote":"If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E11"]},"defer-timing":{"status":"met","support":["E3","E7"]},"async-timing":{"status":"met","support":["E1","E5","E7"]},"ordering":{"status":"met","support":["E4","E6"]},"inline-ignored":{"status":"met","support":["E2"]},"modules":{"status":"met","support":["E8","E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E11"]},"defer-timing":{"status":"met","support":["E3","E7"]},"async-timing":{"status":"met","support":["E1","E5","E7"]},"ordering":{"status":"met","support":["E4","E6"]},"inline-ignored":{"status":"met","support":["E2"]},"modules":{"status":"met","support":["E8","E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E11"]},"defer-timing":{"status":"met","support":["E3","E7"]},"async-timing":{"status":"met","support":["E1","E5","E7"]},"ordering":{"status":"met","support":["E4","E6"]},"inline-ignored":{"status":"met","support":["E2"]},"modules":{"status":"met","support":["E8","E9","E10"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
Now I have comprehensive information from multiple authoritative sources. Let me write a complete answer.
Answer
The async and defer attributes on HTML <script> elements control when external JavaScript files are downloaded and executed relative to HTML parsing and the DOMContentLoaded event.
The async Attribute
When the async attribute is present on an external classic script, the browser downloads the script in parallel with HTML parsing—parsing does not block while the script downloads. However, once the download completes, the script immediately executes, which blocks both HTML parsing and rendering until execution finishes. The key trade-off is that async scripts execute as soon as they're available, not necessarily in the order they appear in the HTML. If multiple scripts have the async attribute, there is no guarantee about their execution order. Additionally, async scripts can interrupt DOM building if they finish downloading while the parser is still processing HTML. The async attribute must not be used on inline scripts (without a src attribute) for classic scripts, as it would have no effect.
The defer Attribute
The defer attribute causes external classic scripts to download in parallel with HTML parsing, but delays their execution until after the HTML document has been completely parsed. Crucially, deferred scripts execute in the order they appear in the markup. Multiple deferred scripts will execute in sequence, in the order specified in the HTML. Deferred scripts execute just before the DOMContentLoaded event fires, and they prevent DOMContentLoaded from firing until all deferred scripts have loaded and executed. Like async, the defer attribute has no effect on inline scripts.
Interaction with HTML Parsing
Classic scripts without either attribute block HTML parsing immediately—the browser must download, parse, and execute the script before continuing to parse the remaining HTML. Both async and defer eliminate this parser-blocking behavior for the download phase. With async, parsing continues during download but resumes blocking during execution. With defer, parsing continues uninterrupted during both download and execution phases (execution happens later). The HTML parser completes normally regardless of async or defer attributes, but synchronous inline scripts and classic scripts without these attributes will still block the parser.
Relationship to DOMContentLoaded
The DOMContentLoaded event fires when the HTML document has been completely parsed and all deferred scripts and module scripts have been downloaded and executed. Deferred scripts execute before DOMContentLoaded fires, ensuring that any code in those scripts runs before DOMContentLoaded listeners are triggered. However, async scripts may execute after DOMContentLoaded has already fired—the event does not wait for async scripts. The document's readyState property is set to "interactive" after HTML parsing completes but before deferred/module scripts execute, and it's set to "complete" after async scripts execute.
Execution Order with Multiple Scripts
When you have multiple external scripts, the execution order guarantees differ significantly. Deferred scripts maintain execution order: if you have three deferred scripts in sequence in the HTML, they will execute in that same sequence. Async scripts offer no such guarantee—if you have three async scripts, they may execute in any order depending on which finishes downloading first. This is a critical distinction: if your scripts have dependencies (one script uses a function defined in another), you must use defer to ensure correct execution order, or place dependent scripts after their dependencies in the HTML.
Inline Scripts and Module Scripts
Inline scripts (scripts without a src attribute) do not support the async or defer attributes; these attributes are ignored for inline classic scripts. Inline scripts are executed immediately when encountered during parsing, blocking further parsing. However, inline module scripts (with type="module" but no src attribute) are deferred by default, meaning they execute after HTML parsing completes.
Module scripts (type="module") behave differently from classic scripts. External module scripts are deferred by default—they execute after HTML parsing completes, in the order they appear, without needing the defer attribute. The defer attribute has no effect on module scripts since they already defer by default. If you add the async attribute to an external module script, the script and all its dependencies will fetch in parallel, and the module will execute as soon as it's available (like async classic scripts), potentially out of order relative to other module scripts. When module scripts are dynamically injected into the DOM via JavaScript, they behave like async scripts rather than deferred scripts.
{"evidence": [
{
"claim": "The async attribute causes external classic scripts to be fetched in parallel to parsing and evaluated as soon as they are available, potentially before parsing completes",
"quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "The async attribute must not be used on inline scripts as it would have no effect",
"quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "The defer attribute causes scripts to execute after the document has been parsed but before the DOMContentLoaded event",
"quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Deferred scripts execute in the order they appear in the document",
"quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Async scripts can interrupt DOM building if they finish downloading while the parser is still at work",
"quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.",
"url": "https://web.dev/articles/efficiently-load-third-party-javascript"
},
{
"claim": "Async scripts do not execute in a guaranteed order when there are multiple scripts",
"quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.",
"url": "https://web.dev/articles/efficiently-load-third-party-javascript"
},
{
"claim": "DOMContentLoaded fires after deferred scripts execute but before async scripts execute",
"quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.",
"url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"
},
{
"claim": "Module scripts are deferred by default and the defer attribute has no effect on them",
"quote": "The defer attribute has no effect on module scripts — they defer by default.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Module scripts with async attribute fetch and execute like async classic scripts",
"quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "Module scripts without async execute in order after HTML parsing completes",
"quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "Module scripts are deferred automatically and scripts without async or defer are parser-blocking",
"quote": "async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.",
"url": "https://web.dev/learn/performance/optimize-resource-loading"
},
{
"claim": "Inline scripts without type=\"module\" block the parser until parsed and executed",
"quote": "When using inline JavaScript, the parser is similarly blocked until the inline script is parsed and executed.",
"url": "https://web.dev/learn/performance/optimize-resource-loading"
},
{
"claim": "When both async and defer are specified, the element acts as if only async is specified",
"quote": "If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.