RunHaiku 4.5repeat 1
Plan-execute on async vs defer on <script>
claude-haiku-4-5-20251001__plan-execute__research-script-loading__r1
Score
1.000
- Cost
- $0.117
- + $0.086 judging
- Latency
- 1m 15s
- setup 0.0s, grading 4.7s
- Turns
- 23
- Tool calls
- 19
- Subagents
- 0
- Tokens in / out
- 172k / 7.1k
- 132,129 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- The plan was a plan: heldplan-execute.planThe planner’s reply was read through its schema and held at least one step.
- Each call was shown the right tools: heldplan-execute.toolsThe planner was shown no tools and the executor the task’s full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Plan | 15.9s | $0.013 | 3 | 0 | claude-haiku-4-5-20251001 |
| Execute | 59.3s | $0.104 | 20 | 19 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies plan-execute --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E14. “Both attributes eliminate parser-blocking for the download phase itself.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E4, E6, E7. “**`defer` scripts** execute after HTML parsing completes but **before** `DOMContentLoaded` fires.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E2, E3, E7. “**`async` scripts** execute as soon as downloaded, which can be before or after `DOMContentLoaded`.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E5, E8, E2. “**`async` scripts execute in download order, not document order**: there is no guarantee about execution order.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E9, E10. “these attributes have no effect on inline classic scripts”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E11, E13, E12. “The `defer` attribute has no effect on module scripts.”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","mdn-domcontentloaded","whatwg-html-scripting","webdev-optimize-resource-loading","mdn-js-modules","webdev-third-party-javascript"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-optimize-resource-loading","mdn-js-modules","webdev-third-party-javascript"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"webdev-third-party-javascript":0.9618,"mdn-script-element":0.857,"whatwg-html-scripting":0.3288,"webdev-optimize-resource-loading":0.9624,"mdn-domcontentloaded":1,"mdn-js-modules":0.1954}
- cited count
- 5
- verified count
- 5
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 14
- verified quote count
- 14
- verified quote rate
- 1
- evidence
- [{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"For classic scripts, if async is present, the classic script is fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","quote":"For classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts execute as soon as download is complete and do not guarantee any specific execution order.","quote":"scripts with an async attribute will execute as soon as the download is complete. This blocks the page and does not guarantee any specific execution order.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can still block HTML parsing.","quote":"Scripts with async can still block HTML parsing.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Defer scripts execute after HTML parsing is completely finished, but before the DOMContentLoaded event.","quote":"Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Scripts with defer attribute will execute in the order in which they appear in the document.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"The DOMContentLoaded event fires after all deferred scripts and module scripts have downloaded and executed.","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded fires after deferred/module script execution but before async script execution.","quote":"document.readyState is set to \"interactive\" after the completion of the HTML parser but before the execution of scripts with defer or type=\"module\". DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Async scripts may execute out-of-order, while defer scripts are executed in the order they appear in markup.","quote":"Scripts loaded with async are parsed and executed immediately once downloaded, while scripts loaded with defer are executed when HTML document parsing is finished—this occurs at the same time as the browser's DOMContentLoaded event. Additionally, async scripts may execute out-of-order, while defer s\n…[truncated 68 chars]","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The async attribute has no effect on inline classic scripts.","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The defer attribute has no effect on inline classic scripts.","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Module scripts defer by default, and the defer attribute has no effect on them.","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Inline scripts with type='module' are deferred automatically.","quote":"async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"For module scripts, if async attribute is present, the module script and its dependencies are fetched in parallel and evaluated as soon as available (potentially before parsing completes).","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"For external classic scripts, async fetches in parallel and evaluates as soon as available; defer fetches in parallel but evaluates after parsing.","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes). If the async attribute is not present but the defer attribute is present, then the classic \n…[truncated 84 chars]","verified":true,"document":"whatwg-html-scripting"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E14"]},"defer-timing":{"status":"met","support":["E4","E6","E7"]},"async-timing":{"status":"met","support":["E1","E2","E3","E7"]},"ordering":{"status":"met","support":["E5","E8","E2"]},"inline-ignored":{"status":"met","support":["E9","E10"]},"modules":{"status":"met","support":["E11","E13","E12"]}},{"parallel-download":{"status":"met","support":["E1","E14"]},"defer-timing":{"status":"met","support":["E4","E6","E7"]},"async-timing":{"status":"met","support":["E1","E2","E3","E7"]},"ordering":{"status":"met","support":["E2","E5","E8"]},"inline-ignored":{"status":"met","support":["E9","E10"]},"modules":{"status":"met","support":["E11","E13","E12"]}},{"parallel-download":{"status":"met","support":["E1","E14"]},"defer-timing":{"status":"met","support":["E4","E6","E7"]},"async-timing":{"status":"met","support":["E1","E2","E3","E7"]},"ordering":{"status":"met","support":["E5","E8","E2"]},"inline-ignored":{"status":"met","support":["E9","E10"]},"modules":{"status":"met","support":["E11","E13","E12"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
Now let me compile the comprehensive answer with all the key information I've gathered.
Answer
The async and defer attributes on HTML <script> elements control how external scripts are fetched and executed relative to HTML parsing and DOM readiness. They address a fundamental tension in web loading: scripts can modify the DOM and need to execute at precise times, but blocking HTML parsing for script downloads/execution degrades page performance.
What async and defer do
async allows a script to be fetched in parallel with HTML parsing, but it executes as soon as the download completes—potentially interrupting DOM construction. defer also fetches in parallel but delays execution until after HTML parsing finishes. Both attributes eliminate parser-blocking for the download phase itself.
Interaction with HTML parsing
Without either attribute, an external classic script is parser-blocking: the browser stops parsing HTML, downloads the script, parses it, executes it, then resumes parsing. This can severely delay initial page rendering.
With async, the HTML parser continues uninterrupted during the download, but when the download finishes, the parser pauses to execute the script. This means async scripts can still interrupt DOM construction. With defer, the HTML parser continues without interruption during both download and parsing, and the script only executes after the parser completes naturally.
Execution timing relative to DOMContentLoaded
The DOMContentLoaded event is critical for understanding these differences. This event fires after HTML parsing completes and all deferred/module scripts have executed, but it does not wait for async scripts.
deferscripts execute after HTML parsing completes but beforeDOMContentLoadedfires. TheDOMContentLoadedevent is delayed until all deferred scripts finish executing.asyncscripts execute as soon as downloaded, which can be before or afterDOMContentLoaded. The event does not wait for them.- Undeferred regular scripts block parsing and execute immediately;
DOMContentLoadedfires after they complete.
Execution order guarantees with multiple scripts
This is where the attributes diverge sharply:
deferscripts execute in document order: if you have multiple deferred scripts, they always execute in the order they appear in the HTML, even if one downloads faster than another.asyncscripts execute in download order, not document order: there is no guarantee about execution order. If script 2 downloads faster than script 1, script 2 executes first, regardless of which appeared first in the HTML. This is crucial: if script 2 depends on script 1, anasyncsetup can break.
Inline scripts and type="module" scripts
Inline scripts (scripts without a src attribute) do not support async or defer—these attributes have no effect on inline classic scripts. An inline classic script always executes immediately and blocks parsing until complete. This is by design: inline scripts are already in the HTML, so there's no fetch phase to optimize.
Module scripts (type="module") differ fundamentally from classic scripts:
- Module scripts are deferred by default—they always wait until HTML parsing completes before executing, without needing the
deferattribute. Thedeferattribute has no effect on module scripts. - Inline module scripts are also deferred by default—even without
srcordeferattributes, inline modules execute after parsing, unlike inline classic scripts. - Module scripts with
async(both external and inline) execute as soon as they and their dependencies are available, similar toasyncclassic scripts, potentially before parsing completes. Whenasyncis specified on a module, the defer behavior is overridden. - Multiple module scripts maintain execution order when deferred (the default), similar to deferred classic scripts—they execute in document order. This is a safety feature: modules often import from each other, and preserving order prevents broken dependencies.
Special case: async + defer together
If both async and defer are specified on a classic script, async wins—the script behaves as if only async is present.
Evidence
{
"evidence": [
{
"claim": "For classic scripts, if async is present, the classic script is fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).",
"quote": "For classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Async scripts execute as soon as download is complete and do not guarantee any specific execution order.",
"quote": "scripts with an async attribute will execute as soon as the download is complete. This blocks the page and does not guarantee any specific execution order.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Async scripts can still block HTML parsing.",
"quote": "Scripts with async can still block HTML parsing.",
"url": "https://web.dev/articles/efficiently-load-third-party-javascript"
},
{
"claim": "Defer scripts execute after HTML parsing is completely finished, but before the DOMContentLoaded event.",
"quote": "Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.",
"url": "https://web.dev/articles/efficiently-load-third-party-javascript"
},
{
"claim": "Scripts with defer attribute will execute in the order in which they appear in the document.",
"quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "The DOMContentLoaded event fires after all deferred scripts and module scripts have downloaded and executed.",
"quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.",
"url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"
},
{
"claim": "DOMContentLoaded fires after deferred/module script execution but before async script execution.",
"quote": "document.readyState is set to \"interactive\" after the completion of the HTML parser but before the execution of scripts with defer or type=\"module\". DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.",
"url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"
},
{
"claim": "Async scripts may execute out-of-order, while defer scripts are executed in the order they appear in markup.",
"quote": "Scripts loaded with async are parsed and executed immediately once downloaded, while scripts loaded with defer are executed when HTML document parsing is finished—this occurs at the same time as the browser's DOMContentLoaded event. Additionally, async scripts may execute out-of-order, while defer scripts are executed in the order in which they appear in the markup.",
"url": "https://web.dev/learn/performance/optimize-resource-loading"
},
{
"claim": "The async attribute has no effect on inline classic scripts.",
"quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "The defer attribute has no effect on inline classic scripts.",
"quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Module scripts defer by default, and the defer attribute has no effect on them.",
"quote": "The defer attribute has no effect on module scripts — they defer by default.",
"url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"
},
{
"claim": "Inline scripts with type='module' are deferred automatically.",
"quote": "async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.",
"url": "https://web.dev/learn/performance/optimize-resource-loading"
},
{
"claim": "For module scripts, if async attribute is present, the module script and its dependencies are fetched in parallel and evaluated as soon as available (potentially before parsing completes).",
"quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
},
{
"claim": "For external classic scripts, async fetches in parallel and evaluates as soon as available; defer fetches in parallel but evaluates after parsing.",
"quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes). If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.",
"url": "https://html.spec.whatwg.org/multipage/scripting.html"
}
]
}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.