ScaffoldBench

RunHaiku 4.5repeat 2

Plan-execute-verify on async vs defer on <script>

claude-haiku-4-5-20251001__plan-execute-verify__research-script-loading__r2

Score

0.906

wrong answer
design ran as designed
Cost
$0.259
+ $0.076 judging
Latency
3m 19s
setup 0.0s, grading 4.6s
Turns
34
Tool calls
28
Subagents
0
Tokens in / out
314k / 20k
238,773 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • The plan was a plan: heldplan-execute-verify.planThe planner’s reply was read through its schema and held at least one step.
  • Every verdict was read: heldplan-execute-verify.verdictsEvery review round ended with a verdict read from the schema reply, not a default.
  • The loop did what its verdicts said: heldplan-execute-verify.loopEvery verdict that called for a revision got one, and nothing was revised without such a verdict.
  • Each call was shown the right tools: heldplan-execute-verify.toolsThe planner was shown no tools, the verifier the read-only ones, and the executor and reviser the full toolset.

Phases

Plan: 0.0s to 17.4s, finished, 0 tool calls, $0.015
Execute: 17.4s to 1m 12s, finished, 14 tool calls, $0.092
Verify: 1m 12s to 3m 19s, finished, 14 tool calls, $0.152

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Plan17.4s$0.01530claude-haiku-4-5-20251001
Execute54.7s$0.0921514claude-haiku-4-5-20251001
Verify2m 7s$0.1521614claude-haiku-4-5-20251001

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies plan-execute-verify --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1

Score breakdown

  • passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.
    1.00weight 0.13

    met, backed by verified evidence E1, E4. “`defer` scripts are also fetched in parallel but defer all execution until parsing is complete, never blocking the parser. Normal scripts without either attribute "block parsing until these are both complete"”

  • passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.
    1.00weight 0.19

    met, backed by verified evidence E4, E7. “This means `defer` scripts execute before DOMContentLoaded fires”

  • partialStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.
    0.50weight 0.19

    partial, backed by verified evidence E1, E8. “"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async"”

  • passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.
    1.00weight 0.19

    met, backed by verified evidence E2, E5. “scripts execute in the order they finish downloading, not the order they appear in the HTML”

  • passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).
    1.00weight 0.13

    met, backed by verified evidence E3, E6. “`async` and `defer` attributes have no effect on them.”

  • passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.
    1.00weight 0.19

    met, backed by verified evidence E9, E10. “module scripts defer by default—they behave like `defer` scripts without the `async` attribute. With `async`, they behave like async scripts.”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
opened documents
["mdn-script-element","mdn-domcontentloaded","webdev-optimize-resource-loading","whatwg-html-scripting","webdev-third-party-javascript"]
opened by role
{"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-optimize-resource-loading","webdev-third-party-javascript"],"primary":["whatwg-html-scripting"]}
displayed share
{"whatwg-html-scripting":0.2434,"webdev-third-party-javascript":0.9618,"mdn-script-element":1,"webdev-optimize-resource-loading":1,"mdn-domcontentloaded":1}
cited count
4
verified count
4
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
12
verified quote count
12
verified quote rate
1
evidence
[{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"For external classic scripts, async causes the script to be fetched in parallel to parsing and evaluated as soon as it is available","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts execute as soon as download is complete and block the page, with no guaranteed execution order for multiple async scripts","quote":"scripts with an async attribute will execute as soon as the download is complete. This blocks the page and does not guarantee any specific execution order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async must not be used on inline scripts because it has no effect","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Defer causes external classic scripts to be fetched in parallel and evaluated when the page has finished parsing","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Scripts with defer attribute execute in the order they appear in the document","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Defer must not be used on inline scripts because it has no effect","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded fires after deferred scripts and module scripts have executed, but before async scripts","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded fires after deferred and module script execution, but before async script execution","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts defer by default; async attribute can override this default deferring behavior","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies \n…[truncated 143 chars]","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Defer attribute has no effect on module scripts because they defer by default","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Inline module scripts defer by default like external module scripts","quote":"scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"When both async and defer are specified, async takes precedence","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"}]
judge unverified quotes
0
judge verdicts
[{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E7"]},"async-timing":{"status":"partial","support":["E1","E8"]},"ordering":{"status":"met","support":["E2","E5"]},"inline-ignored":{"status":"met","support":["E3","E6"]},"modules":{"status":"met","support":["E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E7"]},"async-timing":{"status":"partial","support":["E1","E8"]},"ordering":{"status":"met","support":["E2","E5"]},"inline-ignored":{"status":"met","support":["E3","E6"]},"modules":{"status":"met","support":["E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E7"]},"async-timing":{"status":"partial","support":["E1","E8"]},"ordering":{"status":"met","support":["E2","E5"]},"inline-ignored":{"status":"met","support":["E3","E6"]},"modules":{"status":"met","support":["E9","E10"]}}]
judge failures
0
grounded weight share
1

Final answer

Based on my research, I now have comprehensive information about the async and defer attributes. Let me compile a complete answer.

Complete Answer

The async and defer attributes on HTML <script> elements control when scripts are fetched and executed relative to HTML parsing and the DOMContentLoaded event. They have fundamentally different behaviors despite both allowing non-blocking fetches.

The async Attribute

For external classic scripts, the async attribute causes the script to be "fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)" (https://html.spec.whatwg.org/multipage/scripting.html). This means the browser downloads the script without blocking HTML parsing, but executes it immediately upon completion, which can interrupt DOM construction. Importantly, async provides no guarantee about execution order when multiple scripts use this attribute; scripts execute in the order they finish downloading, not the order they appear in the HTML. Additionally, "scripts with an async attribute will execute as soon as the download is complete. This blocks the page and does not guarantee any specific execution order" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).

The async attribute "must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).

The defer Attribute

For external classic scripts, defer causes the script to be "fetched in parallel and evaluated when the page has finished parsing" (https://html.spec.whatwg.org/multipage/scripting.html). This means "scripts with a defer attribute will load in the order they are in and will only execute once everything has finished loading" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script). Critically, "Scripts with the defer attribute will execute in the order in which they appear in the document" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script), providing guaranteed execution order.

Like async, "This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).

Interaction with HTML Parsing

The critical difference in parsing interaction is straightforward: async scripts are fetched in parallel but execute immediately when ready, potentially blocking the parser. defer scripts are also fetched in parallel but defer all execution until parsing is complete, never blocking the parser. Normal scripts without either attribute "block parsing until these are both complete" (https://html.spec.whatwg.org/multipage/scripting.html).

DOMContentLoaded Event Timing

The DOMContentLoaded event fires at different points depending on script attributes. "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src="…"> and <script type="module">) have downloaded and executed" (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event). This means defer scripts execute before DOMContentLoaded fires, but "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async" (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event).

Inline Scripts

Inline scripts (scripts with inline content rather than a src attribute) are fundamentally different: async and defer attributes have no effect on them. Inline scripts with no type attribute execute immediately during HTML parsing, blocking the parser. However, "inline scripts with the type="module" attribute" do defer by default (https://web.dev/learn/performance/optimize-resource-loading).

Module Scripts (type="module")

Module scripts have special behavior. For module scripts, "if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)" (https://html.spec.whatwg.org/multipage/scripting.html).

This means module scripts defer by default—they behave like defer scripts without the async attribute. With async, they behave like async scripts. The key difference from classic scripts is that "The defer attribute has no effect on module scripts — they defer by default" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).

When Both Attributes Are Specified

If both async and defer are specified, the async behavior takes precedence. "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).


{"evidence": [{"claim": "For external classic scripts, async causes the script to be fetched in parallel to parsing and evaluated as soon as it is available", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}, {"claim": "Async scripts execute as soon as download is complete and block the page, with no guaranteed execution order for multiple async scripts", "quote": "scripts with an async attribute will execute as soon as the download is complete. This blocks the page and does not guarantee any specific execution order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Async must not be used on inline scripts because it has no effect", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Defer causes external classic scripts to be fetched in parallel and evaluated when the page has finished parsing", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}, {"claim": "Scripts with defer attribute execute in the order they appear in the document", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Defer must not be used on inline scripts because it has no effect", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "DOMContentLoaded fires after deferred scripts and module scripts have executed, but before async scripts", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}, {"claim": "DOMContentLoaded fires after deferred and module script execution, but before async script execution", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}, {"claim": "Module scripts defer by default; async attribute can override this default deferring behavior", "quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}, {"claim": "Defer attribute has no effect on module scripts because they defer by default", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Inline module scripts defer by default like external module scripts", "quote": "scripts (including inline scripts) with type=\"module\" are deferred automatically.", "url": "https://web.dev/learn/performance/optimize-resource-loading"}, {"claim": "When both async and defer are specified, async takes precedence", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.