ScaffoldBench

RunOpus 5.5repeat 2

Plan-execute-verify on async vs defer on <script>

claude-opus-5-5__plan-execute-verify__research-script-loading__r2

Score

1.000

passed
design ran as designed
Cost
$0.685
+ $0.083 judging
Latency
2m 9s
setup 0.0s, grading 6.1s
Turns
36
Tool calls
31
Subagents
0
Tokens in / out
303k / 13k
249,385 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • The plan was a plan: heldplan-execute-verify.planThe planner’s reply was read through its schema and held at least one step.
  • Every verdict was read: heldplan-execute-verify.verdictsEvery review round ended with a verdict read from the schema reply, not a default.
  • The loop did what its verdicts said: heldplan-execute-verify.loopEvery verdict that called for a revision got one, and nothing was revised without such a verdict.
  • Each call was shown the right tools: heldplan-execute-verify.toolsThe planner was shown no tools, the verifier the read-only ones, and the executor and reviser the full toolset.

Phases

Plan: 0.0s to 16.4s, finished, 0 tool calls, $0.032
Execute: 16.4s to 1m 33s, finished, 19 tool calls, $0.400
Verify: 1m 33s to 2m 9s, finished, 12 tool calls, $0.252

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Plan16.4s$0.03220claude-opus-5-5
Execute1m 16s$0.4002019claude-opus-5-5
Verify36.7s$0.2521412claude-opus-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies plan-execute-verify --tasks research-script-loading --models claude-opus-5-5 --repeats 1

Score breakdown

  • passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.
    1.00weight 0.13

    met, backed by verified evidence E1, E2, E5. “`defer` downloads in parallel and runs the scripts in document order after parsing ends”

  • passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.
    1.00weight 0.19

    met, backed by verified evidence E2, E3. “It runs after the document is parsed but before `DOMContentLoaded` fires.”

  • passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.
    1.00weight 0.19

    met, backed by verified evidence E5, E6, E8. “The download doesn't block parsing, but running it can: an async script "can interrupt DOM building" if it arrives mid-parse”

  • passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.
    1.00weight 0.19

    met, backed by verified evidence E4, E7. “Deferred scripts run in the order they appear in the document (MDN, same page).”

  • passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).
    1.00weight 0.13

    met, backed by verified evidence E11, E12. “Both attributes do nothing when there is no `src`.”

  • passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.
    1.00weight 0.19

    met, backed by verified evidence E13, E14, E15. “"The defer attribute has no effect on module scripts"”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
opened documents
["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded","webdev-third-party-javascript","mdn-js-modules"]
opened by role
{"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript","mdn-js-modules"],"primary":["whatwg-html-scripting"]}
displayed share
{"mdn-script-element":0.8766,"whatwg-html-scripting":0.2341,"mdn-domcontentloaded":0.9488,"webdev-third-party-javascript":0.9285,"webdev-optimize-resource-loading":0.0157,"mdn-js-modules":0.0061}
cited count
5
verified count
5
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
16
verified quote count
16
verified quote rate
1
evidence
[{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"A classic external script with neither attribute blocks parsing while it is fetched and run.","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"A defer classic script downloads in parallel and runs after parsing finishes.","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run after parsing and before DOMContentLoaded, and DOMContentLoaded waits for them.","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run in document order.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"An async classic script downloads in parallel and runs as soon as it is available, possibly before parsing ends.","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"An async script that arrives mid-parse interrupts parsing while it runs.","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts have no guaranteed execution order.","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded does not wait for async scripts.","quote":"It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"MDN's DOMContentLoaded page says async scripts run after DOMContentLoaded, which conflicts with the spec's statement that they may run before parsing completes.","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"If both attributes are present, async takes precedence; defer exists only as a fallback for old browsers.","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async has no effect on inline classic scripts.","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The spec's async and defer handling applies only to classic scripts with src or to any module script; inline classic scripts take a different path.","quote":"If el's type is \"classic\" and el has a src attribute, or el's type is \"module\":","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts without async are deferred by default.","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer has no effect on module scripts because they are deferred automatically.","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"An async module script runs as soon as it and its imports are fetched.","quote":"For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Even an inline module script never runs synchronously during parsing.","quote":"Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.","verified":true,"document":"whatwg-html-scripting"}]
judge unverified quotes
0
judge verdicts
[{"parallel-download":{"status":"met","support":["E1","E2","E5"]},"defer-timing":{"status":"met","support":["E2","E3"]},"async-timing":{"status":"met","support":["E5","E6","E8"]},"ordering":{"status":"met","support":["E4","E7"]},"inline-ignored":{"status":"met","support":["E11","E12"]},"modules":{"status":"met","support":["E13","E14","E15"]}},{"parallel-download":{"status":"met","support":["E1","E2","E5"]},"defer-timing":{"status":"met","support":["E3","E2"]},"async-timing":{"status":"met","support":["E5","E6"]},"ordering":{"status":"met","support":["E4","E7"]},"inline-ignored":{"status":"met","support":["E11","E12"]},"modules":{"status":"met","support":["E13","E14","E15"]}},{"parallel-download":{"status":"met","support":["E1","E2","E5"]},"defer-timing":{"status":"met","support":["E2","E3"]},"async-timing":{"status":"met","support":["E5","E6"]},"ordering":{"status":"met","support":["E4","E7"]},"inline-ignored":{"status":"met","support":["E11","E12"]},"modules":{"status":"met","support":["E13","E14","E15"]}}]
judge failures
0
grounded weight share
1

Final answer

I've finished the research. Some planned sources returned 404 (the spec's parsing.html page and javascript.info), so the answer relies on the WHATWG scripting section, MDN and web.dev. Where they differ, I treat the WHATWG spec as normative.

Short answer

With no attributes, a classic script stops the HTML parser until it has downloaded and run. defer downloads in parallel and runs the scripts in document order after parsing ends, before DOMContentLoaded. async downloads in parallel and runs each script as soon as it arrives, in no fixed order, possibly mid-parse. Both are ignored on inline classic scripts. Module scripts are deferred by default, defer does nothing to them, and async works on them even when inline.

No attributes (the baseline)

For a classic script with src and neither attribute, "the script is fetched and evaluated immediately, blocking parsing until these are both complete" (https://html.spec.whatwg.org/multipage/scripting.html). MDN adds that inline scripts without type="module" behave the same way: they run immediately, before parsing continues (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).

defer (classic scripts with src)

async (classic scripts with src)

  • Parsing: The script downloads in parallel and runs "as soon as it is available (potentially before parsing completes)" (https://html.spec.whatwg.org/multipage/scripting.html). The download doesn't block parsing, but running it can: an async script "can interrupt DOM building" if it arrives mid-parse (https://web.dev/articles/efficiently-load-third-party-javascript).
  • Order: Each async script goes into a set (not a list) of "scripts that will execute as soon as possible" and runs when its own fetch completes (spec, same page). MDN puts it plainly: "You get no guarantee that scripts will run in any specific order." In practice, whichever finishes downloading first runs first.
  • DOMContentLoaded: DOMContentLoaded does not wait for async scripts (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event). An async script may run before or after it.

    • Where sources disagree: A note on that same MDN page says DOMContentLoaded fires "before the execution of scripts with async". That conflicts with the spec, which says async scripts may run before parsing finishes, and therefore before DOMContentLoaded. I follow the spec. Read MDN's note as describing a common case, not a rule.
  • Both attributes together: async wins (MDN). The spec says adding defer alongside async exists only so that old browsers without async support fall back to deferring instead of blocking.

Inline classic scripts

Both attributes do nothing when there is no src. MDN warns about this for each attribute. In the spec's processing steps, the async and defer branches apply only "If el's type is \"classic\" and el has a src attribute, or el's type is \"module\"". Inline classic scripts take the other path and normally run immediately, while the parser waits (https://html.spec.whatwg.org/multipage/scripting.html).

type="module"

  • Default: Module scripts behave as if deferred. The module and its imports download in parallel and run when parsing finishes. "The defer attribute has no effect on module scripts" (https://html.spec.whatwg.org/multipage/scripting.html). MDN agrees (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules). Like deferred classic scripts, they run before DOMContentLoaded.
  • With async: The module and all its imports are fetched in parallel and the module runs as soon as everything is ready, possibly mid-parse, in no fixed order.
  • Inline modules: The type check covers all module scripts, not just ones with src, so async applies to inline modules too. Even an inline module with no imports never runs synchronously, because the spec queues a task first.

Summary

ScriptBlocks parsing?Runs whenOrderRelation to DOMContentLoaded
Classic, no attribute (external or inline)YesImmediatelyDocument orderBefore it
defer (external classic)NoAfter parsingDocument orderBefore; DOMContentLoaded waits for it
async (external classic)Only while it runsWhen downloadedNo guaranteeBefore or after; not waited for
defer/async on inline classicYes (attributes ignored)ImmediatelyDocument orderBefore it
type="module"NoAfter parsingDocument orderBefore; DOMContentLoaded waits for it
type="module" async (external or inline)Only while it runsWhen it and its imports are readyNo guaranteeBefore or after
{"evidence": [
{"claim": "A classic external script with neither attribute blocks parsing while it is fetched and run.", "quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "A defer classic script downloads in parallel and runs after parsing finishes.", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Deferred scripts run after parsing and before DOMContentLoaded, and DOMContentLoaded waits for them.", "quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Deferred scripts run in document order.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "An async classic script downloads in parallel and runs as soon as it is available, possibly before parsing ends.", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "An async script that arrives mid-parse interrupts parsing while it runs.", "quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "Async scripts have no guaranteed execution order.", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded does not wait for async scripts.", "quote": "It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "MDN's DOMContentLoaded page says async scripts run after DOMContentLoaded, which conflicts with the spec's statement that they may run before parsing completes.", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "If both attributes are present, async takes precedence; defer exists only as a fallback for old browsers.", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async has no effect on inline classic scripts.", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "The spec's async and defer handling applies only to classic scripts with src or to any module script; inline classic scripts take a different path.", "quote": "If el's type is \"classic\" and el has a src attribute, or el's type is \"module\":", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Module scripts without async are deferred by default.", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer has no effect on module scripts because they are deferred automatically.", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "An async module script runs as soon as it and its imports are fetched.", "quote": "For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Even an inline module script never runs synchronously during parsing.", "quote": "Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.