ScaffoldBench

RunOpus 5.5repeat 2

Gated-execute on async vs defer on <script>

claude-opus-5-5__gated-execute__research-script-loading__r2

Score

1.000

passed
design ran as designed
Cost
$0.347
+ $0.080 judging
Latency
1m 8s
setup 0.0s, grading 4.4s
Turns
19
Tool calls
18
Subagents
0
Tokens in / out
162k / 6.4k
137,255 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • Every state change was reviewed: heldgated-execute.reviewedEvery call that changed the world had been allowed by a review before it ran, counted from the world’s own journal.
  • Every decision was read: heldgated-execute.decisionsEvery review that ran to its end gave its decision through the schema; an unreadable one is a denial.
  • Each call was shown the right tools: heldgated-execute.toolsThe executor was shown the task’s full toolset and each reviewer the read-only tools.

Phases

Execute: 0.0s to 1m 8s, finished, 18 tool calls, $0.347

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Execute1m 8s$0.3471918claude-opus-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies gated-execute --tasks research-script-loading --models claude-opus-5-5 --repeats 1

Score breakdown

  • passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.
    1.00weight 0.13

    met, backed by verified evidence E1, E2. “`async` and `defer` both let the browser download a script while it keeps parsing the HTML.”

  • passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.
    1.00weight 0.19

    met, backed by verified evidence E3, E4. “a deferred script runs "after the document has been parsed, but before firing DOMContentLoaded"”

  • passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.
    1.00weight 0.19

    met, backed by verified evidence E2, E7. “An `async` script can run before or after `DOMContentLoaded`, and the event doesn't wait for it.”

  • passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.
    1.00weight 0.19

    met, backed by verified evidence E5, E6, E7. “Deferred scripts "execute in the order in which they appear in the document"”

  • passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).
    1.00weight 0.13

    met, backed by verified evidence E13, E15. “MDN says both have no effect there.”

  • passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.
    1.00weight 0.19

    met, backed by verified evidence E14, E16. “"The defer attribute has no effect on module scripts" (spec); MDN says modules "defer by default".”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
opened documents
["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded","webdev-third-party-javascript"]
opened by role
{"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript"],"primary":["whatwg-html-scripting"]}
displayed share
{"mdn-script-element":0.4736,"whatwg-html-scripting":0.2516,"mdn-domcontentloaded":1,"webdev-third-party-javascript":0.9407,"webdev-optimize-resource-loading":0.0157}
cited count
4
verified count
4
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
16
verified quote count
16
verified quote rate
1
evidence
[{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Without async or defer, an external classic script blocks parsing while it is fetched and evaluated.","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"An async classic script downloads in parallel with parsing and runs as soon as it is available, possibly before parsing completes; a defer script downloads in parallel and runs after parsing finishes.","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run after parsing but before DOMContentLoaded.","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"DOMContentLoaded waits for deferred scripts to load and run.","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run in document order.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"In the spec, async scripts go into an unordered set that runs as soon as possible, and deferred and non-async module scripts go into a list run after parsing.","quote":"Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts are likely to run out of document order and can interrupt parsing.","quote":"This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts are only guaranteed to run before the window load event.","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded waits for parsing plus deferred and module scripts, but not for async scripts.","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"Conflicting source: an MDN note says DOMContentLoaded fires before async scripts run, which the spec's 'potentially before parsing completes' wording contradicts.","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"If both attributes are present, async wins.","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"defer can be added alongside async so legacy browsers fall back to defer behavior.","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer has no effect on inline classic scripts.","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts are deferred by default and ignore defer; with async they and their dependencies run as soon as available.","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Every module script, inline or external, goes through the async/defer logic, unlike inline classic scripts.","quote":"If el's type is \"classic\" and el has a src attribute, or el's type is \"module\":","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"MDN describes module scripts as deferred by default.","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"}]
judge unverified quotes
0
judge verdicts
[{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E3","E4"]},"async-timing":{"status":"met","support":["E2","E7"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E13","E15"]},"modules":{"status":"met","support":["E14","E16"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E3","E4"]},"async-timing":{"status":"met","support":["E2","E7","E8"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E13"]},"modules":{"status":"met","support":["E14","E16","E15"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E3","E4"]},"async-timing":{"status":"met","support":["E2","E7","E8"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E13"]},"modules":{"status":"met","support":["E14","E16","E15"]}}]
judge failures
0
grounded weight share
1

Final answer

async and defer both let the browser download a script while it keeps parsing the HTML. The difference is when the script runs. A defer script waits until parsing is finished and runs before DOMContentLoaded, with scripts in document order. An async script runs as soon as it has downloaded, in no guaranteed order, and without regard to DOMContentLoaded.

Default case (external classic script, no attributes). The browser fetches and runs the script right away, and parsing waits for both to finish. The spec says: "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete." (https://html.spec.whatwg.org/multipage/scripting.html)

defer (external classic scripts). The script downloads alongside parsing and runs "when the page has finished parsing" (https://html.spec.whatwg.org/multipage/scripting.html).

async (external classic scripts). The script downloads alongside parsing and runs "as soon as it is available (potentially before parsing completes)" (https://html.spec.whatwg.org/multipage/scripting.html).

  • Parsing: The download doesn't block the parser, but running the script can. If it finishes downloading while the parser is still working, it interrupts DOM building (https://web.dev/articles/efficiently-load-third-party-javascript).
  • Order: None is guaranteed. The spec puts these scripts in a "set of scripts that will execute as soon as possible", and each one runs when it is ready. Compare defer's ordered list. web.dev notes "it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML" (same web.dev page).
  • Timing: An async script can run before or after DOMContentLoaded, and the event doesn't wait for it. MDN's DOMContentLoaded page says the event "doesn't wait for … async scripts" and warns that code loaded with <script async> may run after the event has already fired. web.dev only promises that async scripts run before the window's load event.
  • The sources disagree here. A note on that MDN page says DOMContentLoaded fires "before the execution of scripts with async". That conflicts with the spec's "potentially before parsing completes" and with MDN's own <script> page. The spec is the authority, so treat async timing relative to DOMContentLoaded as not guaranteed either way.

Both attributes on one script. async wins (MDN). The spec allows writing defer alongside async so that old browsers which only understand defer fall back to it instead of blocking the parser.

Inline classic scripts (no src). The spec's attribute table doesn't allow async or defer on inline classic scripts, and MDN says both have no effect there. In the spec's algorithm an inline classic script runs immediately. The one exception: if a stylesheet that blocks scripts is pending, the script waits for it, and so does the parser.

type="module" scripts.

  • Default behaves like defer: Without async, the module and all its dependencies download alongside parsing and run when parsing finishes. "The defer attribute has no effect on module scripts" (spec); MDN says modules "defer by default". The spec puts them on the same finished-parsing list as deferred classic scripts, and MDN counts them as deferred scripts that DOMContentLoaded waits for.
  • With async: The module and its dependencies download alongside parsing, and the module runs as soon as everything is available, possibly before parsing completes.
  • Inline modules work too: Unlike classic scripts, async is allowed on inline module scripts. The spec's branch "If el's type is "classic" and el has a src attribute, or el's type is "module"" sends every module, inline or external, through the async/defer logic.

Readiness state. document.readyState becomes "interactive" when the parser finishes, before deferred and module scripts run (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event).

I couldn't open the spec's parsing chapter (it returned 404), so the DOMContentLoaded timing comes from MDN and web.dev, not the spec's own "the end" steps.

{"evidence": [
{"claim": "Without async or defer, an external classic script blocks parsing while it is fetched and evaluated.", "quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "An async classic script downloads in parallel with parsing and runs as soon as it is available, possibly before parsing completes; a defer script downloads in parallel and runs after parsing finishes.", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Deferred scripts run after parsing but before DOMContentLoaded.", "quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded waits for deferred scripts to load and run.", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Deferred scripts run in document order.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "In the spec, async scripts go into an unordered set that runs as soon as possible, and deferred and non-async module scripts go into a list run after parsing.", "quote": "Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Async scripts are likely to run out of document order and can interrupt parsing.", "quote": "This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "Async scripts are only guaranteed to run before the window load event.", "quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "DOMContentLoaded waits for parsing plus deferred and module scripts, but not for async scripts.", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "Conflicting source: an MDN note says DOMContentLoaded fires before async scripts run, which the spec's 'potentially before parsing completes' wording contradicts.", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "If both attributes are present, async wins.", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer can be added alongside async so legacy browsers fall back to defer behavior.", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer has no effect on inline classic scripts.", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Module scripts are deferred by default and ignore defer; with async they and their dependencies run as soon as available.", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Every module script, inline or external, goes through the async/defer logic, unlike inline classic scripts.", "quote": "If el's type is \"classic\" and el has a src attribute, or el's type is \"module\":", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "MDN describes module scripts as deferred by default.", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.