ScaffoldBench

RunSonnet 5.5repeat 1

Single on async vs defer on <script>

claude-sonnet-5-5__single__research-script-loading__r1

Score

1.000

passed
design ran as designed
Cost
$0.137
+ $0.081 judging
Latency
36.9s
setup 0.0s, grading 5.5s
Turns
14
Tool calls
13
Subagents
0
Tokens in / out
127k / 4.9k
109,419 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.

Phases

Main: 0.0s to 36.9s, finished, 13 tool calls, $0.137

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Main36.9s$0.1371413claude-sonnet-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies single --tasks research-script-loading --models claude-sonnet-5-5 --repeats 1

Score breakdown

  • passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.
    1.00weight 0.13

    met, backed by verified evidence E1, E2. “`async` and `defer` both make an external script download in parallel with HTML parsing.”

  • passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.
    1.00weight 0.19

    met, backed by verified evidence E2, E5. “Deferred scripts run after parsing finishes and before DOMContentLoaded fires.”

  • passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.
    1.00weight 0.19

    met, backed by verified evidence E1. “If it arrives mid-parse, it interrupts the parser while it runs.”

  • passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.
    1.00weight 0.19

    met, backed by verified evidence E6, E7. “Deferred scripts run in document order.”

  • passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).
    1.00weight 0.13

    met, backed by verified evidence E8. “For classic scripts, `async` and `defer` do nothing without `src`.”

  • passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.
    1.00weight 0.19

    met, backed by verified evidence E3, E9, E10. “`defer` attribute.** It has no effect on module scripts, according to both the spec and MDN.”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
opened documents
["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded"]
opened by role
{"secondary":["mdn-script-element","mdn-domcontentloaded"],"primary":["whatwg-html-scripting"]}
displayed share
{"mdn-script-element":0.8439,"whatwg-html-scripting":0.2207,"mdn-domcontentloaded":0.4247,"webdev-third-party-javascript":0.0303,"webdev-optimize-resource-loading":0.0154}
cited count
3
verified count
3
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
13
verified quote count
13
verified quote rate
1
evidence
[{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Classic external async scripts are fetched in parallel and run as soon as available, possibly before parsing completes; defer scripts run when parsing is finished; with neither, parsing blocks.","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"With defer (and no async), the classic script is fetched in parallel and evaluated after parsing; with neither attribute it blocks parsing.","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing. If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both\n…[truncated 10 chars]","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts are deferred by default, and defer has no effect on them; async makes them run as soon as available.","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Specifying both async and defer is allowed so legacy browsers fall back to defer behavior.","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run before DOMContentLoaded and delay that event until they finish; they run in document order.","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts execute in document order.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts have no execution order guarantee.","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async and defer have no effect on classic inline scripts (no src).","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Defer has no effect on module scripts, which defer by default.","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Module scripts with async and all their dependencies are fetched in parallel and evaluated as soon as available.","quote":"For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Plain classic scripts and inline non-module scripts run immediately and block the parser.","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"In the spec algorithm, async scripts go into a set that executes as soon as possible, while defer/module scripts go into a list run when parsing finishes.","quote":"Otherwise, if el has a defer attribute or el's type is \"module\":","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"MDN's DOMContentLoaded page states deferred/module scripts run before DOMContentLoaded and async scripts after it (a simplification versus the script page).","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"}]
judge unverified quotes
0
judge verdicts
[{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E2","E5"]},"async-timing":{"status":"met","support":["E1"]},"ordering":{"status":"met","support":["E6","E7"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E3","E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E2","E5"]},"async-timing":{"status":"met","support":["E1"]},"ordering":{"status":"met","support":["E6","E7"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E3","E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E2","E5"]},"async-timing":{"status":"met","support":["E1"]},"ordering":{"status":"met","support":["E6","E7"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E3","E9","E10"]}}]
judge failures
0
grounded weight share
1

Final answer

async and defer both make an external script download in parallel with HTML parsing. They differ in when the script runs and in whether several scripts keep their order.

Classic external scripts (<script src>)

  • Neither attribute. The script is fetched and run immediately, and parsing stops until both are done. The spec says so, and MDN says the same: "Scripts without async, defer or type="module" attributes, as well as inline scripts without the type="module" attribute, are fetched and executed immediately before the browser continues to parse the page."
  • async. The script is fetched in parallel and runs as soon as it arrives, which can be before parsing finishes. If it arrives mid-parse, it interrupts the parser while it runs.
  • defer. The script is fetched in parallel but runs only after the document has finished parsing.
  • Both together. MDN says the element acts as if only async is set. The spec says authors may write both so that legacy browsers that only know defer fall back to deferred behaviour instead of blocking.

Timing relative to DOMContentLoaded

  • defer. Deferred scripts run after parsing finishes and before DOMContentLoaded fires. MDN adds that a deferred script holds DOMContentLoaded back until it has loaded and finished running. While deferred and module scripts run, document.readyState is "interactive".
  • async. There is no fixed relationship. An async script runs whenever its download completes, which can be during parsing, before DOMContentLoaded, or after it.
  • A source disagreement. MDN's DOMContentLoaded page says the event fires "before the execution of scripts with async". That is a simplification. MDN's own <script> page and the spec say async scripts can run before parsing completes. I would not rely on async scripts running after DOMContentLoaded.

Order with several scripts

  • async. There is no ordering guarantee. In the spec's processing model, each async script is added to a set and runs as soon as its own result is ready. MDN's example is three async scripts where jquery.js may load before or after the other two, so dependent code can fail.
  • defer. Deferred scripts run in document order. The spec puts them in a list that is processed when parsing finishes. MDN says "Scripts with the defer attribute will execute in the order in which they appear in the document."

Inline scripts

  • Classic inline scripts. For classic scripts, async and defer do nothing without src. MDN warns that without src they have "no effect". In the spec's algorithm, the async and defer branches apply only to classic scripts that have a src, or to module scripts. An inline classic script runs immediately, blocking the parser.
  • Inline module scripts. async does apply to these, and the spec's algorithm treats them like external modules. I did not find a source that discusses defer on inline modules specifically, but the spec says defer has no effect on any module script.

type="module"

  • Default timing. Module scripts are deferred by default. The spec says that, without async, the module and its dependencies are fetched in parallel and "evaluated when the page has finished parsing". Their place in the order is the same as for defer.
  • defer attribute. It has no effect on module scripts, according to both the spec and MDN.
  • async on modules. The module and all its dependencies are fetched in parallel, and it runs as soon as they are ready. Like classic async, this has no order guarantee and can happen before parsing completes.
  • Fetching. Cross-origin module scripts must use CORS.

Sources: https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script, https://html.spec.whatwg.org/multipage/scripting.html and https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event.

{"evidence": [
{"claim": "Classic external async scripts are fetched in parallel and run as soon as available, possibly before parsing completes; defer scripts run when parsing is finished; with neither, parsing blocks.", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "With defer (and no async), the classic script is fetched in parallel and evaluated after parsing; with neither attribute it blocks parsing.", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing. If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Module scripts are deferred by default, and defer has no effect on them; async makes them run as soon as available.", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Specifying both async and defer is allowed so legacy browsers fall back to defer behavior.", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Deferred scripts run before DOMContentLoaded and delay that event until they finish; they run in document order.", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Deferred scripts execute in document order.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Async scripts have no execution order guarantee.", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Async and defer have no effect on classic inline scripts (no src).", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Defer has no effect on module scripts, which defer by default.", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Module scripts with async and all their dependencies are fetched in parallel and evaluated as soon as available.", "quote": "For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Plain classic scripts and inline non-module scripts run immediately and block the parser.", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "In the spec algorithm, async scripts go into a set that executes as soon as possible, while defer/module scripts go into a list run when parsing finishes.", "quote": "Otherwise, if el has a defer attribute or el's type is \"module\":", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "MDN's DOMContentLoaded page states deferred/module scripts run before DOMContentLoaded and async scripts after it (a simplification versus the script page).", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.