ScaffoldBench

RunSonnet 5.5repeat 1

Orchestrator-workers on async vs defer on <script>

claude-sonnet-5-5__orchestrator-workers__research-script-loading__r1

Score

1.000

passed
design ran as designed
Cost
$0.591
+ $0.098 judging
Latency
1m 29s
setup 0.0s, grading 4.8s
Turns
41
Tool calls
35
Subagents
3
Tokens in / out
328k / 23k
235,849 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • The task was decomposed: heldorchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
  • The fan-out happened: heldorchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
  • The synthesis wrote the answer: heldorchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
  • Each call was shown the right tools: heldorchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.

Phases

Decompose: 0.0s to 9.8s, finished, 0 tool calls, $0.017
Worker 1: spec-classic: 9.8s to 1m 1s, finished, 11 tool calls, $0.156
Worker 2: inline-module: 9.8s to 40.6s, finished, 7 tool calls, $0.110
Worker 3: practical-guides: 9.8s to 49.9s, finished, 17 tool calls, $0.192
Synthesize: 1m 1s to 1m 29s, finished, 0 tool calls, $0.117

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Decompose9.8s$0.01720claude-sonnet-5-5
Worker 1: spec-classic51.2s$0.1561211claude-sonnet-5-5
Worker 2: inline-module30.7s$0.11087claude-sonnet-5-5
Worker 3: practical-guides40.0s$0.1921817claude-sonnet-5-5
Synthesize27.9s$0.11710claude-sonnet-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies orchestrator-workers --tasks research-script-loading --models claude-sonnet-5-5 --repeats 1

Score breakdown

  • passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.
    1.00weight 0.13

    met, backed by verified evidence E1, E2, E3. “`async` and `defer` both make the fetch happen in parallel with parsing.”

  • passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.
    1.00weight 0.19

    met, backed by verified evidence E3, E4, E5. “MDN says deferred scripts run after the document is parsed but before `DOMContentLoaded` fires.”

  • passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.
    1.00weight 0.19

    met, backed by verified evidence E2, E6, E7. “The script is fetched in parallel with parsing and evaluated as soon as it is available. This can happen before parsing completes, so an async script can interrupt parsing at the moment it executes.”

  • passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.
    1.00weight 0.19

    met, backed by verified evidence E9, E10. “Scripts run in document order.”

  • passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).
    1.00weight 0.13

    met, backed by verified evidence E15, E17. “MDN warns that `async` and `defer` must not be used without `src` and have no effect there.”

  • passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.
    1.00weight 0.19

    met, backed by verified evidence E18, E19, E20, E21. “Module scripts are deferred by default.”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
opened documents
["mdn-script-element","whatwg-html-scripting","webdev-optimize-resource-loading","mdn-domcontentloaded","webdev-third-party-javascript"]
opened by role
{"secondary":["mdn-script-element","webdev-optimize-resource-loading","mdn-domcontentloaded","webdev-third-party-javascript"],"primary":["whatwg-html-scripting"]}
displayed share
{"mdn-script-element":0.8439,"whatwg-html-scripting":0.3837,"mdn-domcontentloaded":1,"webdev-third-party-javascript":0.9618,"webdev-optimize-resource-loading":0.3839}
cited count
4
verified count
4
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
22
verified quote count
22
verified quote rate
1
evidence
[{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"A classic external script with no attribute blocks parsing while it is fetched and run","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async classic external scripts are fetched in parallel and evaluated as soon as available, possibly before parsing completes","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"defer classic external scripts are fetched in parallel and evaluated after parsing finishes","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts run after parsing and before DOMContentLoaded","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"DOMContentLoaded waits for deferred scripts to load and finish evaluating","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded does not wait for async scripts","quote":"It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"A late-arriving async script runs after DOMContentLoaded","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded waits for deferred and module scripts","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute in document order","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async scripts have no execution-order guarantee","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"If both async and defer are present, async wins","quote":"If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Spec rationale for specifying both attributes: legacy browsers fall back to defer","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Dynamically injected scripts behave like async scripts","quote":"scripts loaded by injecting <script> markup into the DOM with JavaScript behave like async scripts.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Script elements have a force async flag that is initially true","quote":"A script element has a force async boolean, initially true.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async/defer have no effect on inline classic scripts","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Inline classic scripts and plain scripts run immediately, blocking parsing","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The spec's async/defer branches apply only to classic scripts with src or to module scripts","quote":"If el's type is \"classic\" and el has a src attribute, or el's type is \"module\":","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts are deferred by default and go into the run-after-parsing list regardless of src","quote":"Otherwise, if el has a defer attribute or el's type is \"module\":","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Without async, module scripts are fetched in parallel and evaluated after parsing","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer has no effect on module scripts","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async on a module script runs it as soon as it and its dependencies are available","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Inline module execution is always queued as a task, never synchronous","quote":"Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.","verified":true,"document":"whatwg-html-scripting"}]
judge unverified quotes
0
judge verdicts
[{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E3","E4","E5"]},"async-timing":{"status":"met","support":["E2","E6","E7"]},"ordering":{"status":"met","support":["E9","E10"]},"inline-ignored":{"status":"met","support":["E15","E17"]},"modules":{"status":"met","support":["E18","E19","E20","E21"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E4","E5","E3"]},"async-timing":{"status":"met","support":["E2","E6","E7"]},"ordering":{"status":"met","support":["E9","E10"]},"inline-ignored":{"status":"met","support":["E15","E17"]},"modules":{"status":"met","support":["E18","E19","E20","E21"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E3","E4","E5"]},"async-timing":{"status":"met","support":["E2","E6","E7"]},"ordering":{"status":"met","support":["E9","E10"]},"inline-ignored":{"status":"met","support":["E15","E17"]},"modules":{"status":"met","support":["E18","E19","E20","E21"]}}]
judge failures
0
grounded weight share
1

Final answer

Short version

  • A plain <script src> blocks the HTML parser while it is fetched and run.
  • async and defer both make the fetch happen in parallel with parsing. They differ in when the script runs and in what order scripts run.
  • defer runs scripts after parsing, in document order, before DOMContentLoaded.
  • async runs each script as soon as it arrives, in no guaranteed order, and DOMContentLoaded does not wait for it.

Interaction with HTML parsing

  • No attribute. The spec says the script "is fetched and evaluated immediately, blocking parsing until these are both complete."
  • async on an external classic script. The script is fetched in parallel with parsing and evaluated as soon as it is available. This can happen before parsing completes, so an async script can interrupt parsing at the moment it executes.
  • defer on an external classic script. The script is fetched in parallel and evaluated once the page has finished parsing, so it never interrupts the parser.

Timing relative to DOMContentLoaded

  • defer. MDN says deferred scripts run after the document is parsed but before DOMContentLoaded fires. It also says the event is held back until they have loaded and finished evaluating.
  • async. DOMContentLoaded does not wait for async scripts. An async script that downloads quickly can run before DOMContentLoaded, which follows from the spec's "potentially before parsing completes". One that downloads late runs after it. MDN's event page puts it this way for the late case: "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async."
  • Sourcing caveat. The DOMContentLoaded timing rests on MDN. The research workers could not open the spec's parsing section, which would define when the parser fires the event. The two sources describe async timing from different angles rather than contradicting each other.

Ordering with several scripts

  • defer. Scripts run in document order. In the spec's processing model, parser-inserted deferred scripts go into a list that runs when parsing finishes.
  • async. There is no order guarantee. Each script runs when its own download completes. MDN's example is that a library like jquery.js may load before or after the scripts that depend on it, so dependent code can break.
  • Both attributes together. MDN says the element acts as if only async were specified. The spec says authors may add defer anyway so that legacy browsers that only support defer fall back to deferred behavior instead of blocking.
  • Dynamically inserted scripts. web.dev says scripts injected into the DOM with JavaScript behave like async scripts. The spec's mechanism is a "force async" flag that starts true. Setting script.async = false clears it, and the workers read from the spec's steps that such scripts then run in order. No page stated that last point in plain language.

Inline scripts

  • Classic inline scripts (no src).

    • MDN warns that async and defer must not be used without src and have no effect there.
    • The spec's algorithm only applies the async and defer branches to classic scripts with src, or to module scripts. A classic inline script is executed immediately and blocks the parser.
  • MDN's wording is stronger than the spec's. The spec does not contain a sentence saying inline classic scripts ignore these attributes. It simply does not use them in that case.

type="module" scripts

  • Default behavior.

    • Module scripts are deferred by default. They go into the same run-after-parsing, in-order list as defer scripts. They also hold up DOMContentLoaded the same way, according to MDN's event page, which lists modules alongside deferred scripts.
    • The spec's deferred-branch condition ("if el has a defer attribute or el's type is "module"") has no src requirement, so inline modules are deferred too. web.dev also says inline modules are deferred. No page the workers opened stated the inline-module case in so many words.
  • defer attribute. It has no effect on modules.
  • async on a module. The module and all its dependencies are fetched in parallel and evaluated as soon as they are available, possibly before parsing completes. The spec's attribute table allows async on inline modules too. For an inline module, the spec says a task is queued, so execution is never synchronous at the point the parser reaches the tag.

Practical rule

Use defer for scripts that need the DOM or depend on each other. Use async for independent scripts such as analytics, where order does not matter.

Gaps

  • The javascript.info page returned 404, so it is not used.
  • MDN says an executing async script "blocks the page from rendering". web.dev says async scripts "can interrupt DOM building". This was not checked against the spec and is not a core claim.
{"evidence": [
{"claim": "A classic external script with no attribute blocks parsing while it is fetched and run", "quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async classic external scripts are fetched in parallel and evaluated as soon as available, possibly before parsing completes", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer classic external scripts are fetched in parallel and evaluated after parsing finishes", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer scripts run after parsing and before DOMContentLoaded", "quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded waits for deferred scripts to load and finish evaluating", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded does not wait for async scripts", "quote": "It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "A late-arriving async script runs after DOMContentLoaded", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "DOMContentLoaded waits for deferred and module scripts", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "defer scripts execute in document order", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "async scripts have no execution-order guarantee", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "If both async and defer are present, async wins", "quote": "If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Spec rationale for specifying both attributes: legacy browsers fall back to defer", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Dynamically injected scripts behave like async scripts", "quote": "scripts loaded by injecting <script> markup into the DOM with JavaScript behave like async scripts.", "url": "https://web.dev/learn/performance/optimize-resource-loading"},
{"claim": "Script elements have a force async flag that is initially true", "quote": "A script element has a force async boolean, initially true.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async/defer have no effect on inline classic scripts", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Inline classic scripts and plain scripts run immediately, blocking parsing", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "The spec's async/defer branches apply only to classic scripts with src or to module scripts", "quote": "If el's type is \"classic\" and el has a src attribute, or el's type is \"module\":", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Module scripts are deferred by default and go into the run-after-parsing list regardless of src", "quote": "Otherwise, if el has a defer attribute or el's type is \"module\":", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Without async, module scripts are fetched in parallel and evaluated after parsing", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer has no effect on module scripts", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "async on a module script runs it as soon as it and its dependencies are available", "quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Inline module execution is always queued as a task, never synchronous", "quote": "Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.