RunSonnet 5.5repeat 2
Orchestrator-workers on async vs defer on <script>
claude-sonnet-5-5__orchestrator-workers__research-script-loading__r2
Score
1.000
- Cost
- $0.649
- + $0.077 judging
- Latency
- 1m 22s
- setup 0.0s, grading 4.5s
- Turns
- 53
- Tool calls
- 47
- Subagents
- 3
- Tokens in / out
- 480k / 23k
- 380,855 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- The task was decomposed: heldorchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
- The fan-out happened: heldorchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
- The synthesis wrote the answer: heldorchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
- Each call was shown the right tools: heldorchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Decompose | 9.9s | $0.017 | 2 | 0 | claude-sonnet-5-5 |
| Worker 1: spec-async-defer | 40.9s | $0.158 | 15 | 14 | claude-sonnet-5-5 |
| Worker 2: inline-and-module | 38.3s | $0.143 | 14 | 13 | claude-sonnet-5-5 |
| Worker 3: practical-explainers | 44.8s | $0.224 | 21 | 20 | claude-sonnet-5-5 |
| Synthesize | 27.3s | $0.107 | 1 | 0 | claude-sonnet-5-5 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies orchestrator-workers --tasks research-script-loading --models claude-sonnet-5-5 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E2, E3. “Both attributes make an external classic script download in parallel with HTML parsing instead of blocking the parser.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E2, E4. “deferred scripts run after the document has been parsed but before `DOMContentLoaded`. The event is held back until they have loaded and finished evaluating”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E8. “I would treat "may be before or after" as the safe statement.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E5, E6, E7. “deferred scripts run in the order they appear in the document.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E12. “Inline classic scripts (no `src`):** `async` and `defer` have no effect.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E13, E14, E17. “They are deferred by default, and `defer` has no effect on them.”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","mdn-domcontentloaded","whatwg-html-scripting","mdn-js-modules","webdev-third-party-javascript","webdev-optimize-resource-loading"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","mdn-js-modules","webdev-third-party-javascript","webdev-optimize-resource-loading"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"mdn-script-element":0.8847,"mdn-domcontentloaded":1,"mdn-js-modules":0.1742,"webdev-third-party-javascript":0.9618,"webdev-optimize-resource-loading":0.3839,"whatwg-html-scripting":0.4308}
- cited count
- 4
- verified count
- 4
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 17
- verified quote count
- 17
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Spec: an external classic async script is fetched in parallel with parsing and evaluated as soon as available, possibly before parsing completes","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Spec: a defer script is fetched in parallel and evaluated after parsing finishes","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Spec: a script with neither attribute blocks parsing while it is fetched and evaluated","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts hold back DOMContentLoaded until they have loaded and finished evaluating","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run in document order","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts have no ordering guarantee","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Async scripts may run out of order while defer scripts keep markup order","quote":"Additionally, async scripts may execute out-of-order, while defer scripts are executed in the order in which they appear in the markup.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"MDN says an async or dynamically injected script may run after DOMContentLoaded has already fired","quote":"If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"MDN also states DOMContentLoaded fires after defer/module scripts but before async scripts, which is stricter than the spec wording","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"When both attributes are present, MDN says async behavior wins","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The spec says defer alongside async is a fallback for legacy browsers","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer (and likewise async) has no effect on inline classic scripts","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Module scripts are deferred by default and defer has no effect on them","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"With async, a module and its dependencies are fetched in parallel and evaluated as soon as available","quote":"the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"In the spec's processing model, deferred classic scripts and non-async modules share the run-after-parsing list","quote":"Otherwise, if el has a defer attribute or el's type is \"module\":","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Inline module scripts are never executed synchronously","quote":"Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Dynamically injected scripts behave like async scripts, while module scripts are deferred","quote":"Note: Scripts loaded using the type=\"module\" attribute are deferred by default, while scripts loaded by injecting <script> markup into the DOM with JavaScript behave like async scripts.","verified":true,"document":"webdev-optimize-resource-loading"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E2","E4"]},"async-timing":{"status":"met","support":["E1","E8"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E12"]},"modules":{"status":"met","support":["E13","E14","E17"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E2","E4"]},"async-timing":{"status":"met","support":["E1","E8"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E12"]},"modules":{"status":"met","support":["E13","E14"]}},{"parallel-download":{"status":"met","support":["E1","E2","E3"]},"defer-timing":{"status":"met","support":["E2","E4"]},"async-timing":{"status":"met","support":["E1","E8"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E12"]},"modules":{"status":"met","support":["E13","E14"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
Short version
Both attributes make an external classic script download in parallel with HTML parsing instead of blocking the parser. They differ in when the script runs and in what ordering you can rely on:
deferruns the script after parsing finishes, in document order, beforeDOMContentLoaded.asyncruns the script as soon as it has downloaded, with no ordering guarantee and no fixed position relative toDOMContentLoaded.
Parsing
A plain <script src> with neither attribute is fetched and evaluated immediately, blocking parsing until both are done. The WHATWG spec describes the two attributes this way (https://html.spec.whatwg.org/multipage/scripting.html):
async: the script is fetched in parallel with parsing and evaluated as soon as it is available, "potentially before parsing completes". So an async script can interrupt the parser when it arrives mid-parse.defer: the script is fetched in parallel and evaluated when the page has finished parsing, so it never interrupts the parser.
Timing relative to DOMContentLoaded
defer: MDN says deferred scripts run after the document has been parsed but beforeDOMContentLoaded. The event is held back until they have loaded and finished evaluating (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).document.readyStateis "interactive" while they run.async: the timing is not fixed.- The spec wording ("as soon as it is available") means a fast async script can run before parsing ends, and so before
DOMContentLoaded. - MDN's DOMContentLoaded page says an async or dynamically injected script has often run after the event has already fired (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event).
- The same MDN page also says
DOMContentLoadedfires "before the execution of scripts with async". That is stricter than the spec wording and sits awkwardly with the possibility of an early async script. I would treat "may be before or after" as the safe statement. It depends on network timing.
- The spec wording ("as soon as it is available") means a fast async script can run before parsing ends, and so before
- The spec's parsing and lifecycle pages returned 404 for the workers, so the exact point where the parser fires
DOMContentLoadedcomes from MDN, not from the spec.
Order with several scripts
defer: deferred scripts run in the order they appear in the document. The spec queues them on a list of scripts that run when parsing finishes, and MDN states the document-order guarantee.async: there is no guarantee. The spec keeps async scripts in a set of scripts that run as soon as possible, and MDN says "You get no guarantee that scripts will run in any specific order." web.dev says the same (https://web.dev/learn/performance/optimize-resource-loading).Both attributes on one element: the sources frame this differently.
- MDN says the element acts as if only
asyncwere specified. - The spec says
defermay be added alongsideasyncso that legacy browsers that support onlydeferfall back to deferred behavior instead of blocking. - These do not conflict. Modern browsers use
async, and thedeferis a fallback.
- MDN says the element acts as if only
Inline scripts and type="module"
- Inline classic scripts (no
src):asyncanddeferhave no effect. MDN says they must not be used withoutsrc. The script runs immediately and blocks the parser. Module scripts:
- They are deferred by default, and
deferhas no effect on them. The spec puts a module withoutasyncon the same run-after-parsing list as classicdeferscripts, so modules and deferred classic scripts run in document order and beforeDOMContentLoaded. asyncdoes work on modules. The module and all its dependencies are fetched in parallel and evaluated as soon as available, potentially before parsing completes. That makes it unordered, like classicasync.- Inline modules are also deferred by default, per web.dev (a worker report), and the spec's attribute table allows
asyncon them. An inline module is never executed synchronously. The spec queues a task even when it has no dependencies.
- They are deferred by default, and
- Dynamically inserted scripts: web.dev says scripts injected into the DOM behave like async scripts. The spec's "force async" flag explains this, and setting
script.async = falseclears it. No page I have states the resulting ordering recipe directly. That the scripts then run in insertion order is the worker's reading of the spec algorithm, not a quoted statement.
{"evidence": [
{"claim": "Spec: an external classic async script is fetched in parallel with parsing and evaluated as soon as available, possibly before parsing completes", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Spec: a defer script is fetched in parallel and evaluated after parsing finishes", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Spec: a script with neither attribute blocks parsing while it is fetched and evaluated", "quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Deferred scripts hold back DOMContentLoaded until they have loaded and finished evaluating", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Deferred scripts run in document order", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Async scripts have no ordering guarantee", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Async scripts may run out of order while defer scripts keep markup order", "quote": "Additionally, async scripts may execute out-of-order, while defer scripts are executed in the order in which they appear in the markup.", "url": "https://web.dev/learn/performance/optimize-resource-loading"},
{"claim": "MDN says an async or dynamically injected script may run after DOMContentLoaded has already fired", "quote": "If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "MDN also states DOMContentLoaded fires after defer/module scripts but before async scripts, which is stricter than the spec wording", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "When both attributes are present, MDN says async behavior wins", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "The spec says defer alongside async is a fallback for legacy browsers", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer (and likewise async) has no effect on inline classic scripts", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Module scripts are deferred by default and defer has no effect on them", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "With async, a module and its dependencies are fetched in parallel and evaluated as soon as available", "quote": "the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "In the spec's processing model, deferred classic scripts and non-async modules share the run-after-parsing list", "quote": "Otherwise, if el has a defer attribute or el's type is \"module\":", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Inline module scripts are never executed synchronously", "quote": "Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Dynamically injected scripts behave like async scripts, while module scripts are deferred", "quote": "Note: Scripts loaded using the type=\"module\" attribute are deferred by default, while scripts loaded by injecting <script> markup into the DOM with JavaScript behave like async scripts.", "url": "https://web.dev/learn/performance/optimize-resource-loading"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.