RunSonnet 5.5repeat 1
Plan-execute-verify on async vs defer on <script>
claude-sonnet-5-5__plan-execute-verify__research-script-loading__r1
Score
1.000
- Cost
- $0.350
- + $0.098 judging
- Latency
- 1m 25s
- setup 0.0s, grading 4.5s
- Turns
- 33
- Tool calls
- 28
- Subagents
- 0
- Tokens in / out
- 279k / 11k
- 225,228 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- The plan was a plan: heldplan-execute-verify.planThe planner’s reply was read through its schema and held at least one step.
- Every verdict was read: heldplan-execute-verify.verdictsEvery review round ended with a verdict read from the schema reply, not a default.
- The loop did what its verdicts said: heldplan-execute-verify.loopEvery verdict that called for a revision got one, and nothing was revised without such a verdict.
- Each call was shown the right tools: heldplan-execute-verify.toolsThe planner was shown no tools, the verifier the read-only ones, and the executor and reviser the full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Plan | 11.9s | $0.018 | 2 | 0 | claude-sonnet-5-5 |
| Execute | 45.8s | $0.196 | 18 | 17 | claude-sonnet-5-5 |
| Verify | 27.6s | $0.135 | 13 | 11 | claude-sonnet-5-5 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies plan-execute-verify --tasks research-script-loading --models claude-sonnet-5-5 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E2. “An `async` or `defer` attribute on a classic external script means the browser fetches it in parallel with HTML parsing.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E2, E3, E4. “meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E5, E6. “An `async` script can therefore run before or after the event.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E7, E8, E9. “whichever script finishes downloading first runs first.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E11, E12. “Both attributes do nothing here.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E13, E14. “The defer attribute has no effect on module scripts.)”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"mdn-script-element":0.4736,"whatwg-html-scripting":0.3361,"mdn-domcontentloaded":1,"webdev-third-party-javascript":0.0309,"webdev-optimize-resource-loading":0.0176}
- cited count
- 3
- verified count
- 3
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 18
- verified quote count
- 18
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async classic scripts are fetched in parallel with parsing and run as soon as available, possibly before parsing completes","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"defer classic scripts are fetched in parallel and run after parsing finishes; with neither attribute the script blocks parsing","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing. If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both\n…[truncated 10 chars]","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute after parsing but before DOMContentLoaded","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"deferred scripts delay DOMContentLoaded until they have loaded and run","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded does not wait for async scripts","quote":"It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"async scripts can run after DOMContentLoaded has already fired, so readyState should be checked","quote":"If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts keep document order","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async scripts go in an unordered set that runs scripts as soon as they are ready","quote":"Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"deferred and module (non-async) parser-inserted scripts go in a list executed when parsing finishes","quote":"Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"if both async and defer are present, async behavior wins","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer has no effect on inline classic scripts","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"inline classic scripts without src execute immediately","quote":"Otherwise, immediately execute the script element el, even if other scripts are already executing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"module scripts are deferred by default and defer has no effect on them","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async module scripts run as soon as they and their dependencies are available","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"inline module scripts wait for parsing to finish and for their dependencies","quote":"Also note that no matter where this script element appears in the document, it will not be evaluated until both document parsing has complete and its dependency (dom-utils.mjs) has been fetched and evaluated.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded waits for deferred and module scripts","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"MDN's note says DOMContentLoaded fires before async script execution, which sits uneasily with the spec's 'potentially before parsing completes'","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"defer may accompany async so legacy browsers fall back to defer behavior","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E2","E3","E4"]},"async-timing":{"status":"met","support":["E1","E5","E6"]},"ordering":{"status":"met","support":["E7","E8","E9"]},"inline-ignored":{"status":"met","support":["E11","E12"]},"modules":{"status":"met","support":["E13","E14"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E2","E3"]},"async-timing":{"status":"met","support":["E1","E5","E6"]},"ordering":{"status":"met","support":["E7","E8","E9"]},"inline-ignored":{"status":"met","support":["E11","E12"]},"modules":{"status":"met","support":["E13","E14"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E2","E3"]},"async-timing":{"status":"met","support":["E1","E5","E6"]},"ordering":{"status":"met","support":["E7","E8","E9"]},"inline-ignored":{"status":"met","support":["E11","E12"]},"modules":{"status":"met","support":["E13","E14"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
An async or defer attribute on a classic external script means the browser fetches it in parallel with HTML parsing. The two differ in when the script runs. defer runs it after parsing finishes and before DOMContentLoaded, in document order. async runs it as soon as it has downloaded, in no guaranteed order.
Parsing and execution timing
async: The HTML Standard says "if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)." (https://html.spec.whatwg.org/multipage/scripting.html) So it can interrupt the parser while it executes, but it never waits for parsing to finish.defer: The same page says the script "will be fetched in parallel and evaluated when the page has finished parsing." MDN adds that a deferred script is "meant to be executed after the document has been parsed, but before firing DOMContentLoaded event." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script)- With neither attribute, the script "is fetched and evaluated immediately, blocking parsing until these are both complete." (https://html.spec.whatwg.org/multipage/scripting.html)
Relation to DOMContentLoaded
- Deferred scripts hold the event back. MDN says they "will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script)
- The
DOMContentLoadedpage confirms that the event "doesn't wait for other things like images, subframes, and async scripts to finish loading." (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event) - An
asyncscript can therefore run before or after the event. It runs before if it arrives while parsing is still going. It runs after if it arrives later. Code in an async script should checkdocument.readyStatebefore waiting forDOMContentLoaded. - The sources differ slightly here. A note on the MDN event page says
DOMContentLoaded"is fired after the execution of these scripts, but before the execution of scripts with async". That reads as if async scripts always come after. The spec text (asyncmay run "potentially before parsing completes") and the same page's remark that an async script may run before or after the event show that the order is not fixed. I treat the order as a race that depends on network timing.
Order with several scripts
defer: MDN says "Scripts with the defer attribute will execute in the order in which they appear in the document." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script) In the spec's processing model, parser-inserted deferred scripts, and module scripts withoutasync, are appended to a "list of scripts that will execute when the document has finished parsing". That is an ordered list.async: The processing model puts these scripts in a "set of scripts that will execute as soon as possible". Each one executes when its own result is ready. The spec states no ordering for that set, so whichever script finishes downloading first runs first. This is my reading of the data structure. I did not find an explicit "async is unordered" sentence in the pages I opened.- If both attributes are present,
asyncwins. MDN says "the element will act as if only the async attribute is specified." The spec saysdefermay be added alongsideasyncso that legacy browsers withoutasyncsupport fall back to deferred behavior rather than blocking. - Scripts added dynamically (not parser-inserted) go into a separate "list of scripts that will execute in order as soon as possible" unless they have
async. The spec's "force async" flag makes parser-inserted scripts that failed to run behave as async if they are later executed.
Inline classic scripts
- Both attributes do nothing here. MDN warns that each "must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect." (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script)
- The spec only runs the async/defer branches for a classic script with
src, or for a module. Otherwise it will "immediately execute the script element el, even if other scripts are already executing." (https://html.spec.whatwg.org/multipage/scripting.html)
type="module" scripts
- Modules are deferred by default. The spec says that, without
async, "the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)" (https://html.spec.whatwg.org/multipage/scripting.html) MDN agrees: "The defer attribute has no effect on module scripts — they defer by default." - With
async, the module and all its dependencies are fetched in parallel, and it is "evaluated as soon as it is available (potentially before parsing completes)". It then has the same unordered, race-with-DOMContentLoadedbehavior as an async classic script. - Inline modules differ from inline classic scripts. The spec's processing model sends every module through the async/defer branches, including inline ones. The spec's inline module example says such a script "will not be evaluated until both document parsing has complete and its dependency (dom-utils.mjs) has been fetched and evaluated." So an inline module is deferred. With
asyncit runs once its imports are ready. - Deferred and module scripts both count toward
DOMContentLoaded. MDN says the event waits for "all deferred scripts (<script defer src="…"> and <script type="module">)".
Sources
- The cited MDN script page, MDN
DOMContentLoadedpage and spec section were all opened and read. - I could not open the javascript.info page (404). I did not use the web.dev search results, since I never opened them.
{"evidence": [
{"claim": "async classic scripts are fetched in parallel with parsing and run as soon as available, possibly before parsing completes", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer classic scripts are fetched in parallel and run after parsing finishes; with neither attribute the script blocks parsing", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing. If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer scripts execute after parsing but before DOMContentLoaded", "quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "deferred scripts delay DOMContentLoaded until they have loaded and run", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded does not wait for async scripts", "quote": "It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "async scripts can run after DOMContentLoaded has already fired, so readyState should be checked", "quote": "If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "defer scripts keep document order", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "async scripts go in an unordered set that runs scripts as soon as they are ready", "quote": "Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "deferred and module (non-async) parser-inserted scripts go in a list executed when parsing finishes", "quote": "Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "if both async and defer are present, async behavior wins", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer has no effect on inline classic scripts", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "inline classic scripts without src execute immediately", "quote": "Otherwise, immediately execute the script element el, even if other scripts are already executing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "module scripts are deferred by default and defer has no effect on them", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async module scripts run as soon as they and their dependencies are available", "quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "inline module scripts wait for parsing to finish and for their dependencies", "quote": "Also note that no matter where this script element appears in the document, it will not be evaluated until both document parsing has complete and its dependency (dom-utils.mjs) has been fetched and evaluated.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "DOMContentLoaded waits for deferred and module scripts", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "MDN's note says DOMContentLoaded fires before async script execution, which sits uneasily with the spec's 'potentially before parsing completes'", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "defer may accompany async so legacy browsers fall back to defer behavior", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.