RunHaiku 4.5repeat 2
Propose-select on async vs defer on <script>
claude-haiku-4-5-20251001__propose-select__research-script-loading__r2
Score
1.000
- Cost
- $0.399
- + $0.083 judging
- Latency
- 2m 2s
- setup 0.0s, grading 5.1s
- Turns
- 63
- Tool calls
- 58
- Subagents
- 3
- Tokens in / out
- 877k / 23k
- 757,159 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- All three proposers ran: heldpropose-select.proposalsThree proposers ran in parallel, each from its own angle, and none was starved while another ran.
- The selector chose: heldpropose-select.selectionThe selector picked a proposal by a valid number, read from its schema reply and not a default.
- Each call was shown the right tools: heldpropose-select.toolsEach proposer was shown the read-only tools, the selector none, and the executor the full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Proposal 1 (policy-first) | 1m 12s | $0.123 | 22 | 21 | claude-haiku-4-5-20251001 |
| Proposal 2 (state-first) | 55.7s | $0.091 | 15 | 14 | claude-haiku-4-5-20251001 |
| Proposal 3 (outcome-first) | 1m 6s | $0.128 | 24 | 23 | claude-haiku-4-5-20251001 |
| Select | 50.0s | $0.057 | 2 | 0 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies propose-select --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1. “Both `async` and `defer` allow the HTML parser to continue parsing while the script downloads”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E4, E6. “Deferred scripts execute after HTML parsing completes but **before** the `DOMContentLoaded` event fires.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E2, E7, E13. “async scripts can execute while parsing is still happening (potentially blocking it when they run)”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E5, E12, E3, E13. “**With `defer`**: Strict document order.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E8. “The `async` and `defer` attributes have **no effect** on inline scripts.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E9, E10, E11. “**The `defer` attribute has no effect** on module scripts because they defer by default.”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","whatwg-html-scripting","webdev-third-party-javascript","mdn-domcontentloaded","webdev-optimize-resource-loading","mdn-js-modules"]
- opened by role
- {"secondary":["mdn-script-element","webdev-third-party-javascript","mdn-domcontentloaded","webdev-optimize-resource-loading","mdn-js-modules"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"webdev-third-party-javascript":1,"mdn-script-element":0.8844,"whatwg-html-scripting":0.2261,"webdev-optimize-resource-loading":1,"mdn-domcontentloaded":1,"mdn-js-modules":0.361}
- cited count
- 4
- verified count
- 4
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 14
- verified quote count
- 14
- verified quote rate
- 1
- evidence
- [{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The async attribute tells the browser that a classic script can be fetched in parallel to parsing and evaluated as soon as available.","quote":"For classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can interrupt DOM building if they finish downloading while the parser is still at work.","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts do not guarantee any specific execution order when there are multiple scripts.","quote":"This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The defer attribute tells the browser to execute scripts after the document has been parsed.","quote":"This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts execute in the order they appear in the document.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"The DOMContentLoaded event fires after deferred scripts and module scripts execute.","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded does not wait for async scripts.","quote":"It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"The async and defer attributes have no effect on inline scripts.","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Module scripts defer by default, with no effect from the defer attribute.","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"When async is specified on a module script, it executes as soon as available like async classic scripts.","quote":"For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Module scripts and inline scripts with type=module are automatically deferred.","quote":"async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Scripts with async may execute out-of-order while defer scripts execute in document order.","quote":"Additionally, async scripts may execute out-of-order, while defer scripts are executed in the order in which they appear in the markup.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async classic scripts execute as soon as download completes, not in guaranteed order.","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"If both async and defer are specified, async takes precedence.","quote":"If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1"]},"defer-timing":{"status":"met","support":["E4","E6"]},"async-timing":{"status":"met","support":["E1","E2","E7","E13"]},"ordering":{"status":"met","support":["E5","E12","E3","E13"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10","E11"]}},{"parallel-download":{"status":"met","support":["E1"]},"defer-timing":{"status":"met","support":["E4","E6"]},"async-timing":{"status":"met","support":["E1","E2","E7"]},"ordering":{"status":"met","support":["E5","E12","E3","E13"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10","E11"]}},{"parallel-download":{"status":"met","support":["E1"]},"defer-timing":{"status":"met","support":["E4","E6"]},"async-timing":{"status":"met","support":["E1","E2","E7","E13"]},"ordering":{"status":"met","support":["E5","E12","E3","E13"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10","E11"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
How async and defer Work in HTML Script Elements
The async Attribute
The async attribute applies only to external scripts (those with a src attribute, not inline scripts). When present, it tells the browser that the script can be fetched in parallel with HTML parsing and executed as soon as the download completes, without waiting for the parser to finish. This means async scripts can interrupt HTML parsing while they execute. A critical point is that async scripts provide no guarantee about execution order—if you have multiple async scripts, they execute in whatever order they finish downloading, not necessarily in the order they appear in the HTML. Async scripts execute after they download but before the window's load event, though they may execute before or after the DOMContentLoaded event depending on timing.
The defer Attribute
The defer attribute also applies only to external scripts. When present, the script is fetched in parallel with parsing (like async), but execution is delayed until the HTML parsing is completely finished. Critically, defer guarantees execution order—deferred scripts execute in the order they appear in the document. Deferred scripts execute after HTML parsing completes but before the DOMContentLoaded event fires. This makes defer ideal when scripts depend on the DOM being constructed and when execution order matters.
Interaction with HTML Parsing
Both async and defer allow the HTML parser to continue parsing while the script downloads, which is the main performance benefit. However, they differ in execution timing: async scripts can execute while parsing is still happening (potentially blocking it when they run), while defer scripts always wait for parsing to complete. Without either attribute, external scripts are parser-blocking—the browser must download and execute them before continuing to parse the HTML.
Relative to DOMContentLoaded
The DOMContentLoaded event fires when the HTML document is completely parsed and all deferred scripts (those with defer or type="module") have downloaded and executed. Critically, DOMContentLoaded does not wait for async scripts. This means:
- Deferred scripts: Execute before
DOMContentLoaded - Async scripts: May execute before or after
DOMContentLoadeddepending on download timing - Module scripts (default behavior): Treated like deferred scripts by default, executing before
DOMContentLoaded
Multiple Scripts and Execution Order Guarantees
When you have several external classic scripts:
- With
async: No specific order guarantee. Scripts execute as soon as they download, potentially in a different order than they appear in the HTML. - With
defer: Strict document order. Scripts are guaranteed to execute in the order they appear, regardless of download speed. - With both attributes: If both are specified,
asynctakes precedence, and the element acts as if onlyasyncis present.
Inline Scripts (<script> without src)
The async and defer attributes have no effect on inline scripts. According to the HTML standard, these attributes must not be used on inline scripts because there's nothing to defer or parallelize—the script content is already present in the HTML. Inline scripts execute immediately when encountered during parsing, blocking further parsing until complete. This means any inline script, regardless of attributes, behaves as a synchronous parser-blocking script.
Module Scripts (type="module")
Module scripts behave differently from classic scripts by default:
- Without
async: Module scripts defer by default. They fetch in parallel and execute when page parsing is finished, beforeDOMContentLoaded, in document order. - With
async: If theasyncattribute is present on a module script, it executes as soon as available (like async classic scripts), potentially beforeDOMContentLoaded. - The
deferattribute has no effect on module scripts because they defer by default.
All module scripts execute before DOMContentLoaded unless they have the async attribute, in which case they may execute after DOMContentLoaded if they haven't finished downloading yet. Module scripts maintain document order unless async is specified.
{"evidence": [{"claim": "The async attribute tells the browser that a classic script can be fetched in parallel to parsing and evaluated as soon as available.", "quote": "For classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Async scripts can interrupt DOM building if they finish downloading while the parser is still at work.", "quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "Async scripts do not guarantee any specific execution order when there are multiple scripts.", "quote": "This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "The defer attribute tells the browser to execute scripts after the document has been parsed.", "quote": "This Boolean attribute is set to indicate to a browser that the script is meant to be executed after the document has been parsed, but before firing DOMContentLoaded event.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Deferred scripts execute in the order they appear in the document.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "The DOMContentLoaded event fires after deferred scripts and module scripts execute.", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}, {"claim": "DOMContentLoaded does not wait for async scripts.", "quote": "It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}, {"claim": "The async and defer attributes have no effect on inline scripts.", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Module scripts defer by default, with no effect from the defer attribute.", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "When async is specified on a module script, it executes as soon as available like async classic scripts.", "quote": "For module scripts, if the async attribute is present then the scripts and all their dependencies will be fetched in parallel to parsing and evaluated as soon as they are available.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Module scripts and inline scripts with type=module are automatically deferred.", "quote": "async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.", "url": "https://web.dev/learn/performance/optimize-resource-loading"}, {"claim": "Scripts with async may execute out-of-order while defer scripts execute in document order.", "quote": "Additionally, async scripts may execute out-of-order, while defer scripts are executed in the order in which they appear in the markup.", "url": "https://web.dev/learn/performance/optimize-resource-loading"}, {"claim": "Async classic scripts execute as soon as download completes, not in guaranteed order.", "quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event. This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "If both async and defer are specified, async takes precedence.", "quote": "If the attribute is specified with the defer attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.