RunHaiku 4.5repeat 1
Propose-select on async vs defer on <script>
claude-haiku-4-5-20251001__propose-select__research-script-loading__r1
Score
1.000
- Cost
- $0.375
- + $0.093 judging
- Latency
- 2m 35s
- setup 0.0s, grading 4.9s
- Turns
- 58
- Tool calls
- 53
- Subagents
- 3
- Tokens in / out
- 627k / 25k
- 516,279 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- All three proposers ran: heldpropose-select.proposalsThree proposers ran in parallel, each from its own angle, and none was starved while another ran.
- The selector chose: heldpropose-select.selectionThe selector picked a proposal by a valid number, read from its schema reply and not a default.
- Each call was shown the right tools: heldpropose-select.toolsEach proposer was shown the read-only tools, the selector none, and the executor the full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Proposal 1 (policy-first) | 1m 19s | $0.116 | 18 | 17 | claude-haiku-4-5-20251001 |
| Proposal 2 (state-first) | 49.6s | $0.084 | 18 | 17 | claude-haiku-4-5-20251001 |
| Proposal 3 (outcome-first) | 54.8s | $0.108 | 20 | 19 | claude-haiku-4-5-20251001 |
| Select | 1m 16s | $0.068 | 2 | 0 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies propose-select --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E18. “Neither attribute: Fetched and executed immediately, blocking the parser (parser-blocking resource).”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E4, E11. “Execute after HTML parsing completes but **before** the DOMContentLoaded event fires.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E3, E11, E12. “Execute at the first opportunity after download, which can be before or after DOMContentLoaded, depending on download timing.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E5, E6. “Multiple deferred scripts execute in the **order they appear in the document**, even if some download faster than others.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E8, E9. “The `defer` attribute **must not be used** on inline classic scripts—it has no effect.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E10, E15. “`defer`: Explicitly specified but has no additional effect (they already defer by default).”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","whatwg-html-scripting","webdev-third-party-javascript","mdn-domcontentloaded","webdev-optimize-resource-loading","mdn-js-modules"]
- opened by role
- {"secondary":["mdn-script-element","webdev-third-party-javascript","mdn-domcontentloaded","webdev-optimize-resource-loading","mdn-js-modules"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"webdev-third-party-javascript":0.9618,"whatwg-html-scripting":0.3402,"mdn-script-element":1,"webdev-optimize-resource-loading":0.9624,"mdn-domcontentloaded":1,"mdn-js-modules":0.3609}
- cited count
- 5
- verified count
- 5
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 18
- verified quote count
- 16
- verified quote rate
- 0.8888888888888888 (89%)
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"For classic scripts, async causes the script to be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)","quote":"For classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":false,"document":"whatwg-html-scripting","failure":"the quote does not occur in the page"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can still block HTML parsing because they block during execution","quote":"Scripts with async can still block HTML parsing.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can interrupt DOM building if they finish downloading while the parser is still working","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Defer scripts execute after HTML parsing is completely finished, but before the DOMContentLoaded event","quote":"Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Defer ensures that scripts run in the order they appear in the HTML","quote":"defer ensures that scripts run in the order they appear in the HTML and don't block the parser.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Scripts with defer attribute execute in the order in which they appear in the document","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts do not guarantee execution order and likely won't run in the order they appear","quote":"It is possible (and likely) that async scripts won't run in the order in which they appear in the HTML.","verified":false,"document":"webdev-third-party-javascript","failure":"the quote does not occur in the page"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async attribute has no effect on inline classic scripts","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Defer attribute has no effect on inline classic scripts","quote":"This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Module scripts defer by default","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded fires after execution of defer and module scripts but before async scripts","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded doesn't wait for async scripts to finish","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts execute as soon as download is complete before the window load event","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"Inline scripts with type=module are deferred automatically","quote":"async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.","verified":true,"document":"webdev-optimize-resource-loading"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"For module scripts, if async is present, the module and dependencies are fetched in parallel and evaluated as soon as available","quote":"For module scripts, if the async attribute is present then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"When async attribute is specified with defer, the element acts as if only async is specified","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Inline classic scripts without async or defer are executed immediately before the browser continues parsing","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"},{"url":"https://web.dev/learn/performance/optimize-resource-loading","claim":"A parser-blocking resource interrupts the HTML parser, such as a script element without async or defer attributes","quote":"A parser-blocking resource interrupts the HTML parser, such as a <script> element without async or defer attributes.","verified":true,"document":"webdev-optimize-resource-loading"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E18"]},"defer-timing":{"status":"met","support":["E4","E11"]},"async-timing":{"status":"met","support":["E1","E3","E11","E12"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E8","E9"]},"modules":{"status":"met","support":["E10","E15"]}},{"parallel-download":{"status":"met","support":["E1","E18"]},"defer-timing":{"status":"met","support":["E4","E12"]},"async-timing":{"status":"met","support":["E1","E3","E11","E12","E13"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E8","E9"]},"modules":{"status":"met","support":["E10","E15"]}},{"parallel-download":{"status":"met","support":["E1"]},"defer-timing":{"status":"met","support":["E4","E12"]},"async-timing":{"status":"met","support":["E1","E3","E11","E12","E13"]},"ordering":{"status":"met","support":["E5","E6","E7"]},"inline-ignored":{"status":"met","support":["E8","E9"]},"modules":{"status":"met","support":["E10","E14","E15"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
In HTML, the async and defer attributes on <script> elements control when external scripts are fetched and executed relative to HTML parsing and the DOMContentLoaded event. Each has fundamentally different behavior regarding parsing, execution timing, and order guarantees.
How async Interacts with HTML Parsing
When the async attribute is present on an external classic script, the script is fetched in parallel to parsing without blocking the HTML parser from continuing. However, once the download completes, the script immediately begins execution, which blocks the parser and rendering until execution finishes. This means async scripts can still interrupt DOM construction if they finish downloading while parsing is still in progress. The timing of execution depends entirely on download speed—whichever script finishes first will execute first, regardless of source order.
For module scripts with async, the behavior is similar: the module and all its dependencies are fetched in parallel, and the module executes as soon as it becomes available (potentially before parsing completes).
How defer Interacts with HTML Parsing
Scripts with the defer attribute are fetched in parallel to parsing but do not interrupt parsing. The parser continues unblocked. The script only executes after the HTML parsing has completely finished. This non-blocking behavior allows the page content to display much sooner. Note that defer only works on external scripts; it has no effect on inline scripts.
Module scripts without an explicit async attribute defer by default—they automatically exhibit deferred behavior without needing the attribute specified.
Execution Relative to DOMContentLoaded
The timing relative to the DOMContentLoaded event is critical:
- defer and module scripts (type="module"): Execute after HTML parsing completes but before the DOMContentLoaded event fires. The event is queued to fire only after these deferred scripts have downloaded and executed.
- async: Execute at the first opportunity after download, which can be before or after DOMContentLoaded, depending on download timing. Async scripts do not block DOMContentLoaded—the event fires independently of async script completion.
Execution Order Guarantees with Multiple Scripts
- async: No guarantees about execution order. Multiple async scripts run in no predictable order—they execute as soon as each completes downloading. If script B downloads faster than script A (even though A appears first), B will execute first.
- defer: Strict order is guaranteed. Multiple deferred scripts execute in the order they appear in the document, even if some download faster than others. This makes defer suitable when scripts have dependencies.
- Module scripts: Without
async, module scripts maintain order likedefer. Withasync, module scripts execute as soon as available.
Inline Scripts vs. External Scripts
Inline classic scripts (no src attribute):
- The
asyncattribute must not be used on inline classic scripts—it has no effect and is invalid HTML. - The
deferattribute must not be used on inline classic scripts—it has no effect. - Inline classic scripts are fetched and executed immediately during parsing, blocking the parser.
Inline module scripts (no src attribute, with type="module"):
- These are deferred by default; they do not block parsing.
- The
asyncattribute can be used on inline module scripts and has a meaningful effect: the module executes as soon as the code is ready (not necessarily deferred to after parsing). - Inline module scripts still wait for all their imports to load and evaluate before executing.
External classic scripts with src:
async: Fetched in parallel, executed immediately upon download completion.defer: Fetched in parallel, executed after parsing completes, in document order.- Neither attribute: Fetched and executed immediately, blocking the parser (parser-blocking resource).
External module scripts with src and type="module":
- Default behavior is defer-like—they fetch in parallel and execute after parsing, in order.
async: Changes behavior to execute as soon as available.defer: Explicitly specified but has no additional effect (they already defer by default).- Module scripts require CORS for cross-origin fetches.
Special Cases and Timing Details
When both async and defer are specified on the same script element, the async attribute takes precedence, and the element acts as if only async is set. The defer attribute is retained primarily for backwards compatibility with older browsers that support defer but not async.
If the async attribute is dynamically added to a script element via JavaScript, it sets the internal "force async" flag, which affects its execution mode.
The document's readyState property reflects these timing relationships: it transitions to "interactive" after HTML parsing completes but before deferred and module scripts execute; DOMContentLoaded fires after these scripts complete; and readyState becomes "complete" after async scripts finish.
{"evidence": [{"claim": "For classic scripts, async causes the script to be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)", "quote": "For classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}, {"claim": "Async scripts can still block HTML parsing because they block during execution", "quote": "Scripts with async can still block HTML parsing.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "Async scripts can interrupt DOM building if they finish downloading while the parser is still working", "quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "Defer scripts execute after HTML parsing is completely finished, but before the DOMContentLoaded event", "quote": "Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "Defer ensures that scripts run in the order they appear in the HTML", "quote": "defer ensures that scripts run in the order they appear in the HTML and don't block the parser.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "Scripts with defer attribute execute in the order in which they appear in the document", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Async scripts do not guarantee execution order and likely won't run in the order they appear", "quote": "It is possible (and likely) that async scripts won't run in the order in which they appear in the HTML.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "Async attribute has no effect on inline classic scripts", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Defer attribute has no effect on inline classic scripts", "quote": "This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Module scripts defer by default", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "DOMContentLoaded fires after execution of defer and module scripts but before async scripts", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}, {"claim": "DOMContentLoaded doesn't wait for async scripts to finish", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}, {"claim": "Async scripts execute as soon as download is complete before the window load event", "quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"}, {"claim": "Inline scripts with type=module are deferred automatically", "quote": "async and defer allow external scripts to load without blocking the HTML parser while scripts (including inline scripts) with type=\"module\" are deferred automatically.", "url": "https://web.dev/learn/performance/optimize-resource-loading"}, {"claim": "For module scripts, if async is present, the module and dependencies are fetched in parallel and evaluated as soon as available", "quote": "For module scripts, if the async attribute is present then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"}, {"claim": "When async attribute is specified with defer, the element acts as if only async is specified", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "Inline classic scripts without async or defer are executed immediately before the browser continues parsing", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}, {"claim": "A parser-blocking resource interrupts the HTML parser, such as a script element without async or defer attributes", "quote": "A parser-blocking resource interrupts the HTML parser, such as a <script> element without async or defer attributes.", "url": "https://web.dev/learn/performance/optimize-resource-loading"}]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.