RunHaiku 4.5repeat 2
Gated-execute on async vs defer on <script>
claude-haiku-4-5-20251001__gated-execute__research-script-loading__r2
Score
0.906
- Cost
- $0.066
- + $0.081 judging
- Latency
- 35.2s
- setup 0.0s, grading 5.4s
- Turns
- 12
- Tool calls
- 11
- Subagents
- 0
- Tokens in / out
- 76k / 3.6k
- 51,095 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- Every state change was reviewed: heldgated-execute.reviewedEvery call that changed the world had been allowed by a review before it ran, counted from the world’s own journal.
- Every decision was read: heldgated-execute.decisionsEvery review that ran to its end gave its decision through the schema; an unreadable one is a denial.
- Each call was shown the right tools: heldgated-execute.toolsThe executor was shown the task’s full toolset and each reviewer the read-only tools.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Execute | 35.2s | $0.066 | 12 | 11 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies gated-execute --tasks research-script-loading --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E4. “the `async` attribute tells the browser to fetch the script in parallel with HTML parsing”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E4, E5, E7, E11. “The DOMContentLoaded event fires after all deferred scripts complete execution.”
- partialStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.0.50weight 0.19
partial, backed by verified evidence E1, E2. “execution happens immediately upon completion, potentially interrupting DOM construction”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E3, E6. “Unlike async, deferred scripts maintain execution order.”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E8. “The `async` and `defer` attributes have **no effect** on inline scripts”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E9, E10. “The `defer` attribute is ignored on module scripts”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","webdev-third-party-javascript","mdn-domcontentloaded","whatwg-html-scripting"]
- opened by role
- {"secondary":["mdn-script-element","webdev-third-party-javascript","mdn-domcontentloaded"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"whatwg-html-scripting":0.2175,"webdev-third-party-javascript":0.9618,"webdev-optimize-resource-loading":0.0439,"mdn-script-element":0.8439,"mdn-domcontentloaded":1}
- cited count
- 4
- verified count
- 4
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 11
- verified quote count
- 11
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async scripts are fetched in parallel to parsing and evaluated as soon as available","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"async scripts can interrupt DOM building and block HTML parsing","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work. Scripts with async can still block HTML parsing.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"async scripts do not maintain execution order","quote":"This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"defer scripts are fetched in parallel and evaluated after parsing completes","quote":"If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"defer scripts execute before DOMContentLoaded","quote":"Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts maintain execution order","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts prevent DOMContentLoaded from firing until completion","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async and defer have no effect on inline scripts","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"module scripts defer by default","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies \n…[truncated 143 chars]","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer attribute has no effect on module scripts","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded fires after deferred and module scripts execute but not after async","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E5","E7","E11"]},"async-timing":{"status":"partial","support":["E1","E2"]},"ordering":{"status":"met","support":["E3","E6"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E5","E7"]},"async-timing":{"status":"partial","support":["E1","E2"]},"ordering":{"status":"met","support":["E6","E3"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10"]}},{"parallel-download":{"status":"met","support":["E1","E4"]},"defer-timing":{"status":"met","support":["E4","E5","E7"]},"async-timing":{"status":"partial","support":["E1","E2","E11"]},"ordering":{"status":"met","support":["E3","E6"]},"inline-ignored":{"status":"met","support":["E8"]},"modules":{"status":"met","support":["E9","E10"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
Now I have comprehensive information. Let me write a complete answer based on my research.
Understanding HTML's async and defer Script Attributes
The async and defer attributes on HTML <script> elements fundamentally change when and how scripts are fetched and executed, offering crucial control over page loading performance.
How async Works
For external classic scripts (those with a src attribute), the async attribute tells the browser to fetch the script in parallel with HTML parsing. According to the WHATWG specification, "if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)." This means the script may finish downloading while the HTML parser is still working, and execution happens immediately upon completion, potentially interrupting DOM construction. As web.dev notes, "Scripts with async can still block HTML parsing."
A critical characteristic of async scripts is that they do not maintain execution order. When multiple async scripts are present, "It is possible (and likely) that async scripts won't run in the order in which they appear in the HTML."
Regarding the DOMContentLoaded event, async scripts execute before it fires, but the event does not wait for them. According to MDN, "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async," meaning DOMContentLoaded fires once parsing completes and deferred/module scripts finish, regardless of async script status.
How defer Works
The defer attribute specifies that a script should execute after the document is fully parsed. The WHATWG specification states: "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing."
Unlike async, deferred scripts maintain execution order. MDN confirms: "Scripts with the defer attribute will execute in the order in which they appear in the document." This is essential when scripts depend on each other or on the DOM being complete.
Importantly, deferred scripts block the DOMContentLoaded event. MDN states that "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating." The DOMContentLoaded event fires after all deferred scripts complete execution.
Inline Scripts vs. External Scripts
The async and defer attributes have no effect on inline scripts (scripts without a src attribute). MDN explicitly warns: "This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect." The same restriction applies to defer. Inline scripts execute immediately and block HTML parsing.
Module Scripts
Module scripts (using type="module") behave differently. They defer by default, meaning they wait for parsing to complete before executing. The WHATWG specification explains: "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)"
This means:
- Without
async, module scripts automatically defer (no need to specifydefer) - With
async, module scripts execute as soon as available, like classic async scripts - The
deferattribute is ignored on module scripts
MDN confirms: "The defer attribute has no effect on module scripts — they defer by default." Inline module scripts also defer automatically.
Module scripts participate in DOMContentLoaded: "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed."
Summary Table
| Script Type | Parsing | Execution Timing | Execution Order | DOMContentLoaded |
|---|---|---|---|---|
| Classic (no attributes) | Blocks | Immediate | N/A | Waits for completion |
| Classic async | Parallel | ASAP after download | Not guaranteed | Fires before async completes |
| Classic defer | Parallel | After parsing | Guaranteed (document order) | Waits for completion |
| Inline (any type) | Blocks | Immediate | N/A | Waits for completion |
| Module (no async) | Parallel | After parsing (deferred by default) | Guaranteed | Waits for completion |
| Module async | Parallel | ASAP after download | Not guaranteed | Fires before async completes |
{"evidence": [
{"claim": "async scripts are fetched in parallel to parsing and evaluated as soon as available", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async scripts can interrupt DOM building and block HTML parsing", "quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work. Scripts with async can still block HTML parsing.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "async scripts do not maintain execution order", "quote": "This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "defer scripts are fetched in parallel and evaluated after parsing completes", "quote": "If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer scripts execute before DOMContentLoaded", "quote": "Scripts with the defer attribute execute after HTML parsing is completely finished, but before the DOMContentLoaded event.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "defer scripts maintain execution order", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer scripts prevent DOMContentLoaded from firing until completion", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "async and defer have no effect on inline scripts", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts) for classic scripts, in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "module scripts defer by default", "quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes). Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "defer attribute has no effect on module scripts", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded fires after deferred and module scripts execute but not after async", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.