RunOpus 5.5repeat 1
Gated-execute on async vs defer on <script>
claude-opus-5-5__gated-execute__research-script-loading__r1
Score
1.000
- Cost
- $0.328
- + $0.095 judging
- Latency
- 1m 3s
- setup 0.0s, grading 4.5s
- Turns
- 16
- Tool calls
- 15
- Subagents
- 0
- Tokens in / out
- 116k / 6.3k
- 91,895 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- Every state change was reviewed: heldgated-execute.reviewedEvery call that changed the world had been allowed by a review before it ran, counted from the world’s own journal.
- Every decision was read: heldgated-execute.decisionsEvery review that ran to its end gave its decision through the schema; an unreadable one is a denial.
- Each call was shown the right tools: heldgated-execute.toolsThe executor was shown the task’s full toolset and each reviewer the read-only tools.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Execute | 1m 3s | $0.328 | 16 | 15 | claude-opus-5-5 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies gated-execute --tasks research-script-loading --models claude-opus-5-5 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1, E2. “`async` and `defer` both let the browser download an external classic script without pausing the HTML parser.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E8, E16. “A `defer` script runs after parsing finishes, in document order, and before `DOMContentLoaded`.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E2, E3, E20. “The safe reading is that the order isn't guaranteed either way.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E9, E5, E18, E19. “They run in the order they appear in the document (spec algorithm; MDN).”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E12, E13. “For a script with no `src`, `async` and `defer` aren't allowed and do nothing.”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E14, E15. “"The defer attribute has no effect on module scripts" (spec).”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded","webdev-third-party-javascript"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"mdn-script-element":0.8389,"whatwg-html-scripting":0.2918,"mdn-domcontentloaded":1,"webdev-optimize-resource-loading":0.0154,"webdev-third-party-javascript":0.9618}
- cited count
- 4
- verified count
- 4
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 20
- verified quote count
- 20
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Without async/defer, an external classic script blocks parsing while it is fetched and evaluated; async fetches in parallel and evaluates as soon as available, possibly before parsing completes; defer\n…[truncated 49 chars]","quote":"If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"async classic scripts may execute before parsing completes.","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"async scripts can interrupt DOM building if they finish downloading during parsing.","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"async scripts are guaranteed only to run before the window load event, and may run out of order.","quote":"Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async scripts have no guaranteed execution order.","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded waits for deferred and module scripts but not async scripts.","quote":"It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"An MDN note says DOMContentLoaded fires before async scripts execute, which conflicts with the spec's statement that async scripts may run before parsing completes.","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute after parsing, before DOMContentLoaded, and delay that event.","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer scripts execute in document order.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"If both async and defer are present, the element behaves as async only.","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Specifying defer alongside async is a fallback for legacy browsers that only support defer.","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"async and defer have no effect on inline classic scripts.","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Inline classic scripts and attribute-less scripts execute immediately before parsing continues.","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts are deferred by default; async makes them run as soon as available; defer has no effect on them.","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"With async, a module script and all its dependencies are fetched in parallel and evaluated as soon as available.","quote":"For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded waits for both defer scripts and module scripts to download and execute.","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"Deferred scripts wait for stylesheets, and DOMContentLoaded is queued after them.","quote":"DOMContentLoaded does not wait for stylesheets to load, however deferred scripts do wait for stylesheets, and the DOMContentLoaded event is queued after deferred scripts.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Defer scripts and non-async module scripts share one ordered list that runs when parsing finishes.","quote":"Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Async scripts are held in an unordered set and run as soon as each is ready.","quote":"Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"async scripts may run after DOMContentLoaded has already fired.","quote":"If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.","verified":true,"document":"mdn-domcontentloaded"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E8","E16"]},"async-timing":{"status":"met","support":["E2","E3","E20"]},"ordering":{"status":"met","support":["E9","E5","E18","E19"]},"inline-ignored":{"status":"met","support":["E12","E13"]},"modules":{"status":"met","support":["E14","E15"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E8","E16"]},"async-timing":{"status":"met","support":["E2","E3","E20"]},"ordering":{"status":"met","support":["E9","E5","E18","E19"]},"inline-ignored":{"status":"met","support":["E12"]},"modules":{"status":"met","support":["E14","E15"]}},{"parallel-download":{"status":"met","support":["E1","E2"]},"defer-timing":{"status":"met","support":["E8","E16"]},"async-timing":{"status":"met","support":["E2","E3","E20"]},"ordering":{"status":"met","support":["E9","E5","E18","E19"]},"inline-ignored":{"status":"met","support":["E12","E13"]},"modules":{"status":"met","support":["E14","E15"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
async and defer both let the browser download an external classic script without pausing the HTML parser. They differ in when the script runs. A defer script runs after parsing finishes, in document order, and before DOMContentLoaded. An async script runs as soon as it has downloaded, in no guaranteed order, and its timing relative to DOMContentLoaded is not fixed.
A plain classic script (no attributes). With <script src> and neither attribute, "the script is fetched and evaluated immediately, blocking parsing until these are both complete" (https://html.spec.whatwg.org/multipage/scripting.html).
async (classic, with src)
- Parsing: The script is "fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)" (spec). The download doesn't block the parser, but running the script can. web.dev says async scripts "can interrupt DOM building if they finish downloading while the parser is still at work" (https://web.dev/articles/efficiently-load-third-party-javascript).
DOMContentLoaded: That event "doesn't wait for … async scripts" (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event). So an async script may run before or after the event. The same MDN page lists<script async>as a case where your code may run afterDOMContentLoadedhas already fired. web.dev only promises that async scripts run "before the window's load event."- Conflicting sources: a note on that same MDN page says
DOMContentLoadedfires "before the execution of scripts with async." The spec's "potentially before parsing completes" contradicts that as a general rule. The safe reading is that the order isn't guaranteed either way.
- Conflicting sources: a note on that same MDN page says
- Order: None is guaranteed. In the spec's algorithm, an async script goes into a set of "scripts that will execute as soon as possible," and each one runs whenever its fetch finishes. MDN says: "You get no guarantee that scripts will run in any specific order" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).
defer (classic, with src)
- Parsing: The script is "fetched in parallel and evaluated when the page has finished parsing" (spec), so it never blocks the parser.
DOMContentLoaded: It runs after parsing but before the event, and the event waits for it to load and finish running (MDN script page). MDN's event page adds that deferred scripts wait for stylesheets, and the event is queued after them.- Order: Defer scripts go into an ordered list of scripts that run once the document has finished parsing. They run in the order they appear in the document (spec algorithm; MDN).
- Both attributes at once:
asyncwins. The spec allows writing both so that old browsers that only understanddeferfall back to it instead of blocking.
Inline classic scripts. For a script with no src, async and defer aren't allowed and do nothing. The spec's attribute table marks both as invalid for inline classic scripts. Its algorithm treats them specially only when the script "has a src attribute"; any other inline classic script runs immediately and blocks the parser. MDN says the same.
type="module" scripts
- Default behaviour: A module script with no
asyncacts like a deferred script. It and its imports are fetched in parallel with parsing, and it runs once parsing has finished. It goes into the same ordered list as defer scripts, so it keeps document order and runs beforeDOMContentLoaded(MDN event page). defer: "The defer attribute has no effect on module scripts" (spec).async: The module "and all its dependencies" are fetched in parallel and run "as soon as it is available," with no ordering guarantee.- Inline modules: Unlike inline classic scripts, inline modules accept
async, and the spec's table allows it. Withoutasync, an inline module is also deferred until parsing ends.
Scripts added by JavaScript. Scripts inserted dynamically, rather than by the parser, behave as async by default (the spec's "force async" flag).
{"evidence": [
{"claim": "Without async/defer, an external classic script blocks parsing while it is fetched and evaluated; async fetches in parallel and evaluates as soon as available, possibly before parsing completes; defer fetches in parallel and evaluates after parsing.", "quote": "If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async classic scripts may execute before parsing completes.", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async scripts can interrupt DOM building if they finish downloading during parsing.", "quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "async scripts are guaranteed only to run before the window load event, and may run out of order.", "quote": "Scripts with the async attribute execute at the first opportunity after they finish downloading and before the window's load event.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "async scripts have no guaranteed execution order.", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded waits for deferred and module scripts but not async scripts.", "quote": "It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "An MDN note says DOMContentLoaded fires before async scripts execute, which conflicts with the spec's statement that async scripts may run before parsing completes.", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "defer scripts execute after parsing, before DOMContentLoaded, and delay that event.", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer scripts execute in document order.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "If both async and defer are present, the element behaves as async only.", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Specifying defer alongside async is a fallback for legacy browsers that only support defer.", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async and defer have no effect on inline classic scripts.", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Inline classic scripts and attribute-less scripts execute immediately before parsing continues.", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Module scripts are deferred by default; async makes them run as soon as available; defer has no effect on them.", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "With async, a module script and all its dependencies are fetched in parallel and evaluated as soon as available.", "quote": "For module scripts, if the async attribute is present, then the module script and all its dependencies will be fetched in parallel to parsing, and the module script will be evaluated as soon as it is available (potentially before parsing completes).", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "DOMContentLoaded waits for both defer scripts and module scripts to download and execute.", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "Deferred scripts wait for stylesheets, and DOMContentLoaded is queued after them.", "quote": "DOMContentLoaded does not wait for stylesheets to load, however deferred scripts do wait for stylesheets, and the DOMContentLoaded event is queued after deferred scripts.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "Defer scripts and non-async module scripts share one ordered list that runs when parsing finishes.", "quote": "Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Async scripts are held in an unordered set and run as soon as each is ready.", "quote": "Each Document has a set of scripts that will execute as soon as possible, which is a set of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "async scripts may run after DOMContentLoaded has already fired.", "quote": "If it's included via <script async>, or it's dynamically injected, then by the time it executes, DOMContentLoaded has already fired.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.