RunOpus 5.5repeat 1
Single on async vs defer on <script>
claude-opus-5-5__single__research-script-loading__r1
Score
1.000
- Cost
- $0.331
- + $0.095 judging
- Latency
- 54.5s
- setup 0.0s, grading 4.6s
- Turns
- 14
- Tool calls
- 13
- Subagents
- 0
- Tokens in / out
- 93k / 5.9k
- 66,641 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Main | 54.5s | $0.331 | 14 | 13 | claude-opus-5-5 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies single --tasks research-script-loading --models claude-opus-5-5 --repeats 1
Score breakdown
- passStates that both async and defer let a classic script be fetched in parallel with HTML parsing, so the fetch does not block the parser, unlike a classic script with src and neither attribute.1.00weight 0.13
met, backed by verified evidence E1. “Both let the browser keep parsing the HTML while the file downloads.”
- passStates that a defer script executes only after the document has finished parsing, and before DOMContentLoaded fires.1.00weight 0.19
met, backed by verified evidence E1, E6, E8. “Deferred scripts run after parsing finishes and before the event fires.”
- passStates that an async script executes as soon as it is available, interrupting parsing if parsing is still in progress, so it may run before or after DOMContentLoaded.1.00weight 0.19
met, backed by verified evidence E1, E3. “So an async script can run before or after `DOMContentLoaded`, depending on when it finishes downloading.”
- passStates that multiple defer scripts execute in document order, while async scripts execute in the order their downloads complete, which is not predictable.1.00weight 0.19
met, backed by verified evidence E4, E5, E7, E15. “Deferred scripts go into a *list* of scripts that "will execute when the document has finished parsing", and run in that order”
- passStates that async and defer have no effect on a classic inline script (a script element without a src attribute).1.00weight 0.13
met, backed by verified evidence E13. “MDN warns that adding them "would have no effect".”
- passStates that module scripts are deferred by default, so the defer attribute has no effect on them, while the async attribute does apply to module scripts.1.00weight 0.19
met, backed by verified evidence E2, E16. “"The defer attribute has no effect on module scripts" (spec URL).”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"a5d703ba820ee9c957565ac15b9f98daeac8212372d73d7658a7a6bb2bbc1978","documents":6}
- opened documents
- ["mdn-script-element","whatwg-html-scripting","mdn-domcontentloaded","webdev-third-party-javascript"]
- opened by role
- {"secondary":["mdn-script-element","mdn-domcontentloaded","webdev-third-party-javascript"],"primary":["whatwg-html-scripting"]}
- displayed share
- {"mdn-script-element":1,"whatwg-html-scripting":0.3139,"mdn-domcontentloaded":1,"webdev-third-party-javascript":0.9618,"webdev-optimize-resource-loading":0.0231}
- cited count
- 4
- verified count
- 4
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 16
- verified quote count
- 16
- verified quote rate
- 1
- evidence
- [{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"A classic external script with neither attribute blocks parsing while it is fetched and executed; async fetches in parallel and runs as soon as available, possibly before parsing completes; defer fetc\n…[truncated 51 chars]","quote":"For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes). If the async attribute is not present but the defer attribute is present, then the classic \n…[truncated 219 chars]","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Module scripts are deferred by default, run async if async is present, and ignore defer.","quote":"Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)","verified":true,"document":"whatwg-html-scripting"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts can interrupt DOM construction if they arrive while parsing is still going on.","quote":"It also means they can interrupt DOM building if they finish downloading while the parser is still at work.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://web.dev/articles/efficiently-load-third-party-javascript","claim":"Async scripts run before the window load event and probably not in document order.","quote":"This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.","verified":true,"document":"webdev-third-party-javascript"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Async scripts have no guaranteed execution order.","quote":"You get no guarantee that scripts will run in any specific order.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run after parsing and before DOMContentLoaded, and they delay that event.","quote":"Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Deferred scripts run in document order.","quote":"Scripts with the defer attribute will execute in the order in which they appear in the document.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"DOMContentLoaded waits for defer and module scripts but not for async scripts.","quote":"The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event","claim":"MDN says DOMContentLoaded fires before async scripts run, which conflicts with the spec's statement that async scripts may run before parsing completes.","quote":"DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.","verified":true,"document":"mdn-domcontentloaded"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"When async and defer are both present, the script behaves as async.","quote":"If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Specifying defer together with async lets legacy browsers fall back to defer.","quote":"The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Inline classic scripts, and scripts without async/defer/module, run immediately and block parsing.","quote":"Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.","verified":true,"document":"mdn-script-element"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"defer has no effect on inline scripts.","quote":"Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.","verified":true,"document":"mdn-script-element"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"Even inline module scripts never run synchronously.","quote":"Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://html.spec.whatwg.org/multipage/scripting.html","claim":"The spec keeps a list of scripts (defer and non-async module) that run in order after parsing finishes.","quote":"Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.","verified":true,"document":"whatwg-html-scripting"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script","claim":"Module scripts are deferred by default and ignore defer (MDN).","quote":"The defer attribute has no effect on module scripts — they defer by default.","verified":true,"document":"mdn-script-element"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"parallel-download":{"status":"met","support":["E1"]},"defer-timing":{"status":"met","support":["E1","E6","E8"]},"async-timing":{"status":"met","support":["E1","E3"]},"ordering":{"status":"met","support":["E4","E5","E7","E15"]},"inline-ignored":{"status":"met","support":["E13"]},"modules":{"status":"met","support":["E2","E16"]}},{"parallel-download":{"status":"met","support":["E1"]},"defer-timing":{"status":"met","support":["E1","E6","E8"]},"async-timing":{"status":"met","support":["E1","E3"]},"ordering":{"status":"met","support":["E7","E5","E4","E15"]},"inline-ignored":{"status":"met","support":["E13"]},"modules":{"status":"met","support":["E2","E16"]}},{"parallel-download":{"status":"met","support":["E1"]},"defer-timing":{"status":"met","support":["E1","E6","E8"]},"async-timing":{"status":"met","support":["E1","E3"]},"ordering":{"status":"met","support":["E4","E5","E7","E15"]},"inline-ignored":{"status":"met","support":["E13"]},"modules":{"status":"met","support":["E2","E16"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
async and defer only change behaviour for scripts that the parser has to download. Both let the browser keep parsing the HTML while the file downloads. They differ in when the script runs. A defer script waits until the whole document is parsed, runs in document order, and finishes before DOMContentLoaded. An async script runs as soon as it arrives, in no guaranteed order, and doesn't hold up DOMContentLoaded.
Baseline: a plain <script src> with neither attribute. The HTML spec says such a script "is fetched and evaluated immediately, blocking parsing until these are both complete" (https://html.spec.whatwg.org/multipage/scripting.html).
async (classic script with src)
- Parsing: The file is "fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes)" (https://html.spec.whatwg.org/multipage/scripting.html). Downloading doesn't block parsing, but running the script can. web.dev notes that async scripts "can interrupt DOM building if they finish downloading while the parser is still at work" (https://web.dev/articles/efficiently-load-third-party-javascript).
- Order: None is guaranteed. In the spec's algorithm, async scripts go into a set of scripts that "will execute as soon as possible", and each one runs on its own once it's ready (https://html.spec.whatwg.org/multipage/scripting.html). MDN puts it as "You get no guarantee that scripts will run in any specific order" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).
DOMContentLoaded: It doesn't wait for async scripts (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event). web.dev says they run "before the window's load event" (https://web.dev/articles/efficiently-load-third-party-javascript). So an async script can run before or afterDOMContentLoaded, depending on when it finishes downloading.- Sources disagree on one point. MDN's
DOMContentLoadedpage says the event "is fired after the execution of these scripts, but before the execution of scripts with async" (same MDN URL). The spec says an async script can run "potentially before parsing completes", which means beforeDOMContentLoaded. I'd follow the spec: the MDN sentence describes a typical case, not a guarantee. A script loaded withasyncmay find the event has already fired, so it should checkdocument.readyState, as MDN itself recommends.
defer (classic script with src)
- Parsing: The file is "fetched in parallel and evaluated when the page has finished parsing" (https://html.spec.whatwg.org/multipage/scripting.html), so it never blocks the parser.
- Order: Deferred scripts go into a list of scripts that "will execute when the document has finished parsing", and run in that order (spec URL above). MDN: "Scripts with the defer attribute will execute in the order in which they appear in the document" (https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script).
DOMContentLoaded: Deferred scripts run after parsing finishes and before the event fires. They "will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating" (same MDN URL).document.readyStateis already"interactive"while they run (https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event).- A wording problem in MDN: the summary list on MDN's
<script>page says defer scripts "will only execute once everything has finished loading". That conflicts with MDN's own attribute description and with the spec. It really means "once parsing has finished", not after theloadevent.
Using both attributes together: async wins (MDN). The spec says you can include defer as well so that very old browsers that only understand defer fall back to it instead of blocking (spec URL).
Inline classic scripts (no src)
- The spec's table doesn't allow
asyncordeferon inline classic scripts. MDN warns that adding them "would have no effect". - Inline scripts without
type="module""are fetched and executed immediately before the browser continues to parse the page" (MDN<script>page). One exception: if a stylesheet that blocks scripts is still loading, the inline script waits for it, and the parser waits too.
type="module" scripts
- Default behaviour is like
defer: Withoutasync, a module script and its dependencies are fetched in parallel to parsing and run once parsing has finished. "The defer attribute has no effect on module scripts" (spec URL). - Ordering: Module scripts join the same in-order, after-parsing list as deferred classic scripts, and
DOMContentLoadedwaits for them (MDNDOMContentLoadedpage). - With
async: The module and all its dependencies are fetched in parallel and the module runs "as soon as it is available". - Inline modules work differently from inline classic scripts.
asyncis allowed on them. Without it, an inline module is still put off until parsing finishes, and it never runs synchronously. The spec says this holds "even if the inline module script has no dependencies".
Scripts added by JavaScript: By default, scripts inserted dynamically are treated as async ("force async"). If you set async = false, they go into a list of scripts that "execute in order as soon as possible" (spec URL).
{"evidence": [
{"claim": "A classic external script with neither attribute blocks parsing while it is fetched and executed; async fetches in parallel and runs as soon as available, possibly before parsing completes; defer fetches in parallel and runs when parsing has finished.", "quote": "For external classic scripts, if the async attribute is present, then the classic script will be fetched in parallel to parsing and evaluated as soon as it is available (potentially before parsing completes). If the async attribute is not present but the defer attribute is present, then the classic script will be fetched in parallel and evaluated when the page has finished parsing. If neither attribute is present, then the script is fetched and evaluated immediately, blocking parsing until these are both complete.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Module scripts are deferred by default, run async if async is present, and ignore defer.", "quote": "Otherwise, the module script and its dependencies will be fetched in parallel to parsing and evaluated when the page has finished parsing. (The defer attribute has no effect on module scripts.)", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Async scripts can interrupt DOM construction if they arrive while parsing is still going on.", "quote": "It also means they can interrupt DOM building if they finish downloading while the parser is still at work.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "Async scripts run before the window load event and probably not in document order.", "quote": "This means it's possible (and likely) that async scripts won't run in the order in which they appear in the HTML.", "url": "https://web.dev/articles/efficiently-load-third-party-javascript"},
{"claim": "Async scripts have no guaranteed execution order.", "quote": "You get no guarantee that scripts will run in any specific order.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Deferred scripts run after parsing and before DOMContentLoaded, and they delay that event.", "quote": "Scripts with the defer attribute will prevent the DOMContentLoaded event from firing until the script has loaded and finished evaluating.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Deferred scripts run in document order.", "quote": "Scripts with the defer attribute will execute in the order in which they appear in the document.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "DOMContentLoaded waits for defer and module scripts but not for async scripts.", "quote": "The DOMContentLoaded event fires when the HTML document has been completely parsed, and all deferred scripts (<script defer src=\"…\"> and <script type=\"module\">) have downloaded and executed. It doesn't wait for other things like images, subframes, and async scripts to finish loading.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "MDN says DOMContentLoaded fires before async scripts run, which conflicts with the spec's statement that async scripts may run before parsing completes.", "quote": "DOMContentLoaded is fired after the execution of these scripts, but before the execution of scripts with async.", "url": "https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event"},
{"claim": "When async and defer are both present, the script behaves as async.", "quote": "If the attribute is specified with the async attribute, the element will act as if only the async attribute is specified.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Specifying defer together with async lets legacy browsers fall back to defer.", "quote": "The defer attribute may be specified even if the async attribute is specified, to cause legacy web browsers that only support defer (and not async) to fall back to the defer behavior instead of the blocking behavior that is the default.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Inline classic scripts, and scripts without async/defer/module, run immediately and block parsing.", "quote": "Scripts without async, defer or type=\"module\" attributes, as well as inline scripts without the type=\"module\" attribute, are fetched and executed immediately before the browser continues to parse the page.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "defer has no effect on inline scripts.", "quote": "Warning: This attribute must not be used if the src attribute is absent (i.e., for inline scripts), in this case it would have no effect.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"},
{"claim": "Even inline module scripts never run synchronously.", "quote": "Queueing a task here means that, even if the inline module script has no dependencies or synchronously results in a parse error, we won't proceed to execute the script element synchronously.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "The spec keeps a list of scripts (defer and non-async module) that run in order after parsing finishes.", "quote": "Each Document has a list of scripts that will execute when the document has finished parsing, which is a list of script elements, initially empty.", "url": "https://html.spec.whatwg.org/multipage/scripting.html"},
{"claim": "Module scripts are deferred by default and ignore defer (MDN).", "quote": "The defer attribute has no effect on module scripts — they defer by default.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.