ScaffoldBench

RunOpus 5.5repeat 1

Orchestrator-workers on HTTP 301 vs 308

claude-opus-5-5__orchestrator-workers__research-http-redirects__r1

Score

0.917

wrong answer
design ran as designed
Cost
$0.946
+ $0.106 judging
Latency
2m 5s
setup 0.0s, grading 5.4s
Turns
38
Tool calls
32
Subagents
3
Tokens in / out
266k / 20k
191,214 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • The task was decomposed: heldorchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
  • The fan-out happened: heldorchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
  • The synthesis wrote the answer: heldorchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
  • Each call was shown the right tools: heldorchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.

Phases

Decompose: 0.0s to 15.3s, finished, 0 tool calls, $0.042
Worker 1: rfc9110-text: 15.3s to 51.5s, finished, 9 tool calls, $0.208
Worker 2: spec-history: 15.3s to 1m 18s, finished, 21 tool calls, $0.407
Worker 3: mdn-pages: 15.3s to 38.6s, finished, 2 tool calls, $0.077
Synthesize: 1m 18s to 2m 5s, finished, 0 tool calls, $0.213

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Decompose15.3s$0.04220claude-opus-5-5
Worker 1: rfc9110-text36.2s$0.208109claude-opus-5-5
Worker 2: spec-history1m 3s$0.4072221claude-opus-5-5
Worker 3: mdn-pages23.3s$0.07732claude-opus-5-5
Synthesize47.3s$0.21310claude-opus-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies orchestrator-workers --tasks research-http-redirects --models claude-opus-5-5 --repeats 1

Score breakdown

  • passStates that 301 is "Moved Permanently" and 308 is "Permanent Redirect", and that both indicate the target resource has been assigned a new permanent URI.
    1.00weight 0.17

    met, backed by verified evidence E1, E2. “Both codes mean the same thing: the resource has moved for good. RFC 9110 (HTTP Semantics) defines both with the same sentence: the target resource "has been assigned a new permanent URI”

  • passStates that a client following a 308 must not change the request method: a POST is repeated as a POST, with its body.
    1.00weight 0.25

    met, backed by verified evidence E4, E7, E8. “**308.** The client must keep both the method and the body.”

  • passStates that when following a 301 a user agent MAY change the request method from POST to GET (for historical reasons), which is the behaviour 308 exists to rule out.
    1.00weight 0.25

    met, backed by verified evidence E3, E4, E7. “"For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request"”

  • passStates that both 301 and 308 responses are heuristically cacheable, i.e. a cache may reuse them by default unless the method definition or explicit cache controls say otherwise.
    1.00weight 0.17

    met, backed by verified evidence E12, E13, E14. “**RFC 9110** calls both "heuristically cacheable" unless the method definition or explicit cache controls say otherwise.”

  • partialStates that both codes are currently specified in RFC 9110 (HTTP Semantics), and that 308 was originally defined in RFC 7538, which RFC 9110 obsoletes.
    0.50weight 0.17

    partial, backed by verified evidence E1, E2, E4, E17. “Both codes are defined in RFC 9110, HTTP Semantics: 301 in section 15.4.2 and 308 in section 15.4.9”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"25fbed79c4f7f9e1b850436e0e4872f8e25dd80142f3177231c456f21817b817","documents":8}
opened documents
["rfc9110","rfc7538","mdn-status-301","mdn-status-308","rfc7231","mdn-redirections"]
opened by role
{"primary":["rfc9110"],"superseded":["rfc7538","rfc7231"],"secondary":["mdn-status-301","mdn-status-308","mdn-redirections"]}
displayed share
{"rfc9110":0.0667,"rfc7538":1,"mdn-status-301":1,"mdn-status-308":1,"rfc9111":0.0061,"rfc7231":0.1033,"mdn-redirections":0.6959,"mdn-status-307":0.1087}
cited count
5
verified count
5
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
20
verified quote count
20
verified quote rate
1
evidence
[{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"301 means the target resource has a new permanent URI that future references should use.","quote":"The 301 (Moved Permanently) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 has the same meaning: a new permanent URI for the target resource.","quote":"The 308 (Permanent Redirect) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"For 301, a user agent may change POST to GET for the redirected request.","quote":"For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"307 and 308 were added as method-preserving redirects, while 301 and 302 were adjusted to allow POST to become GET.","quote":"307 (Temporary Redirect) and 308 (Permanent Redirect) [RFC7538] were later added to unambiguously indicate method-preserving redirects, and status codes 301 and 302 have been adjusted to allow a POST request to be redirected as GET.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"When following a redirect, the method changes only if the status code's semantics allow it.","quote":"Change the request method according to the redirecting status code's semantics, if applicable.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"If the method was changed to GET or HEAD, content-specific headers are removed, so the body is dropped.","quote":"If the request method has been changed to GET or HEAD, remove content-specific header fields, including (but not limited to) Content-Encoding, Content-Language, Content-Location, Content-Type, Content-Length, Digest, Last-Modified.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"308 is like 301 but does not allow changing POST to GET.","quote":"Note: This status code is similar to 301 (Moved Permanently)\n ([RFC7231], Section 6.4.2), except that it does not allow changing\n the request method from POST to GET.","verified":true,"document":"rfc7538"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308","claim":"With 308 the client keeps the method and body unchanged.","quote":"The request method and the body will not be modified by the client in the redirected request.","verified":true,"document":"mdn-status-308"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/301","claim":"MDN's 301 page says browsers switch POST to GET on 301, as permitted by the HTTP spec.","quote":"Note: In the Fetch Standard, when a user agent receives a 301 in response to a POST request, it uses the GET method in the subsequent redirection request, as permitted by the HTTP specification.","verified":true,"document":"mdn-status-301"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308","claim":"Conflicting source: MDN's 308 page claims 301 requires the method and body to stay unchanged, contrary to the RFCs and MDN's 301 page.","quote":"A 301 Moved Permanently requires the request method and the body to remain unchanged when redirection is performed, but this is incorrectly handled by older clients to use the GET method instead.","verified":true,"document":"mdn-status-308"},{"url":"https://www.rfc-editor.org/rfc/rfc7231.html","claim":"RFC 7231 made POST-to-GET rewriting for 301/302 conformant because most user agents do it.","quote":"Over 10 years later, most user agents still do method\n rewriting for 301 and 302; therefore, this specification makes\n that behavior conformant when the original request is POST.","verified":true,"document":"rfc7231"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"Under RFC 9110, 301 is heuristically cacheable by default.","quote":"A 301 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"Under RFC 9110, 308 is heuristically cacheable by default.","quote":"A 308 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"RFC 9110 section 15.1 lists 301 and 308 among the heuristically cacheable codes.","quote":"Responses with status codes that are defined as heuristically cacheable (e.g., 200, 203, 204, 206, 300, 301, 308, 404, 405, 410, 414, and 501 in this specification) can be reused by a cache with heuristic expiration unless otherwise indicated by the method definition or explicit cache controls [CACH\n…[truncated 61 chars]","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7231.html","claim":"The earlier spec RFC 7231 described 301 as cacheable by default.","quote":"A 301 response is cacheable by default; i.e., unless otherwise\n indicated by the method definition or explicit cache controls (see\n Section 4.2.2 of [RFC7234]).","verified":true,"document":"rfc7231"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"RFC 7538 described 308 as cacheable by default.","quote":"A 308 response is cacheable by default; i.e., unless otherwise\n indicated by the method definition or explicit cache controls (see\n [RFC7234], Section 4.2.2).","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"RFC 7538 re-published 308 from the Experimental RFC 7238 with no technical changes.","quote":"This specification contains no technical changes from the\n Experimental RFC 7238, which it obsoletes.","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc7231.html","claim":"RFC 7231 defined no permanent method-preserving redirect itself and pointed to RFC 7238 for 308.","quote":"This\n specification defines no equivalent counterpart for 301 (Moved\n Permanently) ([RFC7238], however, defines the status code 308\n (Permanent Redirect) for this purpose).","verified":true,"document":"rfc7231"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 is newer and may not be recognized everywhere.","quote":"This status code is much younger (June 2014) than its sibling codes and thus might not be recognized everywhere.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"Servers cannot rely on automatic redirection by clients that don't recognize 308.","quote":"Thus, servers will not be able to rely\n on automatic redirection happening similar to status codes 301, 302,\n or 307.","verified":true,"document":"rfc7538"}]
judge unverified quotes
0
judge verdicts
[{"names":{"status":"met","support":["E1","E2"]},"method-preserved-308":{"status":"met","support":["E4","E7","E8"]},"method-change-301":{"status":"met","support":["E3","E4","E7"]},"cacheable":{"status":"met","support":["E12","E13","E14"]},"specifications":{"status":"partial","support":["E1","E2","E4","E17"]}},{"names":{"status":"met","support":["E1","E2"]},"method-preserved-308":{"status":"met","support":["E7","E8","E4"]},"method-change-301":{"status":"met","support":["E3","E4"]},"cacheable":{"status":"met","support":["E12","E13","E14"]},"specifications":{"status":"partial","support":["E4","E17"]}},{"names":{"status":"met","support":["E1","E2"]},"method-preserved-308":{"status":"met","support":["E4","E7","E8"]},"method-change-301":{"status":"met","support":["E3","E4"]},"cacheable":{"status":"met","support":["E12","E13","E14"]},"specifications":{"status":"partial","support":["E1","E2","E17"]}}]
judge failures
0
grounded weight share
1

Final answer

HTTP 301 vs 308: both permanent, but only 308 keeps the method

What each code means

Both codes mean the same thing: the resource has moved for good. RFC 9110 (HTTP Semantics) defines both with the same sentence: the target resource "has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs" (https://www.rfc-editor.org/rfc/rfc9110.html). MDN describes both as a permanent move to the URL in the Location header (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/301, https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308). The only real difference is how the client must re-send the request.

How the client treats the method and body

General rule. RFC 9110 says that when following a redirect, the user agent re-sends the original request. It changes the method only "according to the redirecting status code's semantics, if applicable." If the method has been changed to GET or HEAD, it removes content-specific header fields such as Content-Type and Content-Length. In practice that means the body is dropped (https://www.rfc-editor.org/rfc/rfc9110.html).

301. "For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request" (https://www.rfc-editor.org/rfc/rfc9110.html).

  • RFC 9110 explains the history. HTTP/1.0 meant 301 and 302 to keep the method, but browsers converged on switching to GET.
  • So 301 and 302 "have been adjusted to allow a POST request to be redirected as GET."
  • RFC 7231, the previous spec, said the same. It made this rewriting conformant because most user agents did it (https://www.rfc-editor.org/rfc/rfc7231.html).
  • MDN's 301 page says the Fetch Standard uses GET after a 301 response to a POST, "as permitted by the HTTP specification" (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/301).
  • Result: a POST that gets a 301 may be re-sent as a GET with no body.

308. The client must keep both the method and the body.

A source conflict. MDN's 308 page says that "A 301 Moved Permanently requires the request method and the body to remain unchanged," and calls switching to GET something "incorrectly handled by older clients" (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308). This contradicts:

  • RFC 9110 and RFC 7231, which both explicitly allow POST to become GET on a 301.
  • MDN's own 301 page.

The RFC text is the normative spec, so the MDN 308 page's statement about 301 should be treated as inaccurate.

Compatibility note. 308 is "much younger (June 2014) than its sibling codes and thus might not be recognized everywhere" (https://www.rfc-editor.org/rfc/rfc9110.html). RFC 7538 warns that servers cannot rely on automatic redirection by clients that don't recognize 308 (https://www.rfc-editor.org/rfc/rfc7538.html).

Cacheability

Both codes are cacheable by default. The wording has changed between spec versions:

Where each is specified

Today. Both codes are defined in RFC 9110, HTTP Semantics: 301 in section 15.4.2 and 308 in section 15.4.9 (https://www.rfc-editor.org/rfc/rfc9110.html). MDN also cites "HTTP Semantics" for both.

History.

Not verified.

  • Neither of the rfc-editor pages that were opened (RFC 7231 and RFC 7538) showed an "Obsoleted by RFC 9110" line. So I can't confirm from a retrieved source that they are formally obsoleted.
  • The IANA HTTP Status Code Registry returned 404 at every URL tried, so its current references for 301 and 308 were not checked.
{"evidence": [
{"claim": "301 means the target resource has a new permanent URI that future references should use.", "quote": "The 301 (Moved Permanently) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 has the same meaning: a new permanent URI for the target resource.", "quote": "The 308 (Permanent Redirect) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "For 301, a user agent may change POST to GET for the redirected request.", "quote": "For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "307 and 308 were added as method-preserving redirects, while 301 and 302 were adjusted to allow POST to become GET.", "quote": "307 (Temporary Redirect) and 308 (Permanent Redirect) [RFC7538] were later added to unambiguously indicate method-preserving redirects, and status codes 301 and 302 have been adjusted to allow a POST request to be redirected as GET.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "When following a redirect, the method changes only if the status code's semantics allow it.", "quote": "Change the request method according to the redirecting status code's semantics, if applicable.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "If the method was changed to GET or HEAD, content-specific headers are removed, so the body is dropped.", "quote": "If the request method has been changed to GET or HEAD, remove content-specific header fields, including (but not limited to) Content-Encoding, Content-Language, Content-Location, Content-Type, Content-Length, Digest, Last-Modified.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 is like 301 but does not allow changing POST to GET.", "quote": "Note: This status code is similar to 301 (Moved Permanently)\n      ([RFC7231], Section 6.4.2), except that it does not allow changing\n      the request method from POST to GET.", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "With 308 the client keeps the method and body unchanged.", "quote": "The request method and the body will not be modified by the client in the redirected request.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308"},
{"claim": "MDN's 301 page says browsers switch POST to GET on 301, as permitted by the HTTP spec.", "quote": "Note: In the Fetch Standard, when a user agent receives a 301 in response to a POST request, it uses the GET method in the subsequent redirection request, as permitted by the HTTP specification.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/301"},
{"claim": "Conflicting source: MDN's 308 page claims 301 requires the method and body to stay unchanged, contrary to the RFCs and MDN's 301 page.", "quote": "A 301 Moved Permanently requires the request method and the body to remain unchanged when redirection is performed, but this is incorrectly handled by older clients to use the GET method instead.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308"},
{"claim": "RFC 7231 made POST-to-GET rewriting for 301/302 conformant because most user agents do it.", "quote": "Over 10 years later, most user agents still do method\n      rewriting for 301 and 302; therefore, this specification makes\n      that behavior conformant when the original request is POST.", "url": "https://www.rfc-editor.org/rfc/rfc7231.html"},
{"claim": "Under RFC 9110, 301 is heuristically cacheable by default.", "quote": "A 301 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "Under RFC 9110, 308 is heuristically cacheable by default.", "quote": "A 308 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "RFC 9110 section 15.1 lists 301 and 308 among the heuristically cacheable codes.", "quote": "Responses with status codes that are defined as heuristically cacheable (e.g., 200, 203, 204, 206, 300, 301, 308, 404, 405, 410, 414, and 501 in this specification) can be reused by a cache with heuristic expiration unless otherwise indicated by the method definition or explicit cache controls [CACHING]; all other status codes are not heuristically cacheable.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "The earlier spec RFC 7231 described 301 as cacheable by default.", "quote": "A 301 response is cacheable by default; i.e., unless otherwise\n   indicated by the method definition or explicit cache controls (see\n   Section 4.2.2 of [RFC7234]).", "url": "https://www.rfc-editor.org/rfc/rfc7231.html"},
{"claim": "RFC 7538 described 308 as cacheable by default.", "quote": "A 308 response is cacheable by default; i.e., unless otherwise\n   indicated by the method definition or explicit cache controls (see\n   [RFC7234], Section 4.2.2).", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "RFC 7538 re-published 308 from the Experimental RFC 7238 with no technical changes.", "quote": "This specification contains no technical changes from the\n   Experimental RFC 7238, which it obsoletes.", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "RFC 7231 defined no permanent method-preserving redirect itself and pointed to RFC 7238 for 308.", "quote": "This\n      specification defines no equivalent counterpart for 301 (Moved\n      Permanently) ([RFC7238], however, defines the status code 308\n      (Permanent Redirect) for this purpose).", "url": "https://www.rfc-editor.org/rfc/rfc7231.html"},
{"claim": "308 is newer and may not be recognized everywhere.", "quote": "This status code is much younger (June 2014) than its sibling codes and thus might not be recognized everywhere.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "Servers cannot rely on automatic redirection by clients that don't recognize 308.", "quote": "Thus, servers will not be able to rely\n   on automatic redirection happening similar to status codes 301, 302,\n   or 307.", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.