RunSonnet 5.5repeat 2
Orchestrator-workers on HTTP 301 vs 308
claude-sonnet-5-5__orchestrator-workers__research-http-redirects__r2
Score
1.000
- Cost
- $0.249
- + $0.084 judging
- Latency
- 55.1s
- setup 0.0s, grading 4.6s
- Turns
- 19
- Tool calls
- 14
- Subagents
- 2
- Tokens in / out
- 90k / 11k
- 50,568 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- The task was decomposed: heldorchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
- The fan-out happened: heldorchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
- The synthesis wrote the answer: heldorchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
- Each call was shown the right tools: heldorchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Decompose | 6.0s | $0.016 | 2 | 0 | claude-sonnet-5-5 |
| Worker 1: status-301 | 20.3s | $0.058 | 6 | 5 | claude-sonnet-5-5 |
| Worker 2: status-308 | 27.2s | $0.104 | 10 | 9 | claude-sonnet-5-5 |
| Synthesize | 21.9s | $0.071 | 1 | 0 | claude-sonnet-5-5 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies orchestrator-workers --tasks research-http-redirects --models claude-sonnet-5-5 --repeats 1
Score breakdown
- passStates that 301 is "Moved Permanently" and 308 is "Permanent Redirect", and that both indicate the target resource has been assigned a new permanent URI.1.00weight 0.17
met, backed by verified evidence E1, E7. “301 (Moved Permanently) and 308 (Permanent Redirect) both say the resource has permanently moved to a new URI.”
- passStates that a client following a 308 must not change the request method: a POST is repeated as a POST, with its body.1.00weight 0.25
met, backed by verified evidence E8, E9, E11. “With 308, the method and body must not change.”
- passStates that when following a 301 a user agent MAY change the request method from POST to GET (for historical reasons), which is the behaviour 308 exists to rule out.1.00weight 0.25
met, backed by verified evidence E3, E4. “For historical reasons, a user agent may change the request method from POST to GET on the follow-up request. RFC 9110 says to use 308 if that is undesired.”
- passStates that both 301 and 308 responses are heuristically cacheable, i.e. a cache may reuse them by default unless the method definition or explicit cache controls say otherwise.1.00weight 0.17
met, backed by verified evidence E5, E6. “RFC 9110 says a 308 response is heuristically cacheable, the same as 301.”
- passStates that both codes are currently specified in RFC 9110 (HTTP Semantics), and that 308 was originally defined in RFC 7538, which RFC 9110 obsoletes.1.00weight 0.17
met, backed by verified evidence E14, E13, E16. “It was originally defined in RFC 7538 (April 2015), which replaced the experimental RFC 7238 with no technical changes. RFC 9110 lists RFC 7538 as obsoleted.”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"25fbed79c4f7f9e1b850436e0e4872f8e25dd80142f3177231c456f21817b817","documents":8}
- opened documents
- ["rfc9110","rfc7538","mdn-status-301","mdn-status-308","rfc7231"]
- opened by role
- {"primary":["rfc9110"],"superseded":["rfc7538","rfc7231"],"secondary":["mdn-status-301","mdn-status-308"]}
- displayed share
- {"rfc9110":0.0303,"rfc7538":1,"mdn-status-301":1,"mdn-status-308":1,"rfc7231":0.0009}
- cited count
- 5
- verified count
- 5
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 18
- verified quote count
- 17
- verified quote rate
- 0.9444444444444444 (94%)
- evidence
- [{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"301 means the target resource has a new permanent URI and future references should use one of the enclosed URIs","quote":"The 301 (Moved Permanently) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"The user agent may use the Location header value for automatic redirection","quote":"The user agent MAY use the Location field value for automatic redirection.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"For a 301, a user agent may change POST to GET for historical reasons","quote":"For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"RFC 9110 points to 308 when the method change is undesired","quote":"If this behavior is undesired, the 308 (Permanent Redirect) status code can be used instead.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"301 is heuristically cacheable","quote":"A 301 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 is heuristically cacheable under RFC 9110","quote":"A 308 response is heuristically cacheable; i.e., unless otherwise","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 means a new permanent URI, per the current specification","quote":"The 308 (Permanent Redirect) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"308 is like 301 except that the method may not be changed from POST to GET","quote":"Note: This status code is similar to 301 (Moved Permanently) ([RFC7231], Section 6.4.2), except that it does not allow changing the request method from POST to GET.","verified":true,"document":"rfc7538"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/308","claim":"With 308 the client does not modify the method or body","quote":"The request method and the body will not be modified by the client in the redirected request.","verified":true,"document":"mdn-status-308"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/308","claim":"MDN's description of 301 differs from RFC 9110: it says the method should stay unchanged but older clients switch to GET","quote":"A 301 Moved Permanently requires the request method and the body to remain unchanged when redirection is performed, but this is incorrectly handled by older clients to use the GET method instead.","verified":true,"document":"mdn-status-308"},{"url":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/301","claim":"MDN says altering the method after a 308 is prohibited, so 308 avoids request modification","quote":"To avoid user agents modifying the request, use 308 Permanent Redirect instead, as altering the method after a 308 response is prohibited.","verified":true,"document":"mdn-status-301"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"RFC 7538 originally said 308 was cacheable by default","quote":"A 308 response is cacheable by default; i.e., unless otherwise","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"RFC 7538 had no technical changes from the experimental RFC it replaced","quote":"This specification contains no technical changes from the","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"RFC 9110 obsoletes RFC 7231 and RFC 7538 among others","quote":"This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7231.html","claim":"RFC 7231 obsoleted RFC 2616","quote":"Obsoletes: 2616","verified":false,"document":"rfc7231","failure":"the quote has fewer than 6 consecutive words"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"RFC 9110 records no changes from RFC 7538","quote":"B.7. Changes from RFC 7538 None.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 is newer than its sibling codes and may not be recognized everywhere","quote":"This status code is much younger (June 2014) than its","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"RFC 7538 restricts 308 use because clients may not understand it","quote":"Therefore, the use of status code 308 is restricted to cases where","verified":true,"document":"rfc7538"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"names":{"status":"met","support":["E1","E7"]},"method-preserved-308":{"status":"met","support":["E8","E9","E11"]},"method-change-301":{"status":"met","support":["E3","E4"]},"cacheable":{"status":"met","support":["E5","E6"]},"specifications":{"status":"met","support":["E14","E13","E16"]}},{"names":{"status":"met","support":["E1","E7"]},"method-preserved-308":{"status":"met","support":["E8","E9","E11"]},"method-change-301":{"status":"met","support":["E3","E4"]},"cacheable":{"status":"met","support":["E5","E6"]},"specifications":{"status":"met","support":["E14","E13","E16"]}},{"names":{"status":"met","support":["E1","E7"]},"method-preserved-308":{"status":"met","support":["E8","E9","E11"]},"method-change-301":{"status":"met","support":["E3","E4"]},"cacheable":{"status":"met","support":["E5","E6"]},"specifications":{"status":"met","support":["E14","E13","E16"]}}]
- judge failures
- 0
- grounded weight share
- 1
Final answer
Summary
301 (Moved Permanently) and 308 (Permanent Redirect) both say the resource has permanently moved to a new URI. They differ in how the client may treat the request method on the follow-up request. With 301, a client may historically turn a POST into a GET. With 308, the method and body must not change.
301 Moved Permanently
- Meaning: RFC 9110 says the target resource has a new permanent URI, and future references should use one of the URIs the server sent. The server should send a
Locationheader, and the user agent may use it for automatic redirection. - Method and body: For historical reasons, a user agent may change the request method from POST to GET on the follow-up request. RFC 9110 says to use 308 if that is undesired. MDN says the Fetch Standard behaves this way in practice. The reports do not show RFC 9110 saying anything about the body on a 301. The body is normally dropped when POST becomes GET, but that is an inference, not something a source stated.
- Cacheability: RFC 9110 says a 301 response is heuristically cacheable. It can be cached without explicit freshness information, unless the method definition or explicit cache controls say otherwise.
- Specification: It is currently in RFC 9110 (HTTP Semantics), section 15.4.2. RFC 9110 obsoletes RFC 7231, which had itself obsoleted RFC 2616. The worker did not open RFC 2616 itself.
308 Permanent Redirect
- Meaning: RFC 9110 gives the same "new permanent URI" meaning as for 301. RFC 7538 introduced 308 because no existing code was a permanent redirect that kept the method. 307 was the temporary equivalent.
- Method and body: RFC 7538 describes 308 as similar to 301 "except that it does not allow changing the request method from POST to GET". MDN says the client will not modify the method or body in the redirected request. The 308 worker took the method rule from RFC 7538 and MDN, not from the 308 text in RFC 9110. The 301 worker read RFC 9110's wording that altering the method after a 308 is prohibited, but only via MDN.
- Cacheability: RFC 9110 says a 308 response is heuristically cacheable, the same as 301. RFC 7538 originally said "cacheable by default". The reports describe this as a wording change, with RFC 9110 recording no changes from RFC 7538 (Appendix B.7).
- Specification: It is currently in RFC 9110, section 15.4.9. It was originally defined in RFC 7538 (April 2015), which replaced the experimental RFC 7238 with no technical changes. RFC 9110 lists RFC 7538 as obsoleted. The fetched RFC 7538 page did not itself show an "Obsoleted by" line, so that conclusion rests on RFC 9110's header. Two workers quoted slightly different forms of that header: one quoted the abstract's "portions of 7230", the other a header listing 7230 outright.
- Compatibility: RFC 9110 warns that 308 is much younger (June 2014) than its sibling codes and might not be recognized everywhere. RFC 7538 explains that unknown 3xx codes are treated like 300, so servers cannot rely on automatic redirection. It restricts 308 to cases where the client is trusted to understand it or a 300 fallback is harmless.
Where the sources disagree
MDN's 308 page says 301 "requires the request method and the body to remain unchanged", and that older clients wrongly switch to GET. RFC 9110's own 301 text explicitly permits the POST-to-GET change. I treat RFC 9110 as authoritative for what the specification allows. MDN describes the intended semantics and real client behavior.
{"evidence": [
{"claim": "301 means the target resource has a new permanent URI and future references should use one of the enclosed URIs", "quote": "The 301 (Moved Permanently) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "The user agent may use the Location header value for automatic redirection", "quote": "The user agent MAY use the Location field value for automatic redirection.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "For a 301, a user agent may change POST to GET for historical reasons", "quote": "For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "RFC 9110 points to 308 when the method change is undesired", "quote": "If this behavior is undesired, the 308 (Permanent Redirect) status code can be used instead.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "301 is heuristically cacheable", "quote": "A 301 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 is heuristically cacheable under RFC 9110", "quote": "A 308 response is heuristically cacheable; i.e., unless otherwise", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 means a new permanent URI, per the current specification", "quote": "The 308 (Permanent Redirect) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 is like 301 except that the method may not be changed from POST to GET", "quote": "Note: This status code is similar to 301 (Moved Permanently) ([RFC7231], Section 6.4.2), except that it does not allow changing the request method from POST to GET.", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "With 308 the client does not modify the method or body", "quote": "The request method and the body will not be modified by the client in the redirected request.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/308"},
{"claim": "MDN's description of 301 differs from RFC 9110: it says the method should stay unchanged but older clients switch to GET", "quote": "A 301 Moved Permanently requires the request method and the body to remain unchanged when redirection is performed, but this is incorrectly handled by older clients to use the GET method instead.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/308"},
{"claim": "MDN says altering the method after a 308 is prohibited, so 308 avoids request modification", "quote": "To avoid user agents modifying the request, use 308 Permanent Redirect instead, as altering the method after a 308 response is prohibited.", "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/301"},
{"claim": "RFC 7538 originally said 308 was cacheable by default", "quote": "A 308 response is cacheable by default; i.e., unless otherwise", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "RFC 7538 had no technical changes from the experimental RFC it replaced", "quote": "This specification contains no technical changes from the", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "RFC 9110 obsoletes RFC 7231 and RFC 7538 among others", "quote": "This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "RFC 7231 obsoleted RFC 2616", "quote": "Obsoletes: 2616", "url": "https://www.rfc-editor.org/rfc/rfc7231.html"},
{"claim": "RFC 9110 records no changes from RFC 7538", "quote": "B.7. Changes from RFC 7538 None.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 is newer than its sibling codes and may not be recognized everywhere", "quote": "This status code is much younger (June 2014) than its", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "RFC 7538 restricts 308 use because clients may not understand it", "quote": "Therefore, the use of status code 308 is restricted to cases where", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"}
]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.