ScaffoldBench

RunSonnet 5.5repeat 2

Single on HTTP 301 vs 308

claude-sonnet-5-5__single__research-http-redirects__r2

Score

0.875

wrong answer
design ran as designed
Cost
$0.098
+ $0.080 judging
Latency
27.2s
setup 0.0s, grading 4.6s
Turns
8
Tool calls
7
Subagents
0
Tokens in / out
53k / 3.8k
39,096 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.

Phases

Main: 0.0s to 27.2s, finished, 7 tool calls, $0.098

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Main27.2s$0.09887claude-sonnet-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies single --tasks research-http-redirects --models claude-sonnet-5-5 --repeats 1

Score breakdown

  • passStates that 301 is "Moved Permanently" and 308 is "Permanent Redirect", and that both indicate the target resource has been assigned a new permanent URI.
    1.00weight 0.17

    met, backed by verified evidence E1, E4. “**301 (Moved Permanently):** RFC 9110 says it "indicates that the target resource has been assigned a new permanent URI”

  • partialStates that a client following a 308 must not change the request method: a POST is repeated as a POST, with its body.
    0.50weight 0.25

    partial, backed by verified evidence E7. “The method is therefore preserved.”

  • passStates that when following a 301 a user agent MAY change the request method from POST to GET (for historical reasons), which is the behaviour 308 exists to rule out.
    1.00weight 0.25

    met, backed by verified evidence E2, E7. “For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request.”

  • passStates that both 301 and 308 responses are heuristically cacheable, i.e. a cache may reuse them by default unless the method definition or explicit cache controls say otherwise.
    1.00weight 0.17

    met, backed by verified evidence E3, E5. “RFC 9110 says a 301 response "is heuristically cacheable", unless the method definition or explicit cache controls say otherwise.”

  • passStates that both codes are currently specified in RFC 9110 (HTTP Semantics), and that 308 was originally defined in RFC 7538, which RFC 9110 obsoletes.
    1.00weight 0.17

    met, backed by verified evidence E9. “RFC 9110 obsoletes RFC 7231, which defined 301 earlier. It also obsoletes RFC 7538, which defined 308.”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"25fbed79c4f7f9e1b850436e0e4872f8e25dd80142f3177231c456f21817b817","documents":8}
opened documents
["rfc9110","rfc7538"]
opened by role
{"primary":["rfc9110"],"superseded":["rfc7538"]}
displayed share
{"rfc9110":0.0262,"rfc7538":0.782}
cited count
2
verified count
2
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
12
verified quote count
12
verified quote rate
1
evidence
[{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"301 means the resource has a new permanent URI and future references should use it","quote":"The 301 (Moved Permanently) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"With 301, clients may change POST to GET, and 308 is the alternative if that is undesired","quote":"For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"301 is heuristically cacheable","quote":"A 301 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 means a new permanent URI, same semantics as 301 for the target","quote":"The 308 (Permanent Redirect) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 is heuristically cacheable","quote":"A 308 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"308 might not be recognized everywhere","quote":"This status code is much younger (June 2014) than its sibling codes and thus might not be recognized everywhere.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"308 differs from 301 in not allowing POST to be changed to GET","quote":"This status code is similar to 301 (Moved Permanently) ([RFC7231], Section 6.4.2), except that it does not allow changing the request method from POST to GET.","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"RFC 7538 originally described 308 as cacheable by default","quote":"A 308 response is cacheable by default; i.e., unless otherwise indicated by the method definition or explicit cache controls (see [RFC7234], Section 4.2.2).","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"RFC 9110 obsoletes RFC 7231 and RFC 7538, so it is the current specification of both codes","quote":"This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230.","verified":true,"document":"rfc9110"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"Unknown 3xx codes are treated like 300, so 308 use should be limited to clients that understand it","quote":"Therefore, the use of status code 308 is restricted to cases where the server has sufficient confidence in the client's understanding the new code or when a fallback to the semantics of status code 300 is not problematic.","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc7538.html","claim":"RFC 7538 made no technical changes from the experimental RFC 7238","quote":"This specification contains no technical changes from the Experimental RFC 7238, which it obsoletes.","verified":true,"document":"rfc7538"},{"url":"https://www.rfc-editor.org/rfc/rfc9110.html","claim":"The temporary counterpart 307 explicitly forbids changing the method","quote":"the user agent MUST NOT change the request method if it performs an automatic redirection to that URI.","verified":true,"document":"rfc9110"}]
judge unverified quotes
0
judge verdicts
[{"names":{"status":"met","support":["E1","E4"]},"method-preserved-308":{"status":"partial","support":["E7"]},"method-change-301":{"status":"met","support":["E2","E7"]},"cacheable":{"status":"met","support":["E3","E5"]},"specifications":{"status":"met","support":["E9"]}},{"names":{"status":"met","support":["E1","E4"]},"method-preserved-308":{"status":"partial","support":["E7"]},"method-change-301":{"status":"met","support":["E2"]},"cacheable":{"status":"met","support":["E3","E5"]},"specifications":{"status":"met","support":["E9"]}},{"names":{"status":"met","support":["E1","E4"]},"method-preserved-308":{"status":"partial","support":["E7"]},"method-change-301":{"status":"met","support":["E2"]},"cacheable":{"status":"met","support":["E3","E5"]},"specifications":{"status":"met","support":["E9"]}}]
judge failures
0
grounded weight share
1

Final answer

Short answer: 301 and 308 both mean "this resource has a new permanent URI". They differ in one respect. With 301, a client may change a POST to a GET when it follows the redirect. With 308, it may not.

What each means

  • 301 (Moved Permanently): RFC 9110 says it "indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs." The server should send a Location header with the new URI, and the user agent may use it to redirect automatically. (https://www.rfc-editor.org/rfc/rfc9110.html)
  • 308 (Permanent Redirect): The wording is essentially identical. It also signals a new permanent URI, with the same Location handling. (https://www.rfc-editor.org/rfc/rfc9110.html)
  • In both, the spec says a user agent with link-editing ability can permanently replace references to the old URI. It adds that this "is usually ignored unless the user agent is actively editing references".

Request method and body when following the redirect

  • 301: RFC 9110 has a note saying "For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request." The same note says that if this behaviour is undesired, 308 can be used instead. In practice, a POST to a 301 URL may arrive at the new URL as a GET, and a GET normally carries no body. (https://www.rfc-editor.org/rfc/rfc9110.html)
  • 308: The original RFC 7538 describes it as similar to 301 "except that it does not allow changing the request method from POST to GET." The method is therefore preserved. RFC 9110's 308 section does not repeat that rule. It states it only through the 301 note above. (https://www.rfc-editor.org/rfc/rfc7538.html)
  • Body: I found no explicit sentence about body handling in either section I read. The expectation that a 308 client resends the same body is my inference from "method not changed". It matches how 307, the temporary counterpart, is described, where RFC 9110 says the user agent "MUST NOT change the request method". (https://www.rfc-editor.org/rfc/rfc9110.html)

Cacheability
Both are cacheable by default, but not unconditionally.

Where each is specified today

Compatibility caveat

  • RFC 9110 notes that 308 "is much younger (June 2014) than its sibling codes and thus might not be recognized everywhere."
  • RFC 7538 explains that unknown 3xx codes are treated like 300 (Multiple Choices), so automatic redirection is not guaranteed. It advises using 308 only when the server is confident the client understands it, or when falling back to 300 behaviour is harmless. (https://www.rfc-editor.org/rfc/rfc7538.html)

I could not open the rfc-editor info page for RFC 9110, which returned a 404. I therefore did not independently confirm that no later RFC has replaced it. I only confirmed that the RFC 9110 text I read is the one that obsoleted RFC 7538.

{"evidence": [
{"claim": "301 means the resource has a new permanent URI and future references should use it", "quote": "The 301 (Moved Permanently) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "With 301, clients may change POST to GET, and 308 is the alternative if that is undesired", "quote": "For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "301 is heuristically cacheable", "quote": "A 301 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 means a new permanent URI, same semantics as 301 for the target", "quote": "The 308 (Permanent Redirect) status code indicates that the target resource has been assigned a new permanent URI and any future references to this resource ought to use one of the enclosed URIs.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 is heuristically cacheable", "quote": "A 308 response is heuristically cacheable; i.e., unless otherwise indicated by the method definition or explicit cache controls (see Section 4.2.2 of [CACHING]).", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 might not be recognized everywhere", "quote": "This status code is much younger (June 2014) than its sibling codes and thus might not be recognized everywhere.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "308 differs from 301 in not allowing POST to be changed to GET", "quote": "This status code is similar to 301 (Moved Permanently) ([RFC7231], Section 6.4.2), except that it does not allow changing the request method from POST to GET.", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "RFC 7538 originally described 308 as cacheable by default", "quote": "A 308 response is cacheable by default; i.e., unless otherwise indicated by the method definition or explicit cache controls (see [RFC7234], Section 4.2.2).", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "RFC 9110 obsoletes RFC 7231 and RFC 7538, so it is the current specification of both codes", "quote": "This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"},
{"claim": "Unknown 3xx codes are treated like 300, so 308 use should be limited to clients that understand it", "quote": "Therefore, the use of status code 308 is restricted to cases where the server has sufficient confidence in the client's understanding the new code or when a fallback to the semantics of status code 300 is not problematic.", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "RFC 7538 made no technical changes from the experimental RFC 7238", "quote": "This specification contains no technical changes from the Experimental RFC 7238, which it obsoletes.", "url": "https://www.rfc-editor.org/rfc/rfc7538.html"},
{"claim": "The temporary counterpart 307 explicitly forbids changing the method", "quote": "the user agent MUST NOT change the request method if it performs an automatic redirection to that URI.", "url": "https://www.rfc-editor.org/rfc/rfc9110.html"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.