ScaffoldBench

RunOpus 5.5repeat 1

Single on OWASP vs NIST on password hashing

claude-opus-5-5__single__research-password-hashing__r1

Score

1.000

passed
design ran as designed
Cost
$0.278
+ $0.097 judging
Latency
49.5s
setup 0.0s, grading 4.7s
Turns
10
Tool calls
9
Subagents
0
Tokens in / out
88k / 5.0k
66,408 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.

Phases

Main: 0.0s to 49.5s, finished, 9 tool calls, $0.278

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Main49.5s$0.278109claude-opus-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies single --tasks research-password-hashing --models claude-opus-5-5 --repeats 1

Score breakdown

  • passStates that OWASP recommends Argon2id first, with a minimum configuration of 19 MiB of memory, an iteration count of 2 and 1 degree of parallelism.
    1.00weight 0.19

    met, backed by verified evidence E1. “Argon2id first, with at least 19 MiB of memory, 2 iterations and 1 degree of parallelism.”

  • passStates OWASP's alternatives as conditions: scrypt if Argon2id is not available, bcrypt only for legacy systems, and PBKDF2 (at least 600,000 iterations with HMAC-SHA-256) if FIPS-140 compliance is required.
    1.00weight 0.13

    met, backed by verified evidence E2, E3, E4. “Use scrypt if Argon2id isn't available. Use bcrypt only on older systems where neither is available”

  • passStates that the current NIST SP 800-63B (revision 4) requires passwords to be salted and hashed with a suitable password hashing scheme that takes a cost factor, and says an approved scheme from the latest revision of SP 800-132 (or updated NIST guidelines) SHOULD be used.
    1.00weight 0.19

    met, backed by verified evidence E8, E9, E11. “Salt and hash them with a "suitable password hashing scheme".”

  • passStates that the superseded revision 3 named PBKDF2 and Balloon as example key derivation functions and said a memory-hard function SHOULD be used, and that revision 4 no longer contains that wording.
    1.00weight 0.13

    met, backed by verified evidence E14. “It named PBKDF2 and Balloon as examples and suggested "typically at least 10,000 iterations" for PBKDF2. Rev. 4 drops the memory-hard wording and the number.”

  • passExplains that for this team the two do not actually conflict: OWASP's own guidance is to use PBKDF2 (HMAC-SHA-256, at least 600,000 iterations) when FIPS-140 compliance is required, and NIST permits PBKDF2, so PBKDF2-HMAC-SHA-256 satisfies both; Argon2id remains OWASP's choice only where FIPS does not apply.
    1.00weight 0.25

    met, backed by verified evidence E4, E5, E9, E13. “OWASP itself resolves this by recommending PBKDF2 whenever FIPS is in scope.”

  • passStates at least one requirement both sources share, such as a unique salt for every password or a deliberately expensive hash with a tunable cost (work) factor instead of a fast general-purpose hash.
    1.00weight 0.13

    met, backed by verified evidence E10, E11. “Use a random salt of at least 32 bits per user”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"e12dc326c69a32d91f85f6355dc63e2a7d672b605685a1ffa1582a42bb1c561e","documents":5}
opened documents
["owasp-password-storage","nist-sp800-63b-4","nist-sp800-63b-3"]
opened by role
{"primary":["owasp-password-storage","nist-sp800-63b-4"],"superseded":["nist-sp800-63b-3"]}
displayed share
{"owasp-password-storage":0.8205,"nist-sp800-63b-4":0.0902,"nist-sp800-63b-3":0.0461,"rfc9106":0.009,"owasp-authentication":0.007}
cited count
3
verified count
3
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
15
verified quote count
15
verified quote rate
1
evidence
[{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP's default recommendation is Argon2id with minimum parameters.","quote":"Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP recommends scrypt as the fallback when Argon2id is unavailable.","quote":"If Argon2id is not available, use scrypt with a minimum CPU/memory cost parameter of (2^17), a minimum block size of 8 (1024 bytes), and a parallelization parameter of 1.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP limits bcrypt to legacy systems.","quote":"The bcrypt password hashing function should only be used for password storage in legacy systems where Argon2 and scrypt are not available.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"For FIPS-140 compliance, OWASP recommends PBKDF2 with HMAC-SHA-256 and at least 600,000 iterations.","quote":"If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP says PBKDF2 should be preferred where NIST 800-63B-4 or FIPS-140 apply.","quote":"Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP suggests an optional pepper as defense in depth.","quote":"Consider using a pepper to provide additional defense in depth (though alone, it provides no additional secure characteristics).","verified":true,"document":"owasp-password-storage"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-63B-4 supersedes the earlier SP 800-63B.","quote":"This publication supersedes NIST Special Publication (SP) 800-63B.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-63B-4 mandates salted hashing with a suitable password hashing scheme.","quote":"Passwords SHALL be salted and hashed using a suitable password hashing scheme.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-63B-4 says an approved scheme from SP 800-132 SHOULD be used (a recommendation, not a SHALL).","quote":"An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-63B-4 requires a salt of at least 32 bits.","quote":"The salt SHALL be at least 32 bits in length and chosen to minimize salt value collisions among stored hashes (i.e., to prevent multiple subscriber accounts from having the same hashed password).","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-63B-4 recommends a high cost factor that increases over time, with no fixed number.","quote":"The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-63B-4 recommends an additional keyed hashing or encryption step using a secret key.","quote":"In addition, verifiers SHOULD perform an additional iteration of a keyed hashing or encryption operation using a secret key known only to the verifier.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-132, which the requirement points to, is NIST's password-based key derivation recommendation.","quote":"[SP800-132] Turan M, Barker E, Burr W, Chen L (2010) Recommendation for Password-Based Key Derivation.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-3/sp800-63b.html","claim":"The previous revision (Rev. 3) recommended a memory-hard function, wording absent from Rev. 4.","quote":"A memory-hard function SHOULD be used because it increases the cost of an attack.","verified":true,"document":"nist-sp800-63b-3"},{"url":"https://pages.nist.gov/800-63-3/sp800-63b.html","claim":"Rev. 3 gave a typical minimum of 10,000 PBKDF2 iterations.","quote":"Therefore, the iteration count SHOULD be as large as verification server performance will allow, typically at least 10,000 iterations.","verified":true,"document":"nist-sp800-63b-3"}]
judge unverified quotes
0
judge verdicts
[{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2","E3","E4"]},"nist-current":{"status":"met","support":["E8","E9","E11"]},"nist-revision":{"status":"met","support":["E14"]},"reconciliation":{"status":"met","support":["E4","E5","E9","E13"]},"common-ground":{"status":"met","support":["E10","E11"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2","E3","E4"]},"nist-current":{"status":"met","support":["E8","E9"]},"nist-revision":{"status":"met","support":["E14"]},"reconciliation":{"status":"met","support":["E4","E5"]},"common-ground":{"status":"partial","support":["E10","E11"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2","E3","E4"]},"nist-current":{"status":"met","support":["E8","E9","E11"]},"nist-revision":{"status":"met","support":["E14","E15"]},"reconciliation":{"status":"met","support":["E4","E5","E9","E13"]},"common-ground":{"status":"met","support":["E10","E11"]}}]
judge failures
0
grounded weight share
1

Final answer

Use PBKDF2-HMAC-SHA-256 with at least 600,000 iterations, run through a FIPS 140-validated module. OWASP recommends exactly this when FIPS compliance is required. It also meets NIST SP 800-63B-4, which points to SP 800-132, the NIST document that defines PBKDF2. The two sources don't really conflict. NIST sets the rules and gives no iteration number, and OWASP fills in the settings.

What OWASP recommends (https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html)

  • Order of preference: Argon2id first, with at least 19 MiB of memory, 2 iterations and 1 degree of parallelism. Use scrypt if Argon2id isn't available. Use bcrypt only on older systems where neither is available, with a work factor of 10 or more and a 72-byte password limit.
  • For FIPS: the summary says to use PBKDF2 with HMAC-SHA-256 and at least 600,000 iterations when FIPS-140 compliance is required. The PBKDF2 section adds that it "should be the preferred algorithm" in that case, because NIST SP 800-63B-4 permits it and FIPS-140 validated implementations exist.
  • Other hash functions: with HMAC-SHA-512 it gives 220,000 iterations. HMAC-SHA-1 is for older systems only, citing NIST SP 800-131A Rev. 2.
  • Pepper: it suggests adding a secret "pepper" as extra protection, stored apart from the database, for example in a secrets vault or hardware security module (HSM).

What NIST SP 800-63B-4 requires (https://pages.nist.gov/800-63-4/sp800-63b.html)
This is the current revision. It supersedes SP 800-63B, and the page I read is dated 26 August 2025.

  • Must (SHALL):

    • Store passwords in a form that resists offline attacks.
    • Salt and hash them with a "suitable password hashing scheme".
    • Use a salt of at least 32 bits, and store the salt and the hash for each password.
  • Should (SHOULD), so recommended but not mandatory:

    • Use an approved scheme from the latest SP 800-132 "or updated NIST guidelines on password hashing schemes". The reference list gives SP 800-132 as "Recommendation for Password-Based Key Derivation", the document that defines PBKDF2.
    • Set the cost factor as high as practical and raise it over time.
    • Store which scheme and cost factor each hash used.
    • Add one more keyed hashing or encryption step with a secret key, ideally kept in an HSM or similar protected hardware. If you do this, the key SHALL be generated by an approved random bit generator and stored separately from the hashes.
  • No iteration number: Rev. 4 gives no minimum iteration count and does not name Argon2, scrypt or bcrypt.

What changed since the previous revision. Rev. 3 (https://pages.nist.gov/800-63-3/sp800-63b.html) said "A memory-hard function SHOULD be used". It named PBKDF2 and Balloon as examples and suggested "typically at least 10,000 iterations" for PBKDF2. Rev. 4 drops the memory-hard wording and the number. Older articles quoting "10,000 iterations" or "NIST wants memory-hard" are therefore out of date.

How to reconcile them. The only tension is that OWASP prefers memory-hard Argon2id, while NIST's approved route goes through SP 800-132, i.e. PBKDF2. OWASP itself resolves this by recommending PBKDF2 whenever FIPS is in scope. In practice:

  1. Use PBKDF2-HMAC-SHA-256 from a FIPS 140-validated module, with at least 600,000 iterations, tuned upward to what your servers can handle. OWASP's general advice is to keep each hash under about one second.
  2. Use a random salt of at least 32 bits per user, and store the algorithm and parameters with each hash so you can migrate later. One option OWASP describes is re-hashing with the new settings at the user's next login.
  3. Add the keyed step with a secret key held in an HSM. NIST recommends it and OWASP's "post-hashing pepper" (an HMAC over the finished hash) is the same idea.
  4. Don't use Argon2id, scrypt or bcrypt as the main scheme. That isn't because Rev. 4 forbids them, since its scheme requirement is a SHOULD. It's because they aren't the SP 800-132 approved scheme. That makes them hard to justify in a FIPS-scoped system.

Neither source says exactly what a FIPS audit will demand, so confirm module validation with your compliance team.

{"evidence": [
{"claim": "OWASP's default recommendation is Argon2id with minimum parameters.", "quote": "Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP recommends scrypt as the fallback when Argon2id is unavailable.", "quote": "If Argon2id is not available, use scrypt with a minimum CPU/memory cost parameter of (2^17), a minimum block size of 8 (1024 bytes), and a parallelization parameter of 1.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP limits bcrypt to legacy systems.", "quote": "The bcrypt password hashing function should only be used for password storage in legacy systems where Argon2 and scrypt are not available.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "For FIPS-140 compliance, OWASP recommends PBKDF2 with HMAC-SHA-256 and at least 600,000 iterations.", "quote": "If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP says PBKDF2 should be preferred where NIST 800-63B-4 or FIPS-140 apply.", "quote": "Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP suggests an optional pepper as defense in depth.", "quote": "Consider using a pepper to provide additional defense in depth (though alone, it provides no additional secure characteristics).", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "SP 800-63B-4 supersedes the earlier SP 800-63B.", "quote": "This publication supersedes NIST Special Publication (SP) 800-63B.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "SP 800-63B-4 mandates salted hashing with a suitable password hashing scheme.", "quote": "Passwords SHALL be salted and hashed using a suitable password hashing scheme.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "SP 800-63B-4 says an approved scheme from SP 800-132 SHOULD be used (a recommendation, not a SHALL).", "quote": "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "SP 800-63B-4 requires a salt of at least 32 bits.", "quote": "The salt SHALL be at least 32 bits in length and chosen to minimize salt value collisions among stored hashes (i.e., to prevent multiple subscriber accounts from having the same hashed password).", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "SP 800-63B-4 recommends a high cost factor that increases over time, with no fixed number.", "quote": "The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "SP 800-63B-4 recommends an additional keyed hashing or encryption step using a secret key.", "quote": "In addition, verifiers SHOULD perform an additional iteration of a keyed hashing or encryption operation using a secret key known only to the verifier.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "SP 800-132, which the requirement points to, is NIST's password-based key derivation recommendation.", "quote": "[SP800-132] Turan M, Barker E, Burr W, Chen L (2010) Recommendation for Password-Based Key Derivation.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "The previous revision (Rev. 3) recommended a memory-hard function, wording absent from Rev. 4.", "quote": "A memory-hard function SHOULD be used because it increases the cost of an attack.", "url": "https://pages.nist.gov/800-63-3/sp800-63b.html"},
{"claim": "Rev. 3 gave a typical minimum of 10,000 PBKDF2 iterations.", "quote": "Therefore, the iteration count SHOULD be as large as verification server performance will allow, typically at least 10,000 iterations.", "url": "https://pages.nist.gov/800-63-3/sp800-63b.html"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.