ScaffoldBench

RunSonnet 5.5repeat 1

Single on OWASP vs NIST on password hashing

claude-sonnet-5-5__single__research-password-hashing__r1

Score

0.875

wrong answer
design ran as designed
Cost
$0.086
+ $0.081 judging
Latency
32.0s
setup 0.0s, grading 4.8s
Turns
9
Tool calls
8
Subagents
0
Tokens in / out
50k / 3.7k
38,341 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.

Phases

Main: 0.0s to 32.0s, finished, 8 tool calls, $0.086

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Main32.0s$0.08698claude-sonnet-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies single --tasks research-password-hashing --models claude-sonnet-5-5 --repeats 1

Score breakdown

  • passStates that OWASP recommends Argon2id first, with a minimum configuration of 19 MiB of memory, an iteration count of 2 and 1 degree of parallelism.
    1.00weight 0.19

    met, backed by verified evidence E1. “The first choice is Argon2id, with at least 19 MiB of memory, 2 iterations and 1 degree of parallelism.”

  • passStates OWASP's alternatives as conditions: scrypt if Argon2id is not available, bcrypt only for legacy systems, and PBKDF2 (at least 600,000 iterations with HMAC-SHA-256) if FIPS-140 compliance is required.
    1.00weight 0.13

    met, backed by verified evidence E2. “If Argon2id isn't available, it says to use scrypt. It allows bcrypt for legacy systems.”

  • passStates that the current NIST SP 800-63B (revision 4) requires passwords to be salted and hashed with a suitable password hashing scheme that takes a cost factor, and says an approved scheme from the latest revision of SP 800-132 (or updated NIST guidelines) SHOULD be used.
    1.00weight 0.19

    met, backed by verified evidence E4, E5, E7. “Scheme choice (SHOULD):** an approved scheme from the latest revision of SP 800-132 or from updated NIST guidance.”

  • failStates that the superseded revision 3 named PBKDF2 and Balloon as example key derivation functions and said a memory-hard function SHOULD be used, and that revision 4 no longer contains that wording.
    0.00weight 0.13

    Not stated.

  • passExplains that for this team the two do not actually conflict: OWASP's own guidance is to use PBKDF2 (HMAC-SHA-256, at least 600,000 iterations) when FIPS-140 compliance is required, and NIST permits PBKDF2, so PBKDF2-HMAC-SHA-256 satisfies both; Argon2id remains OWASP's choice only where FIPS does not apply.
    1.00weight 0.25

    met, backed by verified evidence E2, E3. “Under FIPS, use PBKDF2-HMAC-SHA256 with at least 600,000 iterations. This is OWASP's FIPS-specific advice, and OWASP says PBKDF2 is permitted by 800-63B-4.”

  • passStates at least one requirement both sources share, such as a unique salt for every password or a deliberately expensive hash with a tunable cost (work) factor instead of a fast general-purpose hash.
    1.00weight 0.13

    met, backed by verified evidence E6, E7. “Both sources recommend a pepper.”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"e12dc326c69a32d91f85f6355dc63e2a7d672b605685a1ffa1582a42bb1c561e","documents":5}
opened documents
["owasp-password-storage","nist-sp800-63b-4"]
opened by role
{"primary":["owasp-password-storage","nist-sp800-63b-4"]}
displayed share
{"owasp-password-storage":0.4724,"nist-sp800-63b-4":0.0308}
cited count
2
verified count
2
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
8
verified quote count
8
verified quote rate
1
evidence
[{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP recommends Argon2id as the primary choice, with minimum parameters.","quote":"Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP's FIPS-specific recommendation is PBKDF2 with at least 600,000 iterations and HMAC-SHA-256.","quote":"If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP says PBKDF2 is permitted by NIST SP 800-63B-4, has FIPS-validated implementations, and should be preferred when those are required.","quote":"Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.","verified":true,"document":"owasp-password-storage"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST 800-63B-4 requires salted hashing with a suitable password hashing scheme.","quote":"Passwords SHALL be salted and hashed using a suitable password hashing scheme.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST defers the choice of approved scheme to SP 800-132 or updated NIST guidance, and does not name a specific algorithm in this document.","quote":"An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST requires a salt of at least 32 bits and storing both salt and hash.","quote":"The salt SHALL be at least 32 bits in length and chosen to minimize salt value collisions among stored hashes","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST recommends a high, increasing cost factor.","quote":"The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST recommends an additional keyed-hash iteration with a secret key stored separately from the hashes.","quote":"In addition, verifiers SHOULD perform an additional iteration of a keyed hashing or encryption operation using a secret key known only to the verifier.","verified":true,"document":"nist-sp800-63b-4"}]
judge unverified quotes
0
judge verdicts
[{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"met","support":["E4","E5","E7"]},"nist-revision":{"status":"missing","support":[]},"reconciliation":{"status":"met","support":["E2","E3"]},"common-ground":{"status":"met","support":["E6","E7"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"met","support":["E4","E5","E7"]},"nist-revision":{"status":"missing","support":[]},"reconciliation":{"status":"met","support":["E2","E3"]},"common-ground":{"status":"met","support":["E6","E7","E8"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"met","support":["E4","E5","E7"]},"nist-revision":{"status":"missing","support":[]},"reconciliation":{"status":"met","support":["E2","E3"]},"common-ground":{"status":"met","support":["E6","E7","E8"]}}]
judge failures
0
grounded weight share
0.875 (88%)

Final answer

I've read the OWASP cheat sheet and the NIST SP 800-63B-4 page, so I'm writing up the comparison now.

What OWASP recommends

OWASP's Password Storage Cheat Sheet gives a ranked list. The first choice is Argon2id, with at least 19 MiB of memory, 2 iterations and 1 degree of parallelism. If Argon2id isn't available, it says to use scrypt. It allows bcrypt for legacy systems. For FIPS it has a separate rule: "If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256." Its PBKDF2 section says PBKDF2 should be the preferred algorithm when FIPS-validated implementations are required. It gives these iteration counts:

  • PBKDF2-HMAC-SHA256: 600,000
  • PBKDF2-HMAC-SHA512: 220,000
  • PBKDF2-HMAC-SHA1: 1,400,000, which it marks as legacy

Source: https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html

What NIST SP 800-63B-4 requires

I read the current revision, 800-63B-4, at https://pages.nist.gov/800-63-4/sp800-63b.html. The page text I retrieved had no "Final" status label, so I'm relying on the title and the OWASP sheet's reference to 800-63B-4. Section 3.1.1.2 sets these requirements:

  • Salting and hashing (SHALL): passwords must be salted and hashed with "a suitable password hashing scheme."
  • Salt (SHALL): at least 32 bits, chosen to minimize collisions. Both the salt and the hash must be stored.
  • Cost factor (SHOULD): as high as practical without hurting verifier performance, and increased over time.
  • Scheme choice (SHOULD): an approved scheme from the latest revision of SP 800-132 or from updated NIST guidance.
  • Keyed hash (SHOULD): an additional iteration of a keyed hash or encryption using a secret key held only by the verifier. The key should be stored separately, ideally in an HSM or TEE.

My searches of the page found no mention of PBKDF2 or Argon2id. The only "Argon" hit was the word "jargon." So 800-63B-4 doesn't name a specific algorithm. It defers to SP 800-132, which I did not open. I'm therefore not claiming from a source I read what SP 800-132 contains.

How to reconcile them

The two sources don't actually conflict. NIST sets the requirements, and OWASP supplies concrete choices.

  1. Under FIPS, use PBKDF2-HMAC-SHA256 with at least 600,000 iterations. This is OWASP's FIPS-specific advice, and OWASP says PBKDF2 is permitted by 800-63B-4. Use a FIPS 140-validated cryptographic module. OWASP says such implementations exist for PBKDF2.
  2. Argon2id is OWASP's first choice, but I found no source in this session showing it is NIST-approved or FIPS-validated. If FIPS compliance is mandatory, don't use it as the primary scheme. I'd consider it only if the compliance scope permits.
  3. Meet the NIST items that don't depend on the algorithm:

    • a salt of at least 32 bits, and in practice a longer random one
    • a stored record of the scheme and cost factor, so you can migrate later
    • a cost factor that you raise over time
    • a secret key (pepper) for an extra keyed-hash step, stored in an HSM or vault, separate from the database
  4. Confirm the exact approval status with your compliance assessor, because NIST says the approved list lives in SP 800-132 or in later NIST guidance.

Both sources recommend a pepper. OWASP says it adds defense in depth, and NIST says to add a keyed-hash iteration.

{"evidence": [
{"claim": "OWASP recommends Argon2id as the primary choice, with minimum parameters.", "quote": "Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP's FIPS-specific recommendation is PBKDF2 with at least 600,000 iterations and HMAC-SHA-256.", "quote": "If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP says PBKDF2 is permitted by NIST SP 800-63B-4, has FIPS-validated implementations, and should be preferred when those are required.", "quote": "Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "NIST 800-63B-4 requires salted hashing with a suitable password hashing scheme.", "quote": "Passwords SHALL be salted and hashed using a suitable password hashing scheme.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "NIST defers the choice of approved scheme to SP 800-132 or updated NIST guidance, and does not name a specific algorithm in this document.", "quote": "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "NIST requires a salt of at least 32 bits and storing both salt and hash.", "quote": "The salt SHALL be at least 32 bits in length and chosen to minimize salt value collisions among stored hashes", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "NIST recommends a high, increasing cost factor.", "quote": "The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "NIST recommends an additional keyed-hash iteration with a secret key stored separately from the hashes.", "quote": "In addition, verifiers SHOULD perform an additional iteration of a keyed hashing or encryption operation using a secret key known only to the verifier.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.