ScaffoldBench

RunSonnet 5.5repeat 1

Orchestrator-workers on OWASP vs NIST on password hashing

claude-sonnet-5-5__orchestrator-workers__research-password-hashing__r1

Score

0.875

wrong answer
design ran as designed
Cost
$0.552
+ $0.120 judging
Latency
1m 24s
setup 0.0s, grading 4.9s
Turns
44
Tool calls
38
Subagents
3
Tokens in / out
288k / 22k
202,773 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • The task was decomposed: heldorchestrator-workers.decompositionThe orchestrator’s reply was read through its schema and held at least one subtask.
  • The fan-out happened: heldorchestrator-workers.fan-outEvery subtask got a worker, every worker ran, and the recorded subagent count matches.
  • The synthesis wrote the answer: heldorchestrator-workers.synthesisThe final answer came from the synthesis call, not from a worker’s report standing in for it.
  • Each call was shown the right tools: heldorchestrator-workers.toolsThe orchestrator and the synthesis were shown no tools and every worker the task’s full toolset.

Phases

Decompose: 0.0s to 8.4s, finished, 0 tool calls, $0.014
Worker 1: owasp-guidance: 8.4s to 32.9s, finished, 3 tool calls, $0.075
Worker 2: nist-800-63b: 8.4s to 54.3s, finished, 15 tool calls, $0.170
Worker 3: fips-pbkdf2-reconcile: 8.4s to 49.2s, finished, 20 tool calls, $0.179
Synthesize: 54.3s to 1m 24s, finished, 0 tool calls, $0.113

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Decompose8.4s$0.01420claude-sonnet-5-5
Worker 1: owasp-guidance24.4s$0.07543claude-sonnet-5-5
Worker 2: nist-800-63b45.9s$0.1701615claude-sonnet-5-5
Worker 3: fips-pbkdf2-reconcile40.8s$0.1792120claude-sonnet-5-5
Synthesize29.2s$0.11310claude-sonnet-5-5

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies orchestrator-workers --tasks research-password-hashing --models claude-sonnet-5-5 --repeats 1

Score breakdown

  • passStates that OWASP recommends Argon2id first, with a minimum configuration of 19 MiB of memory, an iteration count of 2 and 1 degree of parallelism.
    1.00weight 0.19

    met, backed by verified evidence E1. “**Argon2id** is the default, with a minimum of 19 MiB of memory, 2 iterations and parallelism 1.”

  • passStates OWASP's alternatives as conditions: scrypt if Argon2id is not available, bcrypt only for legacy systems, and PBKDF2 (at least 600,000 iterations with HMAC-SHA-256) if FIPS-140 compliance is required.
    1.00weight 0.13

    met, backed by verified evidence E2, E3, E4. “**PBKDF2** is for when FIPS-140 compliance is required, with at least 600,000 iterations and HMAC-SHA-256 as the internal hash.”

  • passStates that the current NIST SP 800-63B (revision 4) requires passwords to be salted and hashed with a suitable password hashing scheme that takes a cost factor, and says an approved scheme from the latest revision of SP 800-132 (or updated NIST guidelines) SHOULD be used.
    1.00weight 0.19

    met, backed by verified evidence E13, E14, E15. “Rev. 4 says an approved scheme from SP 800-132 or updated NIST guidelines SHOULD be used.”

  • passStates that the superseded revision 3 named PBKDF2 and Balloon as example key derivation functions and said a memory-hard function SHOULD be used, and that revision 4 no longer contains that wording.
    1.00weight 0.13

    met, backed by verified evidence E21, E22. “It named PBKDF2 and Balloon as examples, said a memory-hard function SHOULD be used, and gave "typically at least 10,000 iterations". Rev. 4 dropped all three.”

  • partialExplains that for this team the two do not actually conflict: OWASP's own guidance is to use PBKDF2 (HMAC-SHA-256, at least 600,000 iterations) when FIPS-140 compliance is required, and NIST permits PBKDF2, so PBKDF2-HMAC-SHA-256 satisfies both; Argon2id remains OWASP's choice only where FIPS does not apply.
    0.50weight 0.25

    partial, backed by verified evidence E4, E5. “If FIPS governs the service, use PBKDF2-HMAC-SHA-256 with at least 600,000 iterations. This is the only choice both OWASP and NIST support.”

  • passStates at least one requirement both sources share, such as a unique salt for every password or a deliberately expensive hash with a tunable cost (work) factor instead of a fast general-purpose hash.
    1.00weight 0.13

    met, backed by verified evidence E7, E13, E16. “Use a random, unique salt. The NIST floor is 32 bits”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"e12dc326c69a32d91f85f6355dc63e2a7d672b605685a1ffa1582a42bb1c561e","documents":5}
opened documents
["owasp-password-storage","nist-sp800-63b-4","nist-sp800-63b-3"]
opened by role
{"primary":["owasp-password-storage","nist-sp800-63b-4"],"superseded":["nist-sp800-63b-3"]}
displayed share
{"owasp-password-storage":1,"nist-sp800-63b-4":0.1058,"nist-sp800-63b-3":0.0604,"rfc9106":0.0422,"owasp-authentication":0.0403}
cited count
3
verified count
3
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
22
verified quote count
22
verified quote rate
1
evidence
[{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP's default recommendation is Argon2id with a minimum of 19 MiB memory, 2 iterations, parallelism 1.","quote":"Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP names scrypt as the fallback when Argon2id is unavailable.","quote":"If Argon2id is not available, use scrypt with a minimum CPU/memory cost parameter of (2^17), a minimum block size of 8 (1024 bytes), and a parallelization parameter of 1.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP limits bcrypt to legacy systems.","quote":"For legacy systems using bcrypt, use a work factor of 10 or more and with a password limit of 72 bytes.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"When FIPS-140 compliance is required, OWASP says to use PBKDF2 with 600,000+ iterations and HMAC-SHA-256.","quote":"If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP justifies PBKDF2 for FIPS by citing NIST SP 800-63B-4 and FIPS-140 validated implementations.","quote":"Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP marks PBKDF2-HMAC-SHA1 as legacy only.","quote":"PBKDF2-HMAC-SHA1: 1,400,000 iterations — legacy only, do not select for new systems.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP requires a unique salt per password.","quote":"A unique salt must be added to each password to prevent attackers from using precomputed lookup tables like rainbow tables.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP says a pepper should be stored separately from the password database.","quote":"The pepper should be stored separately from the password database.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP advises hashing time under one second as a general rule when tuning the work factor.","quote":"As a general rule, calculating a hash should take less than one second.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP suggests upgrading work factors by re-hashing at next login.","quote":"The most common approach to upgrading the work factor is to wait until the user next authenticates, then re-hash their password with the new work factor.","verified":true,"document":"owasp-password-storage"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"SP 800-63B-4 (Rev. 4) supersedes the earlier SP 800-63B.","quote":"This publication supersedes NIST Special Publication (SP) 800-63B.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-3/sp800-63b.html","claim":"The Rev. 3 page states it has been superseded by SP 800-63-4, so Rev. 4 is current.","quote":"This revision of NIST SP 800-63 has been superseded by NIST SP 800-63-4 as of August 1, 2025.","verified":true,"document":"nist-sp800-63b-3"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Rev. 4 requires salted hashing with a suitable password hashing scheme.","quote":"Passwords SHALL be salted and hashed using a suitable password hashing scheme.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Rev. 4 sets the cost factor as a SHOULD with no numeric value.","quote":"The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Rev. 4 names no algorithm and defers to SP 800-132 or updated NIST guidelines, as a SHOULD.","quote":"An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Rev. 4 sets a 32-bit minimum salt.","quote":"The salt SHALL be at least 32 bits in length and chosen to minimize salt value collisions among stored hashes","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Rev. 4 recommends an additional keyed hashing or encryption step with a secret key (pepper-like).","quote":"In addition, verifiers SHOULD perform an additional iteration of a keyed hashing or encryption operation using a secret key known only to the verifier.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Rev. 4 requires the secret key to be stored separately from the hashed passwords.","quote":"The secret key value SHALL be stored separately from the hashed passwords.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Rev. 4 ties FIPS 140 requirements to CMVP-validated cryptography.","quote":"Throughout this document, [FIPS140] requirements are satisfied by security technologies, products, and services that utilize implementations of cryptography validated by the Cryptography Module Validation Program [CMVP].","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"Federal verifiers at AAL1 must use FIPS 140 Level 1 validated cryptography.","quote":"Cryptography used by verifiers operated by or on behalf of federal agencies at AAL1 SHALL be validated to meet the requirements of [FIPS140] Level 1.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-3/sp800-63b.html","claim":"The superseded Rev. 3 named PBKDF2 and Balloon as examples and recommended a memory-hard function; Rev. 4 dropped these.","quote":"Examples of suitable key derivation functions include Password-based Key Derivation Function 2 (PBKDF2) [SP 800-132] and Balloon [BALLOON]. A memory-hard function SHOULD be used because it increases the cost of an attack.","verified":true,"document":"nist-sp800-63b-3"},{"url":"https://pages.nist.gov/800-63-3/sp800-63b.html","claim":"Rev. 3 gave a PBKDF2 iteration guideline of at least 10,000, far lower than OWASP's 600,000.","quote":"Therefore, the iteration count SHOULD be as large as verification server performance will allow, typically at least 10,000 iterations.","verified":true,"document":"nist-sp800-63b-3"}]
judge unverified quotes
0
judge verdicts
[{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2","E3","E4"]},"nist-current":{"status":"met","support":["E13","E14","E15"]},"nist-revision":{"status":"met","support":["E21","E22"]},"reconciliation":{"status":"partial","support":["E4","E5"]},"common-ground":{"status":"met","support":["E7","E13","E16"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2","E3","E4"]},"nist-current":{"status":"met","support":["E13","E14","E15"]},"nist-revision":{"status":"met","support":["E21","E22"]},"reconciliation":{"status":"partial","support":["E4","E5"]},"common-ground":{"status":"met","support":["E7","E13","E16"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2","E3","E4"]},"nist-current":{"status":"met","support":["E13","E14","E15"]},"nist-revision":{"status":"met","support":["E21","E22"]},"reconciliation":{"status":"partial","support":["E4","E5"]},"common-ground":{"status":"met","support":["E7","E13","E16"]}}]
judge failures
0
grounded weight share
1

Final answer

Short answer: If the service has to meet FIPS requirements, use PBKDF2 with HMAC-SHA-256 and at least 600,000 iterations, run inside a FIPS 140-validated cryptographic module. If FIPS does not apply, OWASP's default is Argon2id. Three points could not be confirmed from the sources: whether Argon2, scrypt or bcrypt is FIPS-approved, the contents of SP 800-132, and the "Final" status of Rev. 4 on csrc.nist.gov.

What OWASP recommends

I took this from the OWASP Password Storage Cheat Sheet (https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html). It ranks the algorithms as follows:

  1. Argon2id is the default, with a minimum of 19 MiB of memory, 2 iterations and parallelism 1. The page lists equal-strength variants that trade memory for iterations, such as 46 MiB with t=1 or 12 MiB with t=3.
  2. scrypt is the fallback if Argon2id is unavailable, with N=2^17, r=8 and p=1 as the minimum.
  3. bcrypt is for legacy systems only, with a work factor of at least 10 and a 72-byte password limit.
  4. PBKDF2 is for when FIPS-140 compliance is required, with at least 600,000 iterations and HMAC-SHA-256 as the internal hash. For other hashes, the page gives 220,000 iterations for HMAC-SHA-512. It gives 1,400,000 for HMAC-SHA-1, marked "legacy only, do not select for new systems".

OWASP says PBKDF2 should be preferred when FIPS is required because it is "permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations".

OWASP also covers three supporting practices:

  • Salts: Every password needs a unique salt.
  • Peppers: A pepper is optional defense in depth. It is a shared secret stored separately from the database, and changing it forces password resets.
  • Work factor: Tune it by experiment on your own servers. As a general rule a hash should take under one second. Raise the factor over time by re-hashing at the user's next login.

What the current NIST SP 800-63B requires

The current revision is SP 800-63B-4 (Rev. 4), at https://pages.nist.gov/800-63-4/sp800-63b.html. It says it supersedes the earlier SP 800-63B. The Rev. 3 page (https://pages.nist.gov/800-63-3/sp800-63b.html) carries a banner saying it was superseded by SP 800-63-4 as of August 1, 2025.

The csrc.nist.gov "final" page returned a 404, so I could not confirm a formal "Final" status from that site. The supersession wording is my basis for treating Rev. 4 as current.

Rev. 4 requires the following:

  • Salted, hashed storage: Passwords SHALL be salted and hashed using a suitable password hashing scheme.
  • Salt length: The salt SHALL be at least 32 bits.
  • Cost factor: This is only a SHOULD. It is "as high as practical" and raised over time, with no number given.
  • Algorithm: None is named. Rev. 4 says an approved scheme from SP 800-132 or updated NIST guidelines SHOULD be used.
  • Keyed hash (pepper): Verifiers SHOULD add a keyed hash or encryption step using a secret key stored separately from the hashes.
  • FIPS 140: Cryptography used by verifiers operated by or on behalf of federal agencies at AAL1 SHALL be validated to FIPS 140 Level 1. Rev. 4 ties FIPS 140 to modules validated under the CMVP. The workers reported the same wording at AAL2, and Level 1 or higher at AAL3.

Rev. 3 was more specific. It named PBKDF2 and Balloon as examples, said a memory-hard function SHOULD be used, and gave "typically at least 10,000 iterations". Rev. 4 dropped all three.

Where the sources disagree or leave gaps

  • OWASP's NIST claim: OWASP says PBKDF2 is "permitted by NIST SP 800-63B-4". The Rev. 4 text the workers read does not name PBKDF2. It defers to SP 800-132 or later guidance. OWASP's claim is a reasonable reading, but it is not in Rev. 4's wording.
  • Iteration counts: The 600,000 figure is OWASP's. NIST Rev. 4 gives no number, and the old Rev. 3 figure of 10,000 comes from a superseded revision. Treat OWASP's figure as the operative one.
  • Unopened sources: I could not open SP 800-132 itself (several URLs returned 404). I also found no NIST or CMVP page saying whether Argon2, scrypt or bcrypt is FIPS-approved. OWASP's FIPS exception implies they are not, but no source says so directly.
  • Scope: The FIPS 140 requirement in 800-63B-4 is stated for verifiers operated by or on behalf of federal agencies. Whether it binds this team depends on whether the service falls under that scope.

How the team should reconcile the two

  1. Choose the algorithm: If FIPS governs the service, use PBKDF2-HMAC-SHA-256 with at least 600,000 iterations. This is the only choice both OWASP and NIST support. Use Argon2id only if the team's FIPS scope allows a non-validated algorithm, and check current SP 800-132 or successor guidance and the CMVP before relying on it.
  2. Salt and store metadata: Use a random, unique salt. The NIST floor is 32 bits, and a longer salt is the safer practice. Store the scheme and cost factor with each hash so you can migrate later, as both sources encourage.
  3. Run it in a validated module: Run the hashing in a FIPS 140-validated module. I did not verify any CMVP certificates myself.
  4. Add a pepper: Add a secret keyed step, stored separately and ideally in an HSM. NIST makes this a SHOULD and OWASP calls it optional. It supplements the KDF and does not replace it.
  5. Tune the work factor: Benchmark on your own hardware, keep hashing time reasonable, and raise the cost over time.
{"evidence": [
{"claim": "OWASP's default recommendation is Argon2id with a minimum of 19 MiB memory, 2 iterations, parallelism 1.", "quote": "Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP names scrypt as the fallback when Argon2id is unavailable.", "quote": "If Argon2id is not available, use scrypt with a minimum CPU/memory cost parameter of (2^17), a minimum block size of 8 (1024 bytes), and a parallelization parameter of 1.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP limits bcrypt to legacy systems.", "quote": "For legacy systems using bcrypt, use a work factor of 10 or more and with a password limit of 72 bytes.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "When FIPS-140 compliance is required, OWASP says to use PBKDF2 with 600,000+ iterations and HMAC-SHA-256.", "quote": "If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP justifies PBKDF2 for FIPS by citing NIST SP 800-63B-4 and FIPS-140 validated implementations.", "quote": "Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP marks PBKDF2-HMAC-SHA1 as legacy only.", "quote": "PBKDF2-HMAC-SHA1: 1,400,000 iterations — legacy only, do not select for new systems.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP requires a unique salt per password.", "quote": "A unique salt must be added to each password to prevent attackers from using precomputed lookup tables like rainbow tables.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP says a pepper should be stored separately from the password database.", "quote": "The pepper should be stored separately from the password database.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP advises hashing time under one second as a general rule when tuning the work factor.", "quote": "As a general rule, calculating a hash should take less than one second.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "OWASP suggests upgrading work factors by re-hashing at next login.", "quote": "The most common approach to upgrading the work factor is to wait until the user next authenticates, then re-hash their password with the new work factor.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"},
{"claim": "SP 800-63B-4 (Rev. 4) supersedes the earlier SP 800-63B.", "quote": "This publication supersedes NIST Special Publication (SP) 800-63B.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "The Rev. 3 page states it has been superseded by SP 800-63-4, so Rev. 4 is current.", "quote": "This revision of NIST SP 800-63 has been superseded by NIST SP 800-63-4 as of August 1, 2025.", "url": "https://pages.nist.gov/800-63-3/sp800-63b.html"},
{"claim": "Rev. 4 requires salted hashing with a suitable password hashing scheme.", "quote": "Passwords SHALL be salted and hashed using a suitable password hashing scheme.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "Rev. 4 sets the cost factor as a SHOULD with no numeric value.", "quote": "The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "Rev. 4 names no algorithm and defers to SP 800-132 or updated NIST guidelines, as a SHOULD.", "quote": "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "Rev. 4 sets a 32-bit minimum salt.", "quote": "The salt SHALL be at least 32 bits in length and chosen to minimize salt value collisions among stored hashes", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "Rev. 4 recommends an additional keyed hashing or encryption step with a secret key (pepper-like).", "quote": "In addition, verifiers SHOULD perform an additional iteration of a keyed hashing or encryption operation using a secret key known only to the verifier.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "Rev. 4 requires the secret key to be stored separately from the hashed passwords.", "quote": "The secret key value SHALL be stored separately from the hashed passwords.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "Rev. 4 ties FIPS 140 requirements to CMVP-validated cryptography.", "quote": "Throughout this document, [FIPS140] requirements are satisfied by security technologies, products, and services that utilize implementations of cryptography validated by the Cryptography Module Validation Program [CMVP].", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "Federal verifiers at AAL1 must use FIPS 140 Level 1 validated cryptography.", "quote": "Cryptography used by verifiers operated by or on behalf of federal agencies at AAL1 SHALL be validated to meet the requirements of [FIPS140] Level 1.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"},
{"claim": "The superseded Rev. 3 named PBKDF2 and Balloon as examples and recommended a memory-hard function; Rev. 4 dropped these.", "quote": "Examples of suitable key derivation functions include Password-based Key Derivation Function 2 (PBKDF2) [SP 800-132] and Balloon [BALLOON]. A memory-hard function SHOULD be used because it increases the cost of an attack.", "url": "https://pages.nist.gov/800-63-3/sp800-63b.html"},
{"claim": "Rev. 3 gave a PBKDF2 iteration guideline of at least 10,000, far lower than OWASP's 600,000.", "quote": "Therefore, the iteration count SHOULD be as large as verification server performance will allow, typically at least 10,000 iterations.", "url": "https://pages.nist.gov/800-63-3/sp800-63b.html"}
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.