RunHaiku 4.5repeat 2
Single on OWASP vs NIST on password hashing
claude-haiku-4-5-20251001__single__research-password-hashing__r2
Score
0.719
- Cost
- $0.184
- + $0.071 judging
- Latency
- 1m 48s
- setup 0.0s, grading 4.5s
- Turns
- 33
- Tool calls
- 32
- Subagents
- 0
- Tokens in / out
- 645k / 7.6k
- 597,915 read from cache
Did the design run?
- Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
- No API failures: heldharness.apiNo model call failed at the API.
- Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
- No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
- Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
- No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
- Builtin tools: heldharness.builtin-toolsA check recorded with the run.
- Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
- No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
- Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
- Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.
Phases
Drawn from the phase records: timing and cost per call, no tool calls or messages.
| Phase | Duration | Cost | Turns | Tools | Model |
|---|---|---|---|---|---|
| Main | 1m 48s | $0.184 | 33 | 32 | claude-haiku-4-5-20251001 |
The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.
Runs you can replay
- A gate that blocked the right moveGated-execute · Haiku 4.5
- Three proposals beat one agentPropose-select · Haiku 4.5
- A verifier that earned its keepPlan-execute-verify · Sonnet 5.5
- A revision that did not helpPlan-execute-verify · Haiku 4.5
- The control, missing a stepSingle · Haiku 4.5
Record this cell yourself: npm run bench -- run --strategies single --tasks research-password-hashing --models claude-haiku-4-5-20251001 --repeats 1
Score breakdown
- passStates that OWASP recommends Argon2id first, with a minimum configuration of 19 MiB of memory, an iteration count of 2 and 1 degree of parallelism.1.00weight 0.19
met, backed by verified evidence E1. “The primary recommendation is Argon2id with minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.”
- passStates OWASP's alternatives as conditions: scrypt if Argon2id is not available, bcrypt only for legacy systems, and PBKDF2 (at least 600,000 iterations with HMAC-SHA-256) if FIPS-140 compliance is required.1.00weight 0.13
met, backed by verified evidence E2. “For systems without Argon2id support, OWASP recommends scrypt with specific parameters”
- partialStates that the current NIST SP 800-63B (revision 4) requires passwords to be salted and hashed with a suitable password hashing scheme that takes a cost factor, and says an approved scheme from the latest revision of SP 800-132 (or updated NIST guidelines) SHOULD be used.0.50weight 0.19
partial, backed by verified evidence E3, E4. “NIST SP 800-63B-4 requires that "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used."”
- failStates that the superseded revision 3 named PBKDF2 and Balloon as example key derivation functions and said a memory-hard function SHOULD be used, and that revision 4 no longer contains that wording.0.00weight 0.13
Not stated.
- passExplains that for this team the two do not actually conflict: OWASP's own guidance is to use PBKDF2 (HMAC-SHA-256, at least 600,000 iterations) when FIPS-140 compliance is required, and NIST permits PBKDF2, so PBKDF2-HMAC-SHA-256 satisfies both; Argon2id remains OWASP's choice only where FIPS does not apply.1.00weight 0.25
met, backed by verified evidence E2, E5. “use PBKDF2 with HMAC-SHA-256 and a work factor of 600,000 iterations”
- partialStates at least one requirement both sources share, such as a unique salt for every password or a deliberately expensive hash with a tunable cost (work) factor instead of a fast general-purpose hash.0.50weight 0.13
partial, backed by verified evidence E4. “The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance”
Penalties
No penalties were applied.
Scorer metadata
- corpus
- {"version":2,"snapshot":"2026-10-01","hash":"e12dc326c69a32d91f85f6355dc63e2a7d672b605685a1ffa1582a42bb1c561e","documents":5}
- opened documents
- ["owasp-password-storage","nist-sp800-63b-4","nist-sp800-63b-3"]
- opened by role
- {"primary":["owasp-password-storage","nist-sp800-63b-4"],"superseded":["nist-sp800-63b-3"]}
- displayed share
- {"owasp-password-storage":0.9877,"owasp-authentication":0.0216,"nist-sp800-63b-3":0.0547,"nist-sp800-63b-4":0.1244,"rfc9106":0.0551}
- cited count
- 2
- verified count
- 2
- hallucinated count
- 0
- citation integrity
- 1
- hallucinated urls
- []
- evidence block
- true
- evidence block repaired
- false
- evidence count
- 5
- verified quote count
- 5
- verified quote rate
- 1
- evidence
- [{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP recommends using Argon2id as the primary password hashing algorithm with specific minimum parameters","quote":"Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP recommends using PBKDF2 with 600,000 iterations of HMAC-SHA-256 when FIPS-140 compliance is required","quote":"If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.","verified":true,"document":"owasp-password-storage"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"The current NIST standard SP 800-63B-4 requires using an approved password hashing scheme published in SP 800-132 or updated NIST guidelines","quote":"An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST SP 800-63B-4 requires the cost factor to be as high as practical and increased over time","quote":"The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance. It SHOULD be increased over time to account for increases in computing performance.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"PBKDF2 is the reconciling choice because it has FIPS-140 validated implementations and is permitted by NIST SP 800-63B-4","quote":"Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.","verified":true,"document":"owasp-password-storage"}]
- judge unverified quotes
- 0
- judge verdicts
- [{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"partial","support":["E3","E4"]},"nist-revision":{"status":"missing","support":[]},"reconciliation":{"status":"met","support":["E2","E5"]},"common-ground":{"status":"partial","support":["E4"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"partial","support":["E3","E4"]},"nist-revision":{"status":"missing","support":[]},"reconciliation":{"status":"met","support":["E2","E5"]},"common-ground":{"status":"partial","support":["E4"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"partial","support":["E3","E4"]},"nist-revision":{"status":"missing","support":[]},"reconciliation":{"status":"met","support":["E2","E5"]},"common-ground":{"status":"partial","support":["E4"]}}]
- judge failures
- 0
- grounded weight share
- 0.875 (88%)
Final answer
Now I have enough information to provide a comprehensive answer. Let me gather the key quotes for the evidence section.
Based on my research, I can now provide you with a complete answer about password hashing algorithm recommendations for US federal compliance.
Answer
OWASP Password Storage Guidance:
OWASP's Password Storage Cheat Sheet recommends a tiered approach depending on requirements and constraints. The primary recommendation is Argon2id with minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism. For systems without Argon2id support, OWASP recommends scrypt with specific parameters (minimum CPU/memory cost parameter of 2^17, minimum block size of 8, and parallelization parameter of 1). For legacy systems, bcrypt with a work factor of 10 or more is acceptable. Critically, OWASP explicitly states: "If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256."
Current NIST SP 800-63B Requirement:
The current revision of NIST SP 800-63B is revision 4 (800-63B-4), which superseded revision 3 as of August 1, 2025. NIST SP 800-63B-4 requires that "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used." The standard requires that "The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance" and "It SHOULD be increased over time to account for increases in computing performance."
NIST SP 800-132 is the "Recommendation for Password-Based Key Derivation" published in 2010, which specifies PBKDF2 (Password-Based Key Derivation Function 2).
How to Reconcile the Two:
The reconciliation is straightforward and explicitly addressed in OWASP's guidance. OWASP itself explains the reconciliation by stating that "Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required." The key constraint is that PBKDF2 is the only algorithm among the modern recommendations that has FIPS-140 validated implementations.
Therefore, for a team that must satisfy both OWASP security best practices and US federal FIPS-140 requirements, the answer is clear: use PBKDF2 with HMAC-SHA-256 and a work factor of 600,000 iterations. This meets OWASP's explicit FIPS-140 compliance guidance and satisfies NIST SP 800-63B-4's requirements by using an approved password hashing scheme from SP 800-132.
If FIPS-140 compliance is not required, the team should follow OWASP's preferred recommendations and use Argon2id, as it provides superior security properties through memory-hard algorithms that resist GPU and ASIC attacks more effectively than PBKDF2. However, the FIPS-140 constraint eliminates this option for federal systems, leaving PBKDF2 as the only practical choice among the recommended algorithms.
{"evidence": [{"claim": "OWASP recommends using Argon2id as the primary password hashing algorithm with specific minimum parameters", "quote": "Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"}, {"claim": "OWASP recommends using PBKDF2 with 600,000 iterations of HMAC-SHA-256 when FIPS-140 compliance is required", "quote": "If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"}, {"claim": "The current NIST standard SP 800-63B-4 requires using an approved password hashing scheme published in SP 800-132 or updated NIST guidelines", "quote": "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"}, {"claim": "NIST SP 800-63B-4 requires the cost factor to be as high as practical and increased over time", "quote": "The chosen cost factor SHOULD be as high as practical without negatively impacting verifier performance. It SHOULD be increased over time to account for increases in computing performance.", "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"}, {"claim": "PBKDF2 is the reconciling choice because it has FIPS-140 validated implementations and is permitted by NIST SP 800-63B-4", "quote": "Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.", "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"}]}Provenance of the numbers on this page
- run set
- scaffold-or-scale-v1
- models
- Haiku 4.5, Sonnet 5.5, Opus 5.5
- effort
- high
- repeats
- 2
- fixtures
- replay
- runs
- 900 / 900
- strategy cost
- $354.44
- judge cost
- $9.19
- wall clock
- 9h 48m
- git
- fce5807b2d
Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.