ScaffoldBench

RunHaiku 4.5repeat 1

Single on OWASP vs NIST on password hashing

claude-haiku-4-5-20251001__single__research-password-hashing__r1

Score

0.828

wrong answer
design ran as designed
Cost
$0.151
+ $0.079 judging
Latency
1m 21s
setup 0.0s, grading 5.4s
Turns
28
Tool calls
27
Subagents
0
Tokens in / out
463k / 6.9k
422,131 read from cache

Did the design run?

every check held
  • Run completed: heldharness.run-completedThe environment was built, the architecture returned and grading finished, none of them throwing.
  • No API failures: heldharness.apiNo model call failed at the API.
  • Every call ended: heldharness.calls-endedEvery call ended with a result, or was stopped by the run's own limits.
  • No harness tool denied: heldharness.harness-toolsThe sandbox guard never denied a harness tool, nor a tool the session itself offered.
  • Tool servers connected: heldharness.mcp-connectedEvery tool server the harness gave a call was connected when the session started.
  • No foreign tool servers: heldharness.mcp-hermeticNo call loaded a tool server the harness did not give it.
  • Builtin tools: heldharness.builtin-toolsA check recorded with the run.
  • Structured replies arrived: heldharness.structured-outputEvery call that asked for structured output and ran to the end got it.
  • No silent fallbacks: heldharness.not-degradedNo phase fell back on a default (an empty plan, a default winner) instead of its output.
  • Judge answered: heldharness.judgeAt least one grading call returned a usable verdict.
  • Each call was shown the right tools: heldsingle.toolsThe agent was shown exactly the task’s tools, none missing and none extra.

Phases

Main: 0.0s to 1m 21s, finished, 27 tool calls, $0.151

Drawn from the phase records: timing and cost per call, no tool calls or messages.

PhaseDurationCostTurnsToolsModel
Main1m 21s$0.1512827claude-haiku-4-5-20251001

The call-by-call replay is not published for this run. Traces are large, so the repository keeps 9 of this run set’s 900 and leaves the rest out; the score, phases and final answer on this page come from the committed run record.

Runs you can replay

Record this cell yourself: npm run bench -- run --strategies single --tasks research-password-hashing --models claude-haiku-4-5-20251001 --repeats 1

Score breakdown

  • passStates that OWASP recommends Argon2id first, with a minimum configuration of 19 MiB of memory, an iteration count of 2 and 1 degree of parallelism.
    1.00weight 0.19

    met, backed by verified evidence E1. “Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.”

  • passStates OWASP's alternatives as conditions: scrypt if Argon2id is not available, bcrypt only for legacy systems, and PBKDF2 (at least 600,000 iterations with HMAC-SHA-256) if FIPS-140 compliance is required.
    1.00weight 0.13

    met, backed by verified evidence E2. “when FIPS-140 compliance is required, OWASP specifies using PBKDF2 with a work factor of 600,000 or more iterations, combined with HMAC-SHA-256”

  • passStates that the current NIST SP 800-63B (revision 4) requires passwords to be salted and hashed with a suitable password hashing scheme that takes a cost factor, and says an approved scheme from the latest revision of SP 800-132 (or updated NIST guidelines) SHOULD be used.
    1.00weight 0.19

    met, backed by verified evidence E4, E5. “An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.”

  • partialStates that the superseded revision 3 named PBKDF2 and Balloon as example key derivation functions and said a memory-hard function SHOULD be used, and that revision 4 no longer contains that wording.
    0.50weight 0.13

    partial, backed by verified evidence E6. “NIST's earlier revision (SP 800-63B-3) explicitly stated that "A memory-hard function SHOULD be used because it increases the cost of an attack," mentioning PBKDF2 and Balloon as examples”

  • passExplains that for this team the two do not actually conflict: OWASP's own guidance is to use PBKDF2 (HMAC-SHA-256, at least 600,000 iterations) when FIPS-140 compliance is required, and NIST permits PBKDF2, so PBKDF2-HMAC-SHA-256 satisfies both; Argon2id remains OWASP's choice only where FIPS does not apply.
    1.00weight 0.25

    met, backed by verified evidence E2, E3, E5. “Teams requiring federal compliance can use PBKDF2 with appropriate work factors and satisfy both OWASP security principles and NIST requirements.”

  • partialStates at least one requirement both sources share, such as a unique salt for every password or a deliberately expensive hash with a tunable cost (work) factor instead of a fast general-purpose hash.
    0.13weight 0.13

    partial, but no evidence entry supports it. “Passwords "SHALL be salted and hashed using a suitable password hashing scheme."”

Penalties

No penalties were applied.

Scorer metadata

corpus
{"version":2,"snapshot":"2026-10-01","hash":"e12dc326c69a32d91f85f6355dc63e2a7d672b605685a1ffa1582a42bb1c561e","documents":5}
opened documents
["owasp-password-storage","nist-sp800-63b-4","nist-sp800-63b-3"]
opened by role
{"primary":["owasp-password-storage","nist-sp800-63b-4"],"superseded":["nist-sp800-63b-3"]}
displayed share
{"owasp-password-storage":1,"owasp-authentication":0.0332,"rfc9106":0.0381,"nist-sp800-63b-4":0.1804,"nist-sp800-63b-3":0.0556}
cited count
3
verified count
3
hallucinated count
0
citation integrity
1
hallucinated urls
[]
evidence block
true
evidence block repaired
false
evidence count
6
verified quote count
6
verified quote rate
1
evidence
[{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP recommends Argon2id with minimum 19 MiB of memory, iteration count 2, and 1 degree of parallelism","quote":"Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP recommends PBKDF2 with 600,000 iterations and HMAC-SHA-256 for FIPS-140 compliance","quote":"If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.","verified":true,"document":"owasp-password-storage"},{"url":"https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html","claim":"OWASP notes that PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations","quote":"Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.","verified":true,"document":"owasp-password-storage"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST SP 800-63B-4 requires passwords to be salted and hashed using a suitable password hashing scheme","quote":"Passwords SHALL be salted and hashed using a suitable password hashing scheme.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-4/sp800-63b.html","claim":"NIST SP 800-63B-4 allows approved schemes from SP800-132 or updated NIST guidelines","quote":"An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.","verified":true,"document":"nist-sp800-63b-4"},{"url":"https://pages.nist.gov/800-63-3/sp800-63b.html","claim":"NIST SP 800-63B-3 explicitly recommended memory-hard functions for password hashing","quote":"A memory-hard function SHOULD be used because it increases the cost of an attack.","verified":true,"document":"nist-sp800-63b-3"}]
judge unverified quotes
0
judge verdicts
[{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"met","support":["E4","E5"]},"nist-revision":{"status":"partial","support":["E6"]},"reconciliation":{"status":"met","support":["E2","E3","E5"]},"common-ground":{"status":"missing","support":[]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"met","support":["E4","E5"]},"nist-revision":{"status":"partial","support":["E6"]},"reconciliation":{"status":"met","support":["E2","E3","E5"]},"common-ground":{"status":"partial","support":["E4"]}},{"owasp-argon2id":{"status":"met","support":["E1"]},"owasp-conditions":{"status":"met","support":["E2"]},"nist-current":{"status":"met","support":["E4","E5"]},"nist-revision":{"status":"partial","support":["E6"]},"reconciliation":{"status":"met","support":["E2","E3","E5"]},"common-ground":{"status":"partial","support":[]}}]
judge failures
0
grounded weight share
0.875 (88%)

Final answer

Now I have comprehensive information from both sources. Let me write the complete answer.

Answer

A team choosing a password hashing algorithm must balance security best practices with federal compliance requirements. Here's what the current guidance from each source actually says:

OWASP Password Storage Guidance

OWASP's Password Storage Cheat Sheet provides a clear hierarchy of recommendations based on availability and compliance needs:

  1. Primary recommendation: Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism. This memory-hard algorithm is preferred because it resists both GPU-based and side-channel attacks.
  2. Acceptable alternative: If Argon2id is unavailable, use scrypt with a minimum CPU/memory cost parameter of (2^17), a minimum block size of 8 (1024 bytes), and a parallelization parameter of 1.
  3. Legacy systems: For older systems, bcrypt remains acceptable with a work factor of 10 or more, though with a 72-byte password limit.
  4. FIPS-140 compliance: Crucially, when FIPS-140 compliance is required, OWASP specifies using PBKDF2 with a work factor of 600,000 or more iterations, combined with HMAC-SHA-256 as the internal hash function. OWASP explicitly notes that "PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations."

NIST SP 800-63B-4 Requirements

The current revision (SP 800-63B-4) of NIST's Digital Identity Guidelines takes a more flexible but less prescriptive approach:

  • Passwords "SHALL be salted and hashed using a suitable password hashing scheme."
  • The document states that "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used."

NIST defines "approved cryptography" as algorithms that are either "Federal Information Processing Standards (FIPS)-approved or NIST-recommended." SP 800-132 (referenced by NIST) is the specification for PBKDF2.

Notably, NIST's earlier revision (SP 800-63B-3) explicitly stated that "A memory-hard function SHOULD be used because it increases the cost of an attack," mentioning PBKDF2 and Balloon as examples of suitable functions. This indicates NIST's openness to memory-hard algorithms beyond PBKDF2.

How to Reconcile the Two for FIPS Compliance

The reconciliation for teams needing FIPS compliance is straightforward:

  1. If FIPS-140 compliance is required: Both sources agree on PBKDF2. OWASP recommends 600,000+ iterations with HMAC-SHA-256, and NIST references SP 800-132, which defines PBKDF2. PBKDF2 has validated FIPS-140 implementations, making it the bridge between security best practices and federal requirements.
  2. If FIPS compliance is not required: OWASP's preference for Argon2id aligns with NIST's recognition that memory-hard functions "should be used" for increased attack cost. The "updated NIST guidelines on password hashing schemes" referenced in SP 800-63B-4 likely points toward future recognition of algorithms like Argon2, though these haven't been formally codified in a published NIST SP yet.
  3. Key insight: The apparent tension resolves because PBKDF2 is simultaneously a FIPS-approved algorithm (meeting the FIPS requirement) and acceptable under NIST guidelines (listed in SP 800-132 that NIST references). Teams requiring federal compliance can use PBKDF2 with appropriate work factors and satisfy both OWASP security principles and NIST requirements.

{"evidence": [
  {
    "claim": "OWASP recommends Argon2id with minimum 19 MiB of memory, iteration count 2, and 1 degree of parallelism",
    "quote": "Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.",
    "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"
  },
  {
    "claim": "OWASP recommends PBKDF2 with 600,000 iterations and HMAC-SHA-256 for FIPS-140 compliance",
    "quote": "If FIPS-140 compliance is required, use PBKDF2 with a work factor of 600,000 or more and set with an internal hash function of HMAC-SHA-256.",
    "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"
  },
  {
    "claim": "OWASP notes that PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations",
    "quote": "Since PBKDF2 is permitted by NIST SP 800-63B-4 and has FIPS-140 validated implementations, it should be the preferred algorithm when these are required.",
    "url": "https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html"
  },
  {
    "claim": "NIST SP 800-63B-4 requires passwords to be salted and hashed using a suitable password hashing scheme",
    "quote": "Passwords SHALL be salted and hashed using a suitable password hashing scheme.",
    "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"
  },
  {
    "claim": "NIST SP 800-63B-4 allows approved schemes from SP800-132 or updated NIST guidelines",
    "quote": "An approved password hashing scheme published in the latest revision of [SP800-132] or updated NIST guidelines on password hashing schemes SHOULD be used.",
    "url": "https://pages.nist.gov/800-63-4/sp800-63b.html"
  },
  {
    "claim": "NIST SP 800-63B-3 explicitly recommended memory-hard functions for password hashing",
    "quote": "A memory-hard function SHOULD be used because it increases the cost of an attack.",
    "url": "https://pages.nist.gov/800-63-3/sp800-63b.html"
  }
]}

Provenance of the numbers on this page

run set
scaffold-or-scale-v1
models
Haiku 4.5, Sonnet 5.5, Opus 5.5
effort
high
repeats
2
fixtures
replay
runs
900 / 900
strategy cost
$354.44
judge cost
$9.19
wall clock
9h 48m
git
fce5807b2d

Cost excludes judge calls, which are tracked separately so a scoring model never flatters or penalises an architecture. Every figure on this site is read from results/ at build time.