CyberDefense Bench

CyberDefense Bench is a 100-task Harbor dataset for evaluating how well an agent can harden vulnerable local artifacts under demanding blue-team constraints.

Each task simulates an attacker-controlled or attacker-influenced component, then asks the evaluated agent to repair the defensive boundary. The verifier runs local attack probes plus legitimate behavior checks. The dataset does not require external targets or network access.

This version is intentionally hard: tasks include canonicalization bypasses, parser ambiguity, strict output contracts, schema validation, recursive data handling, duplicate detection, key/claim verification, and edge cases that simple single-line patches usually miss.

Task families:

• Path traversal hardening with encoded paths, symlinks, null bytes, mixed separators, and extension policy
• SQL injection prevention through strict parameterized query construction and deterministic query contracts
• Contextual XSS output encoding with safe-link policy
• Command injection prevention through safe execution plans, cwd/env contracts, and no-shell argv
• SSRF URL allowlisting with parser canonicalization and numeric-IP bypass checks
• Password storage hardening with versioned records, peppered PBKDF2, verification, and password policy
• JWT-like HMAC session token verification with kid, issuer/audience, and time-claim checks
• Unsafe parser replacement with duplicate JSON key rejection and policy schema validation
• Recursive secret redaction for structured and unstructured logs
• Archive member validation for traversal, symlink/hardlink/device, duplicate, size, and mode attacks
• CORS, redirect, webhook, upload, expression, template, cache, proxy-header, authorization, and browser-header hardening