blobfishai/devopsbench-100
100 deterministic long-horizon DevOps/SRE agent tasks over an executable NovaCart world with 32-67 call reference trajectories and state-diff vcode verifiers.
harbor run -d blobfishai/devopsbench-100| Task |
|---|
blobfishai/dob100-008-detect-inventory |
blobfishai/dob100-073-catalog-batch-pricing |
blobfishai/dob100-068-gated-rotate-partner-credential |
blobfishai/dob100-052-express-checkout |
blobfishai/dob100-013-localize-analytics-errors |
blobfishai/dob100-100-ws-ledger-missing-account |
blobfishai/dob100-002-rca-analytics-revoked-grant |
blobfishai/dob100-030-rcn-distinct-checkout-bugs |
blobfishai/dob100-017-auth-v1-to-v2 |
blobfishai/dob100-011-detect-storefront-healthy |
blobfishai/dob100-040-port-dedupe-linked-issues |
blobfishai/dob100-046-inventory-pool |
blobfishai/dob100-005-rca-notifications-node-deadlock |
blobfishai/dob100-058-flaky-analytics-rollup |
blobfishai/dob100-021-rca-inventory-pool |
blobfishai/dob100-074-catalog-cache-ttl |
blobfishai/dob100-081-backorders |
blobfishai/dob100-042-port-high-priority-since |
blobfishai/dob100-080-port-report-customer-reports |
blobfishai/dob100-033-impl-chunk |
blobfishai/dob100-084-media-pipeline |
blobfishai/dob100-066-hz-checkout-flag-kill |
blobfishai/dob100-047-notifications-timeout |
blobfishai/dob100-026-rca-storefront-traffic-flood |
blobfishai/dob100-099-port-count-gateway-surface |
blobfishai/dob100-096-cve-requests |
blobfishai/dob100-067-hz-gateway-leak-rollback |
blobfishai/dob100-090-rcn-gateway-owner |
blobfishai/dob100-019-rca-catalog-n-plus-one |
blobfishai/dob100-014-localize-checkout-latency |
blobfishai/dob100-007-detect-checkout-errors |
blobfishai/dob100-093-checkout-hardcoded-secret |
blobfishai/dob100-001-rca-analytics-egress-blocked |
blobfishai/dob100-053-instant-refunds-killswitch |
blobfishai/dob100-086-search-relevance |
blobfishai/dob100-070-judge-checkout-flag |
blobfishai/dob100-060-flaky-checkout-idempotency |
blobfishai/dob100-088-rcn-alerts-for-incident |
blobfishai/dob100-054-legacy-price-rounding-cleanup |
blobfishai/dob100-006-rca-search-unscheduled-reindex |
blobfishai/dob100-022-rca-inventory-migrator-security-context |
blobfishai/dob100-027-rca-analytics-untolerated-taint |
blobfishai/dob100-009-rcn-checkout-error-rate |
blobfishai/dob100-097-cve-stripe-sdk |
blobfishai/dob100-079-port-collect-open-issues |
blobfishai/dob100-048-port-count-open-priority |
blobfishai/dob100-078-media-cdn |
blobfishai/dob100-025-rca-catalog-gc-thrash |
blobfishai/dob100-045-checkout-payments-timeout |
blobfishai/dob100-077-gateway-v510-rollback |
blobfishai/dob100-036-port-close-blocked-issues |
blobfishai/dob100-075-catalog-pricing-from-code |
blobfishai/dob100-082-checkout-upsell |
blobfishai/dob100-069-judge-analytics-oom |
blobfishai/dob100-031-impl-backoff |
blobfishai/dob100-028-rca-media-recreate-strategy |
blobfishai/dob100-020-rca-notifications-timeout |
blobfishai/dob100-010-detect-status-page-recurrence |
blobfishai/dob100-098-retire-debug-endpoint |
blobfishai/dob100-015-localize-gateway-latency |
blobfishai/dob100-076-gateway-pool-reuse |
blobfishai/dob100-032-impl-cachekey |
blobfishai/dob100-037-port-copy-priority-issues |
blobfishai/dob100-051-checkout-v2-layout-cleanup |
blobfishai/dob100-004-rca-media-disk-pressure |
blobfishai/dob100-083-loyalty-points |
blobfishai/dob100-059-flaky-catalog-rounding |
blobfishai/dob100-043-analytics-batch-size |
blobfishai/dob100-044-analytics-prefetch |
blobfishai/dob100-041-port-escalate-to-gateway-oncall |
blobfishai/dob100-085-notification-templates |
blobfishai/dob100-087-split-settlement |
blobfishai/dob100-049-payments-retry |
blobfishai/dob100-039-port-count-unowned-work |
blobfishai/dob100-018-rca-payments-retry |
blobfishai/dob100-024-attr-three-at-once |
blobfishai/dob100-038-port-count-inflight-work |
blobfishai/dob100-055-media-webp |
blobfishai/dob100-089-rcn-customer-facing-incidents |
blobfishai/dob100-056-saved-carts |
blobfishai/dob100-072-judge-payments-retry |
blobfishai/dob100-071-judge-gateway-latency |
blobfishai/dob100-063-flaky-search-index |
blobfishai/dob100-029-rca-inventory-unbound-storage |
blobfishai/dob100-091-rcn-production-deploys |
blobfishai/dob100-092-rcn-running-version |
blobfishai/dob100-064-hand-cluster-runbook |
blobfishai/dob100-003-rca-checkout-unschedulable-replicas |
blobfishai/dob100-061-flaky-gateway-timeout |
blobfishai/dob100-065-hand-gateway-runbook |
blobfishai/dob100-023-rca-inventory-missing-role |
blobfishai/dob100-034-impl-ratelimit |
blobfishai/dob100-057-search-autocomplete |
blobfishai/dob100-035-port-close-backlog-issues |
blobfishai/dob100-012-localize-analytics-crashloop |
blobfishai/dob100-094-cve-libpayproc |
blobfishai/dob100-016-localize-media-latency |
blobfishai/dob100-095-cve-pydantic |
blobfishai/dob100-050-payments-retry-from-code |
blobfishai/dob100-062-w6-copy-13 |
Displaying 100 of 100 tasks