Datasets
| Dataset | Tasks |
|---|---|
| Dataset | Tasks |
|---|---|
| Dataset | Tasks |
|---|---|
Senior SWE-Bench (v2026.06) public dataset
harbor run -d snorkel-ai/senior-swe-bench-v2026.06This repo contains the Harbor dataset for the public tasks in Senior SWE-Bench (v2026.06). For more, check out:
Harbor can read directly from this repo, so you can get started without downloading anything. Once you install Harbor, you can run:
# Set env vars required for your model
# export ANTHROPIC_API_KEY=sk-ant-...
# export OPENAI_API_KEY=sk-proj-...
# export MY_PROVIDERS_API_KEY=...
# [Optional] Set the models for the test stage (defaults below)
# export SSB_OVERRIDE_VA_HARNESS=mini-swe-agent
# export SSB_OVERRIDE_VA_MODEL=anthropic/claude-sonnet-4-6
# export SSB_OVERRIDE_ALL_JUDGE_MODEL=anthropic/claude-sonnet-4-6
# export SSB_OVERRIDE_CLASSIFIER_MODEL=anthropic/claude-haiku-4-5
# Set depending on what you want to run
MODEL=anthropic/claude-opus-4-8
AGENT=mini-swe-agent
# Run Harbor
# Option 1: via Harbor Hub
harbor run -d snorkel-ai/senior-swe-bench-v2026.06 -a $AGENT -m $MODEL
# Option 2: via GitHub repo
harbor run --repo snorkel-ai/senior-swe-bench-v2026.06 -a $AGENT -m $MODELSenior SWE-Bench (v2026.06) public dataset
harbor run -d snorkel-ai/senior-swe-bench-v2026.06This repo contains the Harbor dataset for the public tasks in Senior SWE-Bench (v2026.06). For more, check out:
Harbor can read directly from this repo, so you can get started without downloading anything. Once you install Harbor, you can run:
# Set env vars required for your model
# export ANTHROPIC_API_KEY=sk-ant-...
# export OPENAI_API_KEY=sk-proj-...
# export MY_PROVIDERS_API_KEY=...
# [Optional] Set the models for the test stage (defaults below)
# export SSB_OVERRIDE_VA_HARNESS=mini-swe-agent
# export SSB_OVERRIDE_VA_MODEL=anthropic/claude-sonnet-4-6
# export SSB_OVERRIDE_ALL_JUDGE_MODEL=anthropic/claude-sonnet-4-6
# export SSB_OVERRIDE_CLASSIFIER_MODEL=anthropic/claude-haiku-4-5
# Set depending on what you want to run
MODEL=anthropic/claude-opus-4-8
AGENT=mini-swe-agent
# Run Harbor
# Option 1: via Harbor Hub
harbor run -d snorkel-ai/senior-swe-bench-v2026.06 -a $AGENT -m $MODEL
# Option 2: via GitHub repo
harbor run --repo snorkel-ai/senior-swe-bench-v2026.06 -a $AGENT -m $MODEL| Task |
|---|
snorkel-ai/electric-fix-resolve-pending-shapes |
snorkel-ai/posthog-feat-schema-ingestion-block |
snorkel-ai/paperless-ngx-perf-workflow-queries |
snorkel-ai/gitea-fix-force-push-timeline |
snorkel-ai/firezone-feat-portal-add-recent |
snorkel-ai/gitea-feat-fast-forward-only |
snorkel-ai/electric-feat-sync-service-start |
snorkel-ai/gitea-add-project-column-picker |
snorkel-ai/prefect-fix-subflow-cancellation |
snorkel-ai/gitea-fix-diff-highlight-overlap |
snorkel-ai/paperless-ngx-perf-document-counts |
snorkel-ai/turborepo-perf-reuse-input-hashes |
snorkel-ai/immich-feat-recently-added-assets |
snorkel-ai/immich-fix-server-live-photo |
snorkel-ai/posthog-feat-subscriptions-auto-disable |
snorkel-ai/better-auth-fix-api-return-response |
snorkel-ai/prefect-fix-task-run-recorder |
snorkel-ai/posthog-fix-replay-buffering |
snorkel-ai/posthog-feat-llma-enable-tagger |
snorkel-ai/gitea-refactor-auth-middleware |
snorkel-ai/immich-feat-release-candidate-detection |
snorkel-ai/plausible-feat-shared-dashboard-deeplink |
snorkel-ai/posthog-feat-approval-gating |
snorkel-ai/harbor-add-agent-file-retention |
snorkel-ai/harbor-refactor-optional-sandbox-deps |
snorkel-ai/plausible-fix-cross-site-resource-attach |
snorkel-ai/plausible-fix-top-pages-comparison |
snorkel-ai/posthog-feat-personhog-writer-add |
snorkel-ai/prefect-fix-resolve-race-condition |
snorkel-ai/posthog-feat-prompt-versioning |
snorkel-ai/electric-perf-array-filter-eval |
snorkel-ai/teleport-fix-scoped-kube-clusters |
snorkel-ai/electric-fix-elixir-client-cache |
snorkel-ai/harbor-add-windows-tasks-support |
snorkel-ai/firezone-fix-connlib-align-device |
snorkel-ai/turborepo-feat-add-circular-package |
snorkel-ai/paperless-ngx-refactor-task-system |
snorkel-ai/posthog-fix-llm-gateway-add |
snorkel-ai/electric-fix-classify-admission-control |
snorkel-ai/better-auth-fix-oauth-provider-return |
snorkel-ai/harbor-add-multi-step-tasks |
snorkel-ai/electric-feat-add-variadic-function |
snorkel-ai/gitea-fix-codeql-code-scanning |
snorkel-ai/turborepo-fix-prune-missing-sources |
snorkel-ai/prefect-feat-dbt-per-node-hardening |
snorkel-ai/teleport-add-traits-matching-logic |
snorkel-ai/better-auth-fix-api-key-run |
snorkel-ai/better-auth-fix-resolve-dynamic-baseurl |
snorkel-ai/turborepo-fix-preserve-package-json |
snorkel-ai/paperless-ngx-feat-saved-view-sharing |
Displaying 50 of 50 tasks
| Task |
|---|
snorkel-ai/electric-fix-resolve-pending-shapes |
snorkel-ai/posthog-feat-schema-ingestion-block |
snorkel-ai/paperless-ngx-perf-workflow-queries |
snorkel-ai/gitea-fix-force-push-timeline |
snorkel-ai/firezone-feat-portal-add-recent |
snorkel-ai/gitea-feat-fast-forward-only |
snorkel-ai/electric-feat-sync-service-start |
snorkel-ai/gitea-add-project-column-picker |
snorkel-ai/prefect-fix-subflow-cancellation |
snorkel-ai/gitea-fix-diff-highlight-overlap |
snorkel-ai/paperless-ngx-perf-document-counts |
snorkel-ai/turborepo-perf-reuse-input-hashes |
snorkel-ai/immich-feat-recently-added-assets |
snorkel-ai/immich-fix-server-live-photo |
snorkel-ai/posthog-feat-subscriptions-auto-disable |
snorkel-ai/better-auth-fix-api-return-response |
snorkel-ai/prefect-fix-task-run-recorder |
snorkel-ai/posthog-fix-replay-buffering |
snorkel-ai/posthog-feat-llma-enable-tagger |
snorkel-ai/gitea-refactor-auth-middleware |
snorkel-ai/immich-feat-release-candidate-detection |
snorkel-ai/plausible-feat-shared-dashboard-deeplink |
snorkel-ai/posthog-feat-approval-gating |
snorkel-ai/harbor-add-agent-file-retention |
snorkel-ai/harbor-refactor-optional-sandbox-deps |
snorkel-ai/plausible-fix-cross-site-resource-attach |
snorkel-ai/plausible-fix-top-pages-comparison |
snorkel-ai/posthog-feat-personhog-writer-add |
snorkel-ai/prefect-fix-resolve-race-condition |
snorkel-ai/posthog-feat-prompt-versioning |
snorkel-ai/electric-perf-array-filter-eval |
snorkel-ai/teleport-fix-scoped-kube-clusters |
snorkel-ai/electric-fix-elixir-client-cache |
snorkel-ai/harbor-add-windows-tasks-support |
snorkel-ai/firezone-fix-connlib-align-device |
snorkel-ai/turborepo-feat-add-circular-package |
snorkel-ai/paperless-ngx-refactor-task-system |
snorkel-ai/posthog-fix-llm-gateway-add |
snorkel-ai/electric-fix-classify-admission-control |
snorkel-ai/better-auth-fix-oauth-provider-return |
snorkel-ai/harbor-add-multi-step-tasks |
snorkel-ai/electric-feat-add-variadic-function |
snorkel-ai/gitea-fix-codeql-code-scanning |
snorkel-ai/turborepo-fix-prune-missing-sources |
snorkel-ai/prefect-feat-dbt-per-node-hardening |
snorkel-ai/teleport-add-traits-matching-logic |
snorkel-ai/better-auth-fix-api-key-run |
snorkel-ai/better-auth-fix-resolve-dynamic-baseurl |
snorkel-ai/turborepo-fix-preserve-package-json |
snorkel-ai/paperless-ngx-feat-saved-view-sharing |
Displaying 50 of 50 tasks