Datasets
| Dataset | Tasks |
|---|---|
| Dataset | Tasks |
|---|---|
| Dataset | Tasks |
|---|---|
Harbor Index is an 82-task benchmark for agentic evaluation. It was distilled from more than 6,000 candidate tasks through repeated model runs, automated broken-task identification, human audits, and reward hacking supervision.
harbor run -d harbor-index/harbor-index-1.0| Task |
|---|
harbor-index/bix-ordinal-logit-covid |
harbor-index/gso-speedup-numpy-strings |
harbor-index/arcagi2-grid-transform-a32d |
harbor-index/bix-diff-expr-mirna |
harbor-index/usaco-assign-cows-to-barns |
harbor-index/swebenchpro-improve-process-signal-messages |
harbor-index/hle-vowel-marking-system |
harbor-index/hle-interval-coverage-bound |
harbor-index/qcircuitbench-design-ternary-simon-circuit-n3 |
harbor-index/gpqadiamond-cope-rearrangement-products |
harbor-index/scicode-gaussian-beam-through-lens |
harbor-index/swesmith-fix-oauth1-header-params |
harbor-index/hle-identify-ingvar-runestone |
harbor-index/dacode-predict-essay-scores |
harbor-index/arcagi2-grid-transform-8b7b |
harbor-index/swebenchverified-fix-span-selector-axes-limits |
harbor-index/labbench-count-deg-in-pathway |
harbor-index/labbench-highest-auc-neuron-set |
harbor-index/omnimath-find-perfect-square-functions |
harbor-index/algotune-optimize-matrix-sqrt |
harbor-index/scicode-dna-binding-site-scanner |
harbor-index/cybergym-arvo-368 |
harbor-index/algotune-simplex-projection-speedup |
harbor-index/bix-filter-chip-variants |
harbor-index/arcagi2-grid-transform-faa9 |
harbor-index/omnimath-maximize-gf-sum |
harbor-index/gso-speedup-pillow-filter |
harbor-index/labbench-read-asap2f-step-response |
harbor-index/gso-speedup-pandas-merge |
harbor-index/swebenchverified-fix-annotate-xy-array-copy |
harbor-index/hle-name-alkaloid-compound |
harbor-index/gso-speedup-pandas-period-fmt |
harbor-index/gaia2-timed-1 |
harbor-index/widesearch-list-bri-projects-2025 |
harbor-index/hle-shock-wave-density-profile |
harbor-index/tb-dna-insert |
harbor-index/algotune-optimize-lti-sim |
harbor-index/skillsbench-ocr-receipts-to-excel |
harbor-index/swebenchverified-fix-django-union-queryset-order |
harbor-index/replicationbench-find-galactic-vz-peaks |
harbor-index/arcagi2-grid-transform-de80 |
harbor-index/featurebench-add-feature-xarray-backend-chunks |
harbor-index/bix-immune-pathway-enrichment |
harbor-index/swebenchpro-fix-file-suffix-chooser |
harbor-index/sldbench-discover-vocab-scaling-law |
harbor-index/hle-dirac-fermion-tunneling |
harbor-index/swelancer-fix-ios-keyboard-reopen |
harbor-index/spider2-dbt-airport-arrivals |
harbor-index/featurebench-add-feature-mlflow-bedrock-autolog |
harbor-index/algotune-optimize-outer-product |
harbor-index/gaia2-timed-2 |
harbor-index/gaia2-adapt-hard-2 |
harbor-index/featurebench-add-feature-mlflow-unity-catalog |
harbor-index/gso-speedup-hf-datasets |
harbor-index/codepde-solve-navier-stokes-1d |
harbor-index/bix-cpg-density-jackdaw |
harbor-index/tb-train-fasttext |
harbor-index/gso-speedup-pydantic-enum |
harbor-index/gaia-compare-sciencedirect-domains |
harbor-index/gso-speedup-pandas-seq-to-range |
harbor-index/scicode-tetrahedron-dos-integral |
harbor-index/featurebench-add-feature-lightning-hooks |
harbor-index/skillsbench-model-investment-shock-gdp |
harbor-index/labbench-habenula-fluorescence-change |
harbor-index/swebenchpro-fix-teleport-k8s-session-path |
harbor-index/algotune-optimize-ode-seirs |
harbor-index/swebenchpro-fix-teleport-mtls-ca-limit |
harbor-index/gaia2-adapt-hard-1 |
harbor-index/swebenchverified-fix-sphinx-literal-nitpick |
harbor-index/tb-make-doom-for-mips |
harbor-index/build-word2vec-pipeline |
harbor-index/spider2-dbt-twilio-messaging |
harbor-index/gaia2-ambiguous |
harbor-index/hle-identify-city-from-photo |
harbor-index/swtbenchverified-test-runserver-zero-address |
harbor-index/gaia-find-chess-winning-move |
harbor-index/arcagi2-grid-transform-88e3 |
harbor-index/swebenchverified-fix-django-mti-parent-link |
harbor-index/hle-fibered-category-schemes |
harbor-index/cybergym-ossfuzz-42m |
harbor-index/gaia-find-flavor-graveyard-rhyme |
harbor-index/swelancer-fix-fab-menu-enter-key |
Displaying 82 of 82 tasks
Harbor Index is an 82-task benchmark for agentic evaluation. It was distilled from more than 6,000 candidate tasks through repeated model runs, automated broken-task identification, human audits, and reward hacking supervision.
harbor run -d harbor-index/harbor-index-1.0| Task |
|---|
harbor-index/bix-ordinal-logit-covid |
harbor-index/gso-speedup-numpy-strings |
harbor-index/arcagi2-grid-transform-a32d |
harbor-index/bix-diff-expr-mirna |
harbor-index/usaco-assign-cows-to-barns |
harbor-index/swebenchpro-improve-process-signal-messages |
harbor-index/hle-vowel-marking-system |
harbor-index/hle-interval-coverage-bound |
harbor-index/qcircuitbench-design-ternary-simon-circuit-n3 |
harbor-index/gpqadiamond-cope-rearrangement-products |
harbor-index/scicode-gaussian-beam-through-lens |
harbor-index/swesmith-fix-oauth1-header-params |
harbor-index/hle-identify-ingvar-runestone |
harbor-index/dacode-predict-essay-scores |
harbor-index/arcagi2-grid-transform-8b7b |
harbor-index/swebenchverified-fix-span-selector-axes-limits |
harbor-index/labbench-count-deg-in-pathway |
harbor-index/labbench-highest-auc-neuron-set |
harbor-index/omnimath-find-perfect-square-functions |
harbor-index/algotune-optimize-matrix-sqrt |
harbor-index/scicode-dna-binding-site-scanner |
harbor-index/cybergym-arvo-368 |
harbor-index/algotune-simplex-projection-speedup |
harbor-index/bix-filter-chip-variants |
harbor-index/arcagi2-grid-transform-faa9 |
harbor-index/omnimath-maximize-gf-sum |
harbor-index/gso-speedup-pillow-filter |
harbor-index/labbench-read-asap2f-step-response |
harbor-index/gso-speedup-pandas-merge |
harbor-index/swebenchverified-fix-annotate-xy-array-copy |
harbor-index/hle-name-alkaloid-compound |
harbor-index/gso-speedup-pandas-period-fmt |
harbor-index/gaia2-timed-1 |
harbor-index/widesearch-list-bri-projects-2025 |
harbor-index/hle-shock-wave-density-profile |
harbor-index/tb-dna-insert |
harbor-index/algotune-optimize-lti-sim |
harbor-index/skillsbench-ocr-receipts-to-excel |
harbor-index/swebenchverified-fix-django-union-queryset-order |
harbor-index/replicationbench-find-galactic-vz-peaks |
harbor-index/arcagi2-grid-transform-de80 |
harbor-index/featurebench-add-feature-xarray-backend-chunks |
harbor-index/bix-immune-pathway-enrichment |
harbor-index/swebenchpro-fix-file-suffix-chooser |
harbor-index/sldbench-discover-vocab-scaling-law |
harbor-index/hle-dirac-fermion-tunneling |
harbor-index/swelancer-fix-ios-keyboard-reopen |
harbor-index/spider2-dbt-airport-arrivals |
harbor-index/featurebench-add-feature-mlflow-bedrock-autolog |
harbor-index/algotune-optimize-outer-product |
harbor-index/gaia2-timed-2 |
harbor-index/gaia2-adapt-hard-2 |
harbor-index/featurebench-add-feature-mlflow-unity-catalog |
harbor-index/gso-speedup-hf-datasets |
harbor-index/codepde-solve-navier-stokes-1d |
harbor-index/bix-cpg-density-jackdaw |
harbor-index/tb-train-fasttext |
harbor-index/gso-speedup-pydantic-enum |
harbor-index/gaia-compare-sciencedirect-domains |
harbor-index/gso-speedup-pandas-seq-to-range |
harbor-index/scicode-tetrahedron-dos-integral |
harbor-index/featurebench-add-feature-lightning-hooks |
harbor-index/skillsbench-model-investment-shock-gdp |
harbor-index/labbench-habenula-fluorescence-change |
harbor-index/swebenchpro-fix-teleport-k8s-session-path |
harbor-index/algotune-optimize-ode-seirs |
harbor-index/swebenchpro-fix-teleport-mtls-ca-limit |
harbor-index/gaia2-adapt-hard-1 |
harbor-index/swebenchverified-fix-sphinx-literal-nitpick |
harbor-index/tb-make-doom-for-mips |
harbor-index/build-word2vec-pipeline |
harbor-index/spider2-dbt-twilio-messaging |
harbor-index/gaia2-ambiguous |
harbor-index/hle-identify-city-from-photo |
harbor-index/swtbenchverified-test-runserver-zero-address |
harbor-index/gaia-find-chess-winning-move |
harbor-index/arcagi2-grid-transform-88e3 |
harbor-index/swebenchverified-fix-django-mti-parent-link |
harbor-index/hle-fibered-category-schemes |
harbor-index/cybergym-ossfuzz-42m |
harbor-index/gaia-find-flavor-graveyard-rhyme |
harbor-index/swelancer-fix-fab-menu-enter-key |
Displaying 82 of 82 tasks