Harbor Hub

terminal-bench/terminal-bench

Terminal-Bench is a benchmark for measuring agents' abilities to complete tasks using a terminal.

New Job
harbor run -d terminal-bench/terminal-bench

Terminal-Bench

Terminal-Bench is wide and dynamic. Terminal-Bench measures general agent capabilities on a diverse set of difficult tasks.

Terminal-Bench will be continuously updated with new and improved tasks.

Run

Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.

First, install the Harbor CLI.

uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"

And then run:

harbor run -d terminal-bench/terminal-bench \
  --agent claude-code \
  --model anthropic/claude-fable-5 \
  --n-concurrent 100 \
  --env modal

To upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".

Contribute

See CONTRIBUTING.md for how to propose, implement, and submit new tasks.

We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.

Task
terminal-bench/layout-config-recreation2
terminal-bench/photonic-waveguide-routing
terminal-bench/mp-checkpoint-consolidation
terminal-bench/biped-contact-dynamics
terminal-bench/ks-solver-cpp
terminal-bench/react-lead-form
terminal-bench/lake-temp-glm
terminal-bench/takens-embedding-lean
terminal-bench/satb-audio-transcription
terminal-bench/formal-crypto
terminal-bench/fin-saccr-rwa
terminal-bench/embedding-drift-monitor
terminal-bench/math-eval-grader
terminal-bench/pretrain-shard-corruption
terminal-bench/freight-dispatch-shift
terminal-bench/sound-change-cascade
terminal-bench/cad-model
terminal-bench/glycan-ms2-elucidation
terminal-bench/coq-block-bound
terminal-bench/cargo-flight-dispatch
terminal-bench/payments-pipeline-fix
terminal-bench/heat-pump-warranty
terminal-bench/foodstuff-beta-activity
terminal-bench/protein-autointerp-disulfide
terminal-bench/session-window-debug
terminal-bench/batched-eval-parity
terminal-bench/wal-recovery-ordering
terminal-bench/ctr-optimization
terminal-bench/bun-sourcemap-leak
terminal-bench/legacy-utility-triage
terminal-bench/mvcc-lsm-compaction
terminal-bench/risk-scorer-replay
terminal-bench/atrx-vep-crispr
terminal-bench/nextjs-performance
terminal-bench/live-database-cutover
terminal-bench/cumulative-layout-shift
terminal-bench/data-anonymization
terminal-bench/vba-userform-port
terminal-bench/freecad-impeller
terminal-bench/music-harmony
terminal-bench/telecom-entity-resolution
terminal-bench/distributed-dedup
terminal-bench/ontology-kg-querying
terminal-bench/production-planning
terminal-bench/freecad-platform-drawing
terminal-bench/vpp-loss-divergence
terminal-bench/gsea-proteomics
terminal-bench/intrastat-meldung
terminal-bench/kv-live-surgery
terminal-bench/html-js-filter
terminal-bench/interleaved-vigenere
terminal-bench/uefi-bootkit
terminal-bench/layout-config-recreation
terminal-bench/shadow-relay
terminal-bench/vllm-deepseek-streaming
terminal-bench/roy-polymorph-cn
terminal-bench/rs-archive-clone
terminal-bench/wdm-design
terminal-bench/freecad-spring-clip
terminal-bench/fp8-rmsnorm-gemm
terminal-bench/vf2-speedup-networkx
terminal-bench/jax-speedrun-gpu
terminal-bench/medical-claims-processing
terminal-bench/retro-console-soc
terminal-bench/sglang-qwen-burst
terminal-bench/hof-topology-interpenetration

Displaying 66 of 66 tasks