Terminal-Bench

Terminal-Bench is wide and dynamic. Terminal-Bench measures general agent capabilities on a diverse set of difficult tasks.

Terminal-Bench will be continuously updated with new and improved tasks.

Run

Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.

First, install the Harbor CLI.

uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"

And then run:

harbor run -d terminal-bench/terminal-bench \
  --agent claude-code \
  --model anthropic/claude-fable-5 \
  --n-concurrent 100 \
  --env modal

To upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".

Contribute

See CONTRIBUTING.md for how to propose, implement, and submit new tasks.

We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.

Terminal-Bench

Terminal-Bench is wide and dynamic. Terminal-Bench measures general agent capabilities on a diverse set of difficult tasks.

Terminal-Bench will be continuously updated with new and improved tasks.

Run

Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.

First, install the Harbor CLI.

uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"

And then run:

harbor run -d terminal-bench/terminal-bench \
  --agent claude-code \
  --model anthropic/claude-fable-5 \
  --n-concurrent 100 \
  --env modal

To upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".

Contribute

See CONTRIBUTING.md for how to propose, implement, and submit new tasks.

We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.

Terminal-Bench

Terminal-Bench is wide and dynamic. Terminal-Bench measures general agent capabilities on a diverse set of difficult tasks.

Terminal-Bench will be continuously updated with new and improved tasks.

Run

Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.

First, install the Harbor CLI.

uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"

And then run:

harbor run -d terminal-bench/terminal-bench \
  --agent claude-code \
  --model anthropic/claude-fable-5 \
  --n-concurrent 100 \
  --env modal

To upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".

Contribute

See CONTRIBUTING.md for how to propose, implement, and submit new tasks.

We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.

Terminal-Bench

Terminal-Bench is wide and dynamic. Terminal-Bench measures general agent capabilities on a diverse set of difficult tasks.

Terminal-Bench will be continuously updated with new and improved tasks.

Run

Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.

First, install the Harbor CLI.

uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"

And then run:

harbor run -d terminal-bench/terminal-bench \
  --agent claude-code \
  --model anthropic/claude-fable-5 \
  --n-concurrent 100 \
  --env modal

To upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".

Contribute

See CONTRIBUTING.md for how to propose, implement, and submit new tasks.

We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.