Loading dataset details…
harbor run -d ryanmarten/tb4-previewharbor run -d ryanmarten/tb4-previewTerminal-Bench is a benchmark for measuring agents' abilities to complete tasks using a terminal.
harbor run -d ryanmarten/tb4-previewTerminal-Bench is a benchmark for measuring agents' abilities to complete tasks using a terminal.
harbor run -d ryanmarten/tb4-previewTerminal-Bench is wide and dynamic. Terminal-Bench measures general agent capabilities on a diverse set of difficult tasks.
Terminal-Bench will be continuously updated with new and improved tasks.
Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.
First, install the Harbor CLI.
uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"And then run:
harbor run -d terminal-bench/terminal-bench \
--agent claude-code \
--model anthropic/claude-fable-5 \
--n-concurrent 100 \
--env modalTo upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".
See CONTRIBUTING.md for how to propose, implement, and submit new tasks.
We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.
Terminal-Bench is wide and dynamic. Terminal-Bench measures general agent capabilities on a diverse set of difficult tasks.
Terminal-Bench will be continuously updated with new and improved tasks.
Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.
First, install the Harbor CLI.
uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"And then run:
harbor run -d terminal-bench/terminal-bench \
--agent claude-code \
--model anthropic/claude-fable-5 \
--n-concurrent 100 \
--env modalTo upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".
See CONTRIBUTING.md for how to propose, implement, and submit new tasks.
We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.