Terminal Bench 4 No Prebuild

Terminal Bench 4 No Prebuild is a Terminal-Bench 4.0.0 variant published from the v4.0.0 GitHub release source without prebuilt Docker image pins.

Use this package when runs should build task environments from source instead of pulling the prebuilt Terminal-Bench release images.

Run

Terminal-Bench includes GPU and multi-container tasks. We recommend running on Modal or Daytona.

First, install the Harbor CLI.

uv tool install "harbor[modal,daytona]"
# or pip install "harbor[modal,daytona]"

And then run:

harbor run -d ryanmarten/terminal-bench-4-no-prebuild \
  --agent claude-code \
  --model anthropic/claude-fable-5 \
  --n-concurrent 100 \
  --env modal

To upload results to Harbor Hub (e.g. for leaderboard submission), include the --upload flag or run harbor upload "<job/path>".

Contribute

See CONTRIBUTING.md for how to propose, implement, and submit new tasks.

We also encourage the community to reach out when they find bugs in tasks. Building benchmarks for current model capabilities is hard — community feedback helps keep quality high.