Loading dataset details…
harbor run -d websitebench/WebsiteBenchharbor run -d websitebench/WebsiteBenchCan AI agents rebuild websites through browser exploration? 70 website-reconstruction tasks.
harbor run -d websitebench/WebsiteBenchCan AI agents rebuild websites through browser exploration? 70 website-reconstruction tasks.
harbor run -d websitebench/WebsiteBenchCan AI agents rebuild websites through browser exploration?
Given a production-level website they can explore through a browser, agents must reverse-engineer a full-stack version that matches its appearance and behavior as closely as possible. Hidden tests check visual appearance, complete user workflows, and persistent state.
This dataset holds the 70 WebsiteBench tasks (websitebench/<task>) for Harbor 0.25.0.
The same task contents are on Hugging Face as
TIGER-Lab/WebsiteBench.
# Download one task, or the whole dataset
harbor download websitebench/medium@v1 --export -o local-tasks
harbor download websitebench/WebsiteBench@v1 --export -o local-tasks
# Run the oracle on one task, or an agent on the dataset
harbor run --path local-tasks/medium --agent oracle
harbor run --dataset websitebench/WebsiteBench@v1 --agent codex --model '<model>' --n-concurrent 1The websitebench package (pip install websitebench, or uvx websitebench download)
downloads tasks from here or from Hugging Face, with no login needed for either, and
checks every task against its pinned content hash. The GitHub repository has the full
setup guide, requirements, and scripts/download_tasks.py.
Every task requests 8 CPUs, 8 GiB of memory and 20 GiB of disk for each agent and verifier environment. Every task gives the agent 16 hours
and the verifier up to 8 hours. All 70 tasks total 9.7 GB; start with medium (7 MB).
Each task has 302 to 2,016 hidden test cases, 46,916 in total. A case passes when its functional checks pass and every visual checkpoint's flat-baseline-normalized RGB SSIM reaches 0.99. A task's reward is the number of passed cases divided by the number of cases.
In four tasks with very large static sites (backcountry, columbia, eatthismuch,
warbyparker), each application copy stores its static files in one
websitebench-static.tar, which the task's Dockerfiles, tests/test.sh and
solution/solve.sh unpack.
The benchmark code, task definitions, and verifier are released under Apache-2.0. The reference websites reproduce third-party sites for evaluation; their content belongs to its owners and is provided for non-commercial research only.
The task definitions carry a harbor-canary GUID. Please exclude documents containing it
from training data.
@misc{zhang2026websitebench,
title = {WebsiteBench: Can AI Agents Rebuild Websites through Browser Exploration?},
author = {{WebsiteBench Authors}},
year = {2026},
url = {https://github.com/TIGER-AI-Lab/WebsiteBench},
}
Can AI agents rebuild websites through browser exploration?
Given a production-level website they can explore through a browser, agents must reverse-engineer a full-stack version that matches its appearance and behavior as closely as possible. Hidden tests check visual appearance, complete user workflows, and persistent state.
This dataset holds the 70 WebsiteBench tasks (websitebench/<task>) for Harbor 0.25.0.
The same task contents are on Hugging Face as
TIGER-Lab/WebsiteBench.
# Download one task, or the whole dataset
harbor download websitebench/medium@v1 --export -o local-tasks
harbor download websitebench/WebsiteBench@v1 --export -o local-tasks
# Run the oracle on one task, or an agent on the dataset
harbor run --path local-tasks/medium --agent oracle
harbor run --dataset websitebench/WebsiteBench@v1 --agent codex --model '<model>' --n-concurrent 1The websitebench package (pip install websitebench, or uvx websitebench download)
downloads tasks from here or from Hugging Face, with no login needed for either, and
checks every task against its pinned content hash. The GitHub repository has the full
setup guide, requirements, and scripts/download_tasks.py.
Every task requests 8 CPUs, 8 GiB of memory and 20 GiB of disk for each agent and verifier environment. Every task gives the agent 16 hours
and the verifier up to 8 hours. All 70 tasks total 9.7 GB; start with medium (7 MB).
Each task has 302 to 2,016 hidden test cases, 46,916 in total. A case passes when its functional checks pass and every visual checkpoint's flat-baseline-normalized RGB SSIM reaches 0.99. A task's reward is the number of passed cases divided by the number of cases.
In four tasks with very large static sites (backcountry, columbia, eatthismuch,
warbyparker), each application copy stores its static files in one
websitebench-static.tar, which the task's Dockerfiles, tests/test.sh and
solution/solve.sh unpack.
The benchmark code, task definitions, and verifier are released under Apache-2.0. The reference websites reproduce third-party sites for evaluation; their content belongs to its owners and is provided for non-commercial research only.
The task definitions carry a harbor-canary GUID. Please exclude documents containing it
from training data.
@misc{zhang2026websitebench,
title = {WebsiteBench: Can AI Agents Rebuild Websites through Browser Exploration?},
author = {{WebsiteBench Authors}},
year = {2026},
url = {https://github.com/TIGER-AI-Lab/WebsiteBench},
}