Loading dataset details…
harbor run -d hack-verifiable-environments/hv-terminal-bench-2-1harbor run -d hack-verifiable-environments/hv-terminal-bench-2-1Hack-verifiable version of terminal-bench-2-1. Each task includes hidden solution and test files; inotifywait detects whether agents access them instead of solving legitimately.
harbor run -d hack-verifiable-environments/hv-terminal-bench-2-1Hack-verifiable version of terminal-bench-2-1. Each task includes hidden solution and test files; inotifywait detects whether agents access them instead of solving legitimately.
harbor run -d hack-verifiable-environments/hv-terminal-bench-2-1A hack-verifiable version of Terminal-Bench 2.1. Every task is instrumented so that reward hacking is measured, not inferred — alongside the usual pass/fail signal, each trial reports whether the agent read the hidden solution, read the test files, or modified them.
📝 Blog post: Hack-Verifiable Environments
All agent traces from the paper are public on the Harbor hub. There is currently an issue with Harbor where the jobs view does not list them correctly, so you can browse the jobs directly through the table below.
| Model | Ablation | L0 | L1 | L2 | L3 |
|---|---|---|---|---|---|
| gpt-5.6-sol | link | link | link | link | link |
| glm-5.2 | link | link | link | link | link |
| kimi-k3 | link | link | link | link | link |
| claude-opus-5 | link | link | link | link | link |
| gemini-3.1-pro | link | link | link | link | link |
A hack-verifiable version of Terminal-Bench 2.1. Every task is instrumented so that reward hacking is measured, not inferred — alongside the usual pass/fail signal, each trial reports whether the agent read the hidden solution, read the test files, or modified them.
📝 Blog post: Hack-Verifiable Environments
All agent traces from the paper are public on the Harbor hub. There is currently an issue with Harbor where the jobs view does not list them correctly, so you can browse the jobs directly through the table below.
| Model | Ablation | L0 | L1 | L2 | L3 |
|---|---|---|---|---|---|
| gpt-5.6-sol | link | link | link | link | link |
| glm-5.2 | link | link | link | link | link |
| kimi-k3 | link | link | link | link | link |
| claude-opus-5 | link | link | link | link | link |
| gemini-3.1-pro | link | link | link | link | link |