harbor run -d servicenow-ai-org/enterpriseops-gym-v1-1harbor run -d servicenow-ai-org/enterpriseops-gym-v1-1296 stateful enterprise workflow tasks.
harbor run -d servicenow-ai-org/enterpriseops-gym-v1-1296 stateful enterprise workflow tasks.
harbor run -d servicenow-ai-org/enterpriseops-gym-v1-1This dataset contains 296 stateful enterprise workflow tasks: 92 CSM, 101 HR, and 103 ITSM. Each task starts an isolated, seeded application, exposes its domain's complete MCP tool list, and grades the resulting database state with deterministic SQL checks. No judge model or source checkout is required.
linux/amd64 images.WORK="$HOME/eops-public"
mkdir -p "$WORK"
harbor download \
'servicenow-ai-org/enterpriseops-gym-v1-1@1.0.0' \
--output-dir "$WORK"
TASK="$WORK/enterpriseops-gym-v1-1/eops11-public-enterpriseops-001_csm__task_20251125_152842_063_b393a327_9622ac64"
harbor run --path "$TASK" --agent nop --env docker \
--n-concurrent 1 --jobs-dir "$WORK/jobs" --job-name smoke-csm
The no-op agent makes no changes; a reward of zero is expected. Confirm one
graded trial and zero trial errors in $WORK/jobs/smoke-csm/result.json.
The same downloaded directory contains all 296 tasks.
Provide a supported model key in a restricted env file, then use an MCP-capable
agent. For example, with OPENAI_API_KEY in $WORK/model.env:
harbor run --path "$TASK" --agent codex --model openai/YOUR_MODEL_ID \
--env docker --env-file "$WORK/model.env" --n-concurrent 1 \
--jobs-dir "$WORK/jobs" --job-name model-csm
Start with one task. The task's instruction.md holds the request, the MCP
gateway limits agent access to the intended tools, and the verifier records
fractional reward in the trial result. Increase concurrency only after
checking image pulls, setup, tool calls, grading, and Docker network capacity.
This dataset contains 296 stateful enterprise workflow tasks: 92 CSM, 101 HR, and 103 ITSM. Each task starts an isolated, seeded application, exposes its domain's complete MCP tool list, and grades the resulting database state with deterministic SQL checks. No judge model or source checkout is required.
linux/amd64 images.WORK="$HOME/eops-public"
mkdir -p "$WORK"
harbor download \
'servicenow-ai-org/enterpriseops-gym-v1-1@1.0.0' \
--output-dir "$WORK"
TASK="$WORK/enterpriseops-gym-v1-1/eops11-public-enterpriseops-001_csm__task_20251125_152842_063_b393a327_9622ac64"
harbor run --path "$TASK" --agent nop --env docker \
--n-concurrent 1 --jobs-dir "$WORK/jobs" --job-name smoke-csm
The no-op agent makes no changes; a reward of zero is expected. Confirm one
graded trial and zero trial errors in $WORK/jobs/smoke-csm/result.json.
The same downloaded directory contains all 296 tasks.
Provide a supported model key in a restricted env file, then use an MCP-capable
agent. For example, with OPENAI_API_KEY in $WORK/model.env:
harbor run --path "$TASK" --agent codex --model openai/YOUR_MODEL_ID \
--env docker --env-file "$WORK/model.env" --n-concurrent 1 \
--jobs-dir "$WORK/jobs" --job-name model-csm
Start with one task. The task's instruction.md holds the request, the MCP
gateway limits agent access to the intended tools, and the verifier records
fractional reward in the trial result. Increase concurrency only after
checking image pulls, setup, tool calls, grading, and Docker network capacity.