Organization collection

orca-bench

Datasets published by this organization.

⌘K

Organization inventory

Private datasets are available only to members of this organization.

Dataset	AccessVisibility	Tasks
orca-bench/orca-bench An agent benchmark for root cause analysis — public split of 755 tasks across 77 incidents (245 easy, 264 medium, 246 hard). For the results on all 1,079 tasks, please see https://arxiv.org/abs/2607.28545.	Public	755
orca-bench/orca-bench-verified An agent benchmark for root cause analysis — verified split of 40 tasks across 40 incidents (10 easy, 15 medium, 15 hard). For the results on all 1,079 tasks, please see https://arxiv.org/abs/2607.28545.	Public	40
orca-bench/orca-bench-private An agent benchmark for root cause analysis — held-out private (answers removed) split of 324 tasks across 31 incidents (108 easy, 117 medium, 99 hard). For the results on all 1,079 tasks, please see https://arxiv.org/abs/2607.28545.	Public	324

3 datasets

Organization collection

Datasets published by this organization.

⌘K

Organization inventory

Private datasets are available only to members of this organization.

Dataset	AccessVisibility	Tasks
orca-bench/orca-bench An agent benchmark for root cause analysis — public split of 755 tasks across 77 incidents (245 easy, 264 medium, 246 hard). For the results on all 1,079 tasks, please see https://arxiv.org/abs/2607.28545.	Public	755
orca-bench/orca-bench-verified An agent benchmark for root cause analysis — verified split of 40 tasks across 40 incidents (10 easy, 15 medium, 15 hard). For the results on all 1,079 tasks, please see https://arxiv.org/abs/2607.28545.	Public	40
orca-bench/orca-bench-private An agent benchmark for root cause analysis — held-out private (answers removed) split of 324 tasks across 31 incidents (108 easy, 117 medium, 99 hard). For the results on all 1,079 tasks, please see https://arxiv.org/abs/2607.28545.	Public	324

3 datasets