refresh/blenderbench-20

harbor run -d refresh/blenderbench-20

BlenderBench: 20 published tasks

Public source dataset for the 20 tasks behind the BlenderBench blog’s 40 published Astra and Opus runs. Every task is pinned to a Harbor content digest. The inputs, instructions, solutions, grader configuration and sealed runtime payloads retain their accepted source bytes. No new runs or regrading were performed for this publication.

Source: BlenderBench blog, source package, and the published Trajectories project.

Reference results and the historical exception

published-runs.json links the 40 existing private Harbor Hub trials, retaining original attempt IDs, scores, trajectory/submitted-scene hashes, model labels, and the exact scorer package digest for each record. These are provenance associations, not new executions or native reassignment of imported run locks. The source import’s historical compatibility-lock caveats still apply.

39 published results match the selected task scorer. The Opus ring-box-rigging result used an older accepted scorer; its separate version is preserved under the same task name, with the blog-historical-opus tag. It is linked in the run index but is not an extra dataset task. The dataset selects the latest accepted Astra ring-box version. Comparing all 40 scores as though they used identical selected scoring would be incorrect.

Package preservation

Six factory tasks originally used the arle/ namespace. Only their [task].name namespace was changed to refresh/ for organization ownership. Original task.toml files and source inventories are retained in source-provenance.tar.gz; package-provenance.json records every mapping and source/scorer digest. All task briefs, input assets and grader bytes remain unchanged.

The official Harbor packager omits root authoring files such as refresh.yaml and the duplicate root verifier.yaml. Those omitted files and the original source review/provenance are included in the supplemental archive. The executable scorer uses the unchanged tests/verifier.yaml copy. Source package hashes and Harbor package digests use different hashing schemes and are recorded separately.

Scope and validation

This is a registry dataset of existing Harbor task layouts, not a new qualification campaign. The preserved source check verified all 20 task inventories, sealed runtime payloads, 40 result associations and the historical scorer. Publication does not establish a fresh Oracle/no-op result, rebuild every environment, or reproduce historical model behavior. The existing missing original Opus pressure-pillow checksum receipt remains documented in source provenance.

Tasks use their original pinned Blender/runtime versions and /app artifact paths. Hidden graders, oracle solutions and provenance are outside environment/ and instruction.md, which are the agent-facing material. The dataset and its task dependencies are publicly accessible. Existing organization grants are retained. The separate Harbor reference jobs and trials remain private to their existing recipients; the published Trajectories links below provide public viewing of the original runs. Making this dataset public does not change historical results or job visibility.

Task and reference-run index

Task	Published reference jobs
`blender-image-kitchen-skeleton-runup`	Astra + Opus
`blender-image-kitchen-surfacing`	Astra + Opus
`blender-generated-asymmetric-expedition-poncho-cloth-preparation`	Astra + Opus
`blender-generated-apartment-dining-table-set-lighting`	Astra + Opus
`blender-image-kitchen-camera-animation`	Astra + Opus
`blender-image-kitchen-rigging`	Astra + Opus
`task-010-image-kitchen-pressure-pillow`	Astra + Opus
`blender-generated-mid-century-lounge-chair-model`	Astra + Opus
`blender-generated-bicycle-courier-humanoid-rigging`	Astra + Opus
`blender-ring-cloth-pull`	Astra + Opus
`blender-image-kitchen-cyclorama`	Astra + Opus
`blender-image-kitchen-composition`	Astra + Opus
`blender-image-kitchen-cloth-preparation`	Astra + Opus
`blender-generated-apartment-dining-table-set-complete`	Astra + Opus
`blender-generated-abyssal-explorer-animation`	Astra + Opus
`task-008-image-kitchen-hair-dynamics`	Astra + Opus
`blender-cowboy-tightrope`	Astra + Opus
`blender-hazmat-retreat-escape`	Astra + Opus
`blender-guardian-boxing`	Astra + Opus
`blender-cowboy-defensive-combat`	Astra + Opus

All 40 associated trials

Each Harbor job contains the two original model trials. The exact Harbor trial ID below selects the matching record within that job; the viewer link opens the corresponding original published trajectory. published-runs.json additionally binds each trial to its exact scorer digest and artifact hashes.

Task	Model	Harbor job / trial ID	Published trajectory
`task-010-image-kitchen-pressure-pillow`	`bedrock/us.anthropic.claude-opus-5-5`	c43d1b05-d788-58fa-a858-3deb48c92b27	View
`task-010-image-kitchen-pressure-pillow`	`openai/gpt-6-astra`	078584ec-b0a3-5904-9b27-fc9569c57900	View
`task-008-image-kitchen-hair-dynamics`	`bedrock/us.anthropic.claude-opus-5-5`	ac5f0a7f-72c0-5db4-87f2-ad410c0b6c1d	View
`task-008-image-kitchen-hair-dynamics`	`openai/gpt-6-astra`	26733cb6-3546-5f43-8935-0faebd7d1a32	View
`blender-ring-cloth-pull`	`bedrock/us.anthropic.claude-opus-5-5`	dfd3d323-4b19-5eb2-82de-a844e5f6e519	View
`blender-ring-cloth-pull`	`openai/gpt-6-astra`	353817da-efa1-50f1-96d4-10ba683f7c1e	View
`blender-image-kitchen-surfacing`	`bedrock/us.anthropic.claude-opus-5-5`	b5f71af9-8087-58ac-bc98-77fee0617440	View
`blender-image-kitchen-surfacing`	`openai/gpt-6-astra`	004a44e3-7083-50fe-8fa0-1349e511e41e	View
`blender-image-kitchen-skeleton-runup`	`bedrock/us.anthropic.claude-opus-5-5`	565eefe6-f7c2-5538-9565-abb7ec5de5f3	View
`blender-image-kitchen-skeleton-runup`	`openai/gpt-6-astra`	a8f19ce6-d509-55eb-a331-3d50651d2f76	View
`blender-image-kitchen-rigging`	`bedrock/us.anthropic.claude-opus-5-5`	8cb75a70-7299-5c1c-88d8-ac37b27042fd	View
`blender-image-kitchen-rigging`	`openai/gpt-6-astra`	08a2eb7a-81bd-5463-a021-3398350fcef8	View
`blender-image-kitchen-cyclorama`	`bedrock/us.anthropic.claude-opus-5-5`	f081ac24-0c96-57d1-b0e8-75f9a6b3b02b	View
`blender-image-kitchen-cyclorama`	`openai/gpt-6-astra`	5ac480e0-0064-571f-8028-5afbbf6b0122	View
`blender-image-kitchen-composition`	`bedrock/us.anthropic.claude-opus-5-5`	ceca1ade-b76b-5630-91d9-093c22d327bb	View
`blender-image-kitchen-composition`	`openai/gpt-6-astra`	744d2a78-a2da-5cd5-af5f-9079a2d835a2	View
`blender-image-kitchen-cloth-preparation`	`bedrock/us.anthropic.claude-opus-5-5`	d4e4e5ea-29ea-5f3f-9ddd-7f08ec3b58f6	View
`blender-image-kitchen-cloth-preparation`	`openai/gpt-6-astra`	d8ad9958-9601-5663-b94c-f55d975c47e4	View
`blender-image-kitchen-camera-animation`	`bedrock/us.anthropic.claude-opus-5-5`	b6bd3884-dbe3-5368-963c-1033b2c2edda	View
`blender-image-kitchen-camera-animation`	`openai/gpt-6-astra`	499f2724-001a-5293-84ea-52003cb8825a	View
`blender-generated-mid-century-lounge-chair-model`	`bedrock/us.anthropic.claude-opus-5-5`	5e3e945b-2165-5428-8987-2bd08324af22	View
`blender-generated-mid-century-lounge-chair-model`	`openai/gpt-6-astra`	5afc15d4-7795-586c-b79d-2d5ce5fd71d4	View
`blender-generated-bicycle-courier-humanoid-rigging`	`bedrock/us.anthropic.claude-opus-5-5`	dc34a5dd-5d8c-5df8-b6c8-17f0129bd0fa	View
`blender-generated-bicycle-courier-humanoid-rigging`	`openai/gpt-6-astra`	4a1ecb47-75d6-5f34-a78e-e6fe96248e80	View
`blender-generated-asymmetric-expedition-poncho-cloth-preparation`	`bedrock/us.anthropic.claude-opus-5-5`	787ce885-8de9-5d8f-b36f-d1ade3f865ad	View
`blender-generated-asymmetric-expedition-poncho-cloth-preparation`	`openai/gpt-6-astra`	9375389c-5fb7-5ac3-bce8-4ea5e7be6a92	View
`blender-generated-apartment-dining-table-set-lighting`	`bedrock/us.anthropic.claude-opus-5-5`	25cb1067-ef3a-5ee5-8258-7fe6120c3009	View
`blender-generated-apartment-dining-table-set-lighting`	`openai/gpt-6-astra`	27d259c6-0741-5edc-90ae-b8984de8a468	View
`blender-generated-apartment-dining-table-set-complete`	`bedrock/us.anthropic.claude-opus-5-5`	7c657f4a-4504-522f-808d-a8eedf92e6fd	View
`blender-generated-apartment-dining-table-set-complete`	`openai/gpt-6-astra`	c3661b2b-e4b1-5ff5-8711-141969955889	View
`blender-generated-abyssal-explorer-animation`	`bedrock/us.anthropic.claude-opus-5-5`	53671251-d391-5d8d-8ef2-77d1bf437e94	View
`blender-generated-abyssal-explorer-animation`	`openai/gpt-6-astra`	7a9e9091-f9e2-53a5-a8b2-f65bf5d06cff	View
`blender-hazmat-retreat-escape`	`bedrock/us.anthropic.claude-opus-5-5`	9711a98a-5923-51c1-a5c2-357ada9f9b37	View
`blender-hazmat-retreat-escape`	`openai/gpt-6-astra`	438d775c-34da-55d4-a0ed-17dd19d4e3d2	View
`blender-guardian-boxing`	`openai/gpt-6-astra`	6ecf9499-62b7-5e83-8c07-967ebfc64e1b	View
`blender-guardian-boxing`	`bedrock/us.anthropic.claude-opus-5-5`	676c81cf-140e-54eb-88ff-c0a312e5ec0e	View
`blender-cowboy-tightrope`	`openai/gpt-6-astra`	327f02ad-3bb0-512e-aa8c-878a054e3a4f	View
`blender-cowboy-tightrope`	`bedrock/us.anthropic.claude-opus-5-5`	0ab0429d-42a2-53f4-a812-782b548b2f5e	View
`blender-cowboy-defensive-combat`	`bedrock/us.anthropic.claude-opus-5-5`	711f1a40-da8d-5a41-858c-ff35dc805c5c	View
`blender-cowboy-defensive-combat`	`openai/gpt-6-astra`	62ee98d7-4a4a-5ecc-a3d1-deca15d6b828	View

Related datasets and contact

This supplements the previously shared, access-restricted Refresh computer-use dataset. It provides the existing BlenderBench tasks and Astra/Opus reference runs; no Muse trials were commissioned for this share. This dataset was initially shared privately and was made public on October 8, 2026. The separate reference jobs retain their existing private access. For questions about the source tasks, scoring history or additional samples, contact Chris Settles at chris@refresh.dev (Refresh).