TB2.1 Data Science Track

Data science, scientific computing, machine learning, model training, optimization, querying, and video processing tasks from Terminal-Bench 2.1.

This public Harbor dataset is curated by NovitaAI for a Novita Sandbox hackathon track. It is a category-based subset of Terminal-Bench 2.1. Agents and models are not fixed; the only required runtime environment for the hackathon is Novita Sandbox.

Dataset

• Harbor dataset: NovitaAI/tb21-data-science
• Track size: 26 tasks
• Source benchmark: Terminal-Bench 2.1
• Included source categories: data-science, scientific-computing, machine-learning, data-querying, model-training, optimization, video-processing
• Required hackathon sandbox: -e novita

Quick Start

Run the full track once:

harbor run \
  -d NovitaAI/tb21-data-science \
  -a <agent> \
  -m <model> \
  -e novita \
  -k 1 \
  -n 1 \
  -y

Run a small smoke test from the track:

harbor run \
  -d NovitaAI/tb21-data-science \
  -a <agent> \
  -m <model> \
  -e novita \
  -l 1 \
  -k 1 \
  -n 1 \
  -y

Upload a public result for the hackathon leaderboard:

harbor upload jobs/<job_name> --public

Submit the resulting Harbor Hub job link to the hackathon leaderboard form.

Valid Submission Rules

A valid track submission should satisfy:

• The Harbor job uses this dataset: NovitaAI/tb21-data-science.
• The job config has environment.type = "novita".
• The job does not use extra hints or task-specific extra instructions.
• The submitted Harbor Hub job is public.
• Agent and model are free choice unless a specific event round says otherwise.

Suggested ranking fields:

• Primary: mean reward
• Tie-breaker 1: fewer exceptions/errors
• Tie-breaker 2: lower average duration
• Tie-breaker 3: lower output tokens or total tokens, if the event wants an efficiency prize

Tasks

• terminal-bench/adaptive-rejection-sampler
• terminal-bench/bn-fit-modify
• terminal-bench/caffe-cifar-10
• terminal-bench/count-dataset-tokens
• terminal-bench/distribution-search
• terminal-bench/dna-assembly
• terminal-bench/dna-insert
• terminal-bench/hf-model-inference
• terminal-bench/llm-inference-batching-scheduler
• terminal-bench/mcmc-sampling-stan
• terminal-bench/modernize-scientific-stack
• terminal-bench/mteb-leaderboard
• terminal-bench/mteb-retrieve
• terminal-bench/portfolio-optimization
• terminal-bench/protein-assembly
• terminal-bench/pytorch-model-cli
• terminal-bench/pytorch-model-recovery
• terminal-bench/query-optimize
• terminal-bench/raman-fitting
• terminal-bench/reshard-c4-data
• terminal-bench/rstan-to-pystan
• terminal-bench/sam-cell-seg
• terminal-bench/sparql-university
• terminal-bench/train-fasttext
• terminal-bench/tune-mjcf
• terminal-bench/video-processing