Harbor Hub

infra-bench/infra-bench-v1

20 AI-infra engineering tasks on AMD MI35X (5 categories: model deployment, profiling, kernel implementation, kernel tuning, debug triage)

Published 7/17/2026 by

New Job
harbor run -d infra-bench/infra-bench-v1

InfraBench

Github repository: InfraBench

20 real-world AI-infrastructure engineering tasks on AMD MI35X (CDNA4 / gfx950) GPUs, across five categories: model deployment, profiling analysis, kernel implementation, kernel tuning, and debug triage. Each task runs an agent in a Docker sandbox and is scored by a deterministic per-task verifier.

Built on Harbor framework via the harbor CLI.

Requirements

  • AMD ROCm GPUs (/dev/kfd + /dev/dri) and Docker
  • harbor CLI and an LLM-gateway-authenticated agent
  • Model weights at /data/models (mounted read-only into model-dependent tasks)
export INFRABENCH_GPU_COUNT=8
harbor run -d infra-bench/infra-bench-v1 -a claude-code -m <model> -n 6

GPU Pool for Concurrency:

  • Two tasks need 2 GPUs (qr-rmsnorm-fusion, sglang-mmmu-ipc-crash);
  • the GPU-pool allocator is non-blocking, so keep INFRA_PARALLEL ≤ INFRABENCH_GPU_COUNT - 2.
  • All concurrent trials must run on a single host (pool arbitration is filesystem flock).

Results & scoring

  • Output lands in jobs/<timestamp>/. verifier/reward.txt is the score (1 = pass, 0 = fail);
  • Use harbor view jobs to browse trajectories and harbor analyze jobs to analyze failures.
Task
infra-bench/gemm-fp8-ptpc-quant
infra-bench/cute-layout-composition
infra-bench/flydsl-cdna4-preshuffle-gemm
infra-bench/dwconv3d-occupancy
infra-bench/gemma4-gemm-tuning
infra-bench/engram-triton-kernel
infra-bench/mem-bandwidth-bench
infra-bench/hotspot-analysis-torch-profiler
infra-bench/qr-rmsnorm-fusion
infra-bench/triton-matmul-tuning
infra-bench/gemma4-sglang-serving-opt
infra-bench/hello-rocm
infra-bench/llm-fp8-quantize
infra-bench/pointnet2-hipify
infra-bench/sglang-mmmu-ipc-crash
infra-bench/sglang-sync-stall
infra-bench/llvm-simple-constant-propagation
infra-bench/paged-attention-hd256
infra-bench/vllm-aiter-debug
infra-bench/gluon-a8w8-mfma-att

Displaying 20 of 20 tasks