Harbor Hub

minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

Published 4/5/2026 by

New Job
harbor run -d minnesotanlp/aar
Task
minnesotanlp/aar-dag-easy-sample_101
minnesotanlp/aar-dag-extreme-sample_141
minnesotanlp/aar-linear-extreme-sample_174
minnesotanlp/aar-dag-hard-sample_025
minnesotanlp/aar-linear-extreme-sample_182
minnesotanlp/aar-linear-easy-sample_087
minnesotanlp/aar-dag-hard-sample_132
minnesotanlp/aar-dag-easy-sample_221
minnesotanlp/aar-linear-easy-sample_038
minnesotanlp/aar-dag-extreme-sample_150
minnesotanlp/aar-dag-extreme-sample_060
minnesotanlp/aar-linear-easy-sample_135
minnesotanlp/aar-linear-medium-sample_157
minnesotanlp/aar-dag-easy-sample_126
minnesotanlp/aar-linear-extreme-sample_240
minnesotanlp/aar-linear-extreme-sample_228
minnesotanlp/aar-dag-extreme-sample_030
minnesotanlp/aar-dag-medium-sample_208
minnesotanlp/aar-dag-easy-sample_226
minnesotanlp/aar-linear-hard-sample_084
minnesotanlp/aar-dag-extreme-sample_058
minnesotanlp/aar-dag-hard-sample_235
minnesotanlp/aar-dag-medium-sample_022
minnesotanlp/aar-dag-medium-sample_105
minnesotanlp/aar-linear-easy-sample_035
minnesotanlp/aar-linear-extreme-sample_085
minnesotanlp/aar-dag-easy-sample_130
minnesotanlp/aar-linear-extreme-sample_239
minnesotanlp/aar-dag-extreme-sample_071
minnesotanlp/aar-linear-hard-sample_229
minnesotanlp/aar-linear-easy-sample_214
minnesotanlp/aar-linear-medium-sample_071
minnesotanlp/aar-linear-hard-sample_205
minnesotanlp/aar-dag-hard-sample_087
minnesotanlp/aar-dag-extreme-sample_016
minnesotanlp/aar-linear-extreme-sample_204
minnesotanlp/aar-dag-extreme-sample_050
minnesotanlp/aar-dag-easy-sample_167
minnesotanlp/aar-linear-hard-sample_199
minnesotanlp/aar-dag-easy-sample_131
minnesotanlp/aar-linear-hard-sample_158
minnesotanlp/aar-dag-easy-sample_071
minnesotanlp/aar-linear-hard-sample_147
minnesotanlp/aar-linear-extreme-sample_054
minnesotanlp/aar-dag-hard-sample_241
minnesotanlp/aar-linear-extreme-sample_027
minnesotanlp/aar-dag-extreme-sample_051
minnesotanlp/aar-dag-hard-sample_240
minnesotanlp/aar-linear-easy-sample_143
minnesotanlp/aar-dag-medium-sample_163
minnesotanlp/aar-linear-hard-sample_169
minnesotanlp/aar-dag-extreme-sample_049
minnesotanlp/aar-linear-easy-sample_097
minnesotanlp/aar-dag-easy-sample_168
minnesotanlp/aar-linear-hard-sample_018
minnesotanlp/aar-linear-medium-sample_107
minnesotanlp/aar-linear-easy-sample_000
minnesotanlp/aar-linear-medium-sample_077
minnesotanlp/aar-linear-hard-sample_023
minnesotanlp/aar-dag-extreme-sample_210
minnesotanlp/aar-linear-medium-sample_219
minnesotanlp/aar-linear-hard-sample_128
minnesotanlp/aar-dag-extreme-sample_126
minnesotanlp/aar-linear-medium-sample_121
minnesotanlp/aar-linear-hard-sample_004
minnesotanlp/aar-dag-easy-sample_202
minnesotanlp/aar-linear-extreme-sample_234
minnesotanlp/aar-linear-medium-sample_145
minnesotanlp/aar-linear-hard-sample_069
minnesotanlp/aar-linear-hard-sample_019
minnesotanlp/aar-dag-extreme-sample_116
minnesotanlp/aar-linear-extreme-sample_062
minnesotanlp/aar-linear-hard-sample_200
minnesotanlp/aar-linear-easy-sample_163
minnesotanlp/aar-linear-easy-sample_098
minnesotanlp/aar-linear-medium-sample_134
minnesotanlp/aar-dag-hard-sample_206
minnesotanlp/aar-linear-easy-sample_148
minnesotanlp/aar-dag-hard-sample_115
minnesotanlp/aar-dag-medium-sample_107
minnesotanlp/aar-linear-easy-sample_161
minnesotanlp/aar-dag-easy-sample_213
minnesotanlp/aar-linear-easy-sample_137
minnesotanlp/aar-dag-medium-sample_204
minnesotanlp/aar-linear-medium-sample_025
minnesotanlp/aar-dag-extreme-sample_154
minnesotanlp/aar-linear-hard-sample_080
minnesotanlp/aar-dag-hard-sample_137
minnesotanlp/aar-linear-extreme-sample_095
minnesotanlp/aar-dag-hard-sample_031
minnesotanlp/aar-linear-easy-sample_217
minnesotanlp/aar-dag-extreme-sample_161
minnesotanlp/aar-linear-hard-sample_167
minnesotanlp/aar-linear-extreme-sample_242
minnesotanlp/aar-linear-easy-sample_046
minnesotanlp/aar-linear-hard-sample_041
minnesotanlp/aar-dag-medium-sample_150
minnesotanlp/aar-linear-easy-sample_093
minnesotanlp/aar-linear-hard-sample_135
minnesotanlp/aar-linear-medium-sample_205

Displaying 100 of 1,400 tasks