Harbor Hub

minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

Published 4/5/2026 by

New Job
harbor run -d minnesotanlp/aar
Task
minnesotanlp/aar-linear-easy-sample_044
minnesotanlp/aar-dag-hard-sample_233
minnesotanlp/aar-linear-easy-sample_047
minnesotanlp/aar-linear-medium-sample_224
minnesotanlp/aar-dag-hard-sample_165
minnesotanlp/aar-linear-hard-sample_064
minnesotanlp/aar-linear-hard-sample_127
minnesotanlp/aar-linear-extreme-sample_068
minnesotanlp/aar-dag-hard-sample_029
minnesotanlp/aar-linear-extreme-sample_016
minnesotanlp/aar-dag-extreme-sample_025
minnesotanlp/aar-linear-medium-sample_076
minnesotanlp/aar-linear-extreme-sample_124
minnesotanlp/aar-linear-medium-sample_181
minnesotanlp/aar-dag-easy-sample_046
minnesotanlp/aar-dag-easy-sample_178
minnesotanlp/aar-dag-hard-sample_037
minnesotanlp/aar-linear-easy-sample_194
minnesotanlp/aar-dag-medium-sample_233
minnesotanlp/aar-linear-extreme-sample_152
minnesotanlp/aar-dag-easy-sample_187
minnesotanlp/aar-linear-hard-sample_179
minnesotanlp/aar-linear-extreme-sample_172
minnesotanlp/aar-dag-extreme-sample_127
minnesotanlp/aar-linear-extreme-sample_034
minnesotanlp/aar-linear-hard-sample_165
minnesotanlp/aar-linear-medium-sample_120
minnesotanlp/aar-dag-easy-sample_210
minnesotanlp/aar-dag-hard-sample_047
minnesotanlp/aar-dag-extreme-sample_243
minnesotanlp/aar-dag-hard-sample_170
minnesotanlp/aar-linear-easy-sample_041
minnesotanlp/aar-dag-extreme-sample_214
minnesotanlp/aar-dag-hard-sample_139
minnesotanlp/aar-linear-easy-sample_014
minnesotanlp/aar-linear-hard-sample_071
minnesotanlp/aar-dag-hard-sample_210
minnesotanlp/aar-dag-medium-sample_036
minnesotanlp/aar-dag-extreme-sample_061
minnesotanlp/aar-linear-hard-sample_103
minnesotanlp/aar-dag-hard-sample_199
minnesotanlp/aar-linear-hard-sample_210
minnesotanlp/aar-linear-easy-sample_164
minnesotanlp/aar-dag-hard-sample_052
minnesotanlp/aar-linear-extreme-sample_226
minnesotanlp/aar-dag-easy-sample_203
minnesotanlp/aar-dag-easy-sample_030
minnesotanlp/aar-dag-hard-sample_175
minnesotanlp/aar-linear-medium-sample_109
minnesotanlp/aar-linear-hard-sample_125
minnesotanlp/aar-linear-extreme-sample_115
minnesotanlp/aar-linear-extreme-sample_208
minnesotanlp/aar-linear-medium-sample_063
minnesotanlp/aar-dag-medium-sample_148
minnesotanlp/aar-dag-hard-sample_163
minnesotanlp/aar-linear-extreme-sample_093
minnesotanlp/aar-dag-medium-sample_112
minnesotanlp/aar-dag-medium-sample_034
minnesotanlp/aar-linear-easy-sample_220
minnesotanlp/aar-linear-medium-sample_191
minnesotanlp/aar-linear-easy-sample_069
minnesotanlp/aar-linear-medium-sample_127
minnesotanlp/aar-linear-medium-sample_105
minnesotanlp/aar-linear-extreme-sample_163
minnesotanlp/aar-dag-easy-sample_193
minnesotanlp/aar-linear-extreme-sample_218
minnesotanlp/aar-dag-easy-sample_201
minnesotanlp/aar-linear-medium-sample_065
minnesotanlp/aar-linear-easy-sample_009
minnesotanlp/aar-dag-extreme-sample_029
minnesotanlp/aar-dag-extreme-sample_119
minnesotanlp/aar-linear-hard-sample_047
minnesotanlp/aar-linear-extreme-sample_193
minnesotanlp/aar-dag-hard-sample_208
minnesotanlp/aar-dag-hard-sample_109
minnesotanlp/aar-linear-medium-sample_184
minnesotanlp/aar-linear-hard-sample_057
minnesotanlp/aar-linear-extreme-sample_126
minnesotanlp/aar-dag-hard-sample_229
minnesotanlp/aar-linear-easy-sample_199
minnesotanlp/aar-dag-extreme-sample_179
minnesotanlp/aar-linear-extreme-sample_231
minnesotanlp/aar-linear-easy-sample_177
minnesotanlp/aar-dag-hard-sample_179
minnesotanlp/aar-linear-extreme-sample_057
minnesotanlp/aar-linear-medium-sample_180
minnesotanlp/aar-linear-extreme-sample_179
minnesotanlp/aar-linear-easy-sample_126
minnesotanlp/aar-linear-extreme-sample_049
minnesotanlp/aar-dag-extreme-sample_241
minnesotanlp/aar-linear-hard-sample_227
minnesotanlp/aar-dag-extreme-sample_027
minnesotanlp/aar-dag-extreme-sample_206
minnesotanlp/aar-dag-medium-sample_194
minnesotanlp/aar-linear-easy-sample_193
minnesotanlp/aar-dag-easy-sample_163
minnesotanlp/aar-dag-extreme-sample_059
minnesotanlp/aar-linear-hard-sample_079
minnesotanlp/aar-linear-medium-sample_000
minnesotanlp/aar-dag-hard-sample_058

Displaying 100 of 1,400 tasks