Harbor Hub

minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

Published 4/5/2026 by

New Job
harbor run -d minnesotanlp/aar
Task
minnesotanlp/aar-dag-easy-sample_205
minnesotanlp/aar-linear-hard-sample_136
minnesotanlp/aar-linear-easy-sample_019
minnesotanlp/aar-linear-hard-sample_012
minnesotanlp/aar-linear-easy-sample_017
minnesotanlp/aar-linear-extreme-sample_078
minnesotanlp/aar-dag-medium-sample_100
minnesotanlp/aar-dag-hard-sample_066
minnesotanlp/aar-dag-medium-sample_192
minnesotanlp/aar-dag-medium-sample_050
minnesotanlp/aar-linear-easy-sample_007
minnesotanlp/aar-linear-extreme-sample_106
minnesotanlp/aar-linear-hard-sample_174
minnesotanlp/aar-dag-medium-sample_017
minnesotanlp/aar-dag-hard-sample_073
minnesotanlp/aar-linear-medium-sample_170
minnesotanlp/aar-linear-medium-sample_062
minnesotanlp/aar-linear-hard-sample_063
minnesotanlp/aar-dag-medium-sample_062
minnesotanlp/aar-linear-hard-sample_024
minnesotanlp/aar-linear-hard-sample_162
minnesotanlp/aar-linear-hard-sample_104
minnesotanlp/aar-dag-hard-sample_203
minnesotanlp/aar-linear-hard-sample_032
minnesotanlp/aar-dag-medium-sample_189
minnesotanlp/aar-dag-medium-sample_095
minnesotanlp/aar-dag-hard-sample_018
minnesotanlp/aar-linear-hard-sample_040
minnesotanlp/aar-dag-hard-sample_226
minnesotanlp/aar-linear-easy-sample_182
minnesotanlp/aar-linear-easy-sample_096
minnesotanlp/aar-linear-easy-sample_130
minnesotanlp/aar-linear-easy-sample_052
minnesotanlp/aar-dag-hard-sample_185
minnesotanlp/aar-dag-extreme-sample_133
minnesotanlp/aar-linear-easy-sample_066
minnesotanlp/aar-linear-extreme-sample_047
minnesotanlp/aar-linear-extreme-sample_060
minnesotanlp/aar-dag-medium-sample_223
minnesotanlp/aar-dag-medium-sample_103
minnesotanlp/aar-linear-extreme-sample_103
minnesotanlp/aar-dag-hard-sample_084
minnesotanlp/aar-linear-easy-sample_030
minnesotanlp/aar-dag-medium-sample_073
minnesotanlp/aar-dag-extreme-sample_220
minnesotanlp/aar-linear-extreme-sample_155
minnesotanlp/aar-linear-hard-sample_108
minnesotanlp/aar-linear-hard-sample_206
minnesotanlp/aar-dag-medium-sample_019
minnesotanlp/aar-dag-medium-sample_114
minnesotanlp/aar-dag-extreme-sample_162
minnesotanlp/aar-linear-medium-sample_032
minnesotanlp/aar-linear-hard-sample_184
minnesotanlp/aar-linear-easy-sample_092
minnesotanlp/aar-linear-easy-sample_197
minnesotanlp/aar-linear-extreme-sample_243
minnesotanlp/aar-dag-medium-sample_175
minnesotanlp/aar-linear-medium-sample_159
minnesotanlp/aar-dag-hard-sample_103
minnesotanlp/aar-linear-hard-sample_175
minnesotanlp/aar-linear-extreme-sample_048
minnesotanlp/aar-dag-hard-sample_046
minnesotanlp/aar-dag-extreme-sample_072
minnesotanlp/aar-dag-extreme-sample_142
minnesotanlp/aar-linear-medium-sample_078
minnesotanlp/aar-dag-hard-sample_116
minnesotanlp/aar-dag-medium-sample_046
minnesotanlp/aar-dag-extreme-sample_239
minnesotanlp/aar-linear-extreme-sample_077
minnesotanlp/aar-linear-easy-sample_061
minnesotanlp/aar-dag-extreme-sample_171
minnesotanlp/aar-linear-hard-sample_037
minnesotanlp/aar-dag-hard-sample_035
minnesotanlp/aar-dag-medium-sample_220
minnesotanlp/aar-linear-medium-sample_172
minnesotanlp/aar-dag-hard-sample_184
minnesotanlp/aar-linear-easy-sample_160
minnesotanlp/aar-dag-extreme-sample_242
minnesotanlp/aar-dag-easy-sample_159
minnesotanlp/aar-linear-easy-sample_213
minnesotanlp/aar-dag-medium-sample_068
minnesotanlp/aar-linear-medium-sample_153
minnesotanlp/aar-linear-medium-sample_103
minnesotanlp/aar-dag-medium-sample_124
minnesotanlp/aar-dag-extreme-sample_023
minnesotanlp/aar-linear-medium-sample_161
minnesotanlp/aar-linear-extreme-sample_083
minnesotanlp/aar-linear-hard-sample_072
minnesotanlp/aar-linear-medium-sample_185
minnesotanlp/aar-linear-medium-sample_187
minnesotanlp/aar-dag-extreme-sample_137
minnesotanlp/aar-dag-extreme-sample_018
minnesotanlp/aar-linear-extreme-sample_199
minnesotanlp/aar-linear-medium-sample_192
minnesotanlp/aar-linear-medium-sample_037
minnesotanlp/aar-dag-hard-sample_173
minnesotanlp/aar-linear-easy-sample_020
minnesotanlp/aar-dag-extreme-sample_231
minnesotanlp/aar-linear-hard-sample_110
minnesotanlp/aar-dag-medium-sample_094

Displaying 100 of 1,400 tasks