minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

harbor run -d minnesotanlp/aar

⌘K

minnesotanlp/aar-linear-easy-sample_044

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_233

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_047

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_224

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_165

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_064

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_127

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_068

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_029

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_016

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_025

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_076

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_124

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_181

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_046

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_178

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_037

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_194

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_233

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_152

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_187

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_179

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_172

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_127

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_034

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_165

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_120

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_210

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_047

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_243

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_170

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_041

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_214

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_139

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_014

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_071

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_210

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_036

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_061

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_103

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_199

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_210

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_164

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_052

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_226

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_203

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_030

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_175

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_109

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_125

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_115

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_208

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_063

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_148

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_163

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_093

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_112

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_034

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_220

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_191

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_069

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_127

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_105

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_163

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_193

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_218

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_201

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_065

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_009

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_029

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_119

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_047

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_193

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_208

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_109

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_184

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_057

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_126

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_229

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_199

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_179

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_231

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_177

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_179

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_057

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_180

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_179

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_126

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_049

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_241

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_227

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_027

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_206

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_194

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_193

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_163

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_059

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_079

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_000

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_058

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

Displaying 100 of 1,400 tasks