minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

harbor run -d minnesotanlp/aar

⌘K

minnesotanlp/aar-dag-easy-sample_229

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_101

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_031

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_091

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_135

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_218

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_180

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_209

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_226

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_228

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_106

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_119

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_005

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_132

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_039

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_198

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_075

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_143

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_143

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_205

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_013

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_205

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_138

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_211

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_150

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_188

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_222

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_209

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_201

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_040

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_102

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_140

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_205

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_177

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_069

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_194

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_206

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_052

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_054

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_218

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_189

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_146

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_132

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_058

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_054

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_026

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_049

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_113

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_167

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_135

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_142

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_051

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_186

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_100

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_171

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_061

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_078

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_043

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_064

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_157

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_213

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_115

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_100

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_140

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_149

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_000

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_000

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_042

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_146

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_225

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_003

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_077

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_173

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_101

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_233

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_118

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_041

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_001

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_099

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_118

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_097

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_092

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_212

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_031

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_047

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_107

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_214

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_188

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_083

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_167

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_225

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_058

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_213

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_064

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_022

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_156

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_211

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_022

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_015

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_238

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

Displaying 100 of 1,400 tasks