minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

harbor run -d minnesotanlp/aar

⌘K

minnesotanlp/aar-dag-extreme-sample_196

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_212

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_236

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_022

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_119

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_065

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_166

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_118

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_021

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_171

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_082

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_170

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_240

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_207

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_144

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_221

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_201

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_230

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_080

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_083

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_015

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_066

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_186

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_090

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_023

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_042

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_210

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_193

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_174

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_156

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_050

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_073

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_032

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_011

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_021

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_143

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_104

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_173

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_048

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_102

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_058

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_043

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_004

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_053

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_216

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_022

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_070

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_148

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_055

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_215

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_112

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_183

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_171

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_102

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_156

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_169

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_202

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_004

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_205

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_232

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_074

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_189

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_110

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_228

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_075

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_175

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_110

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_079

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_072

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_159

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_129

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_027

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_177

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_056

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_237

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_131

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_034

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_111

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_204

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_010

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_006

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_185

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_133

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_238

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_071

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_217

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_176

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_054

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_218

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_116

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_038

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_201

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_177

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_073

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_083

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_180

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_138

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_036

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_070

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_135

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

Displaying 100 of 1,400 tasks