minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

harbor run -d minnesotanlp/aar

⌘K

minnesotanlp/aar-linear-easy-sample_010

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_006

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_019

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_090

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_169

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_159

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_109

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_063

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_222

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_186

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_097

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_149

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_210

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_094

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_098

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_219

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_126

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_198

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_221

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_016

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_041

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_224

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_189

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_162

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_236

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_042

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_152

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_072

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_150

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_002

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_057

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_224

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_170

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_137

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_228

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_018

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_062

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_043

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_030

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_060

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_071

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_154

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_119

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_029

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_040

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_113

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_093

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_053

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_163

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_001

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_056

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_081

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_212

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_077

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_019

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_093

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_114

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_035

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_051

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_080

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_196

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_066

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_090

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_156

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_037

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_227

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_222

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_223

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_068

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_033

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_164

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_217

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_169

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_067

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_059

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_200

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_207

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_151

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_187

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_225

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_165

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_035

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_158

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_089

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_221

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_144

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_198

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_044

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_006

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_100

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_107

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_122

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_144

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_129

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_000

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_227

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_081

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_193

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_103

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_185

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

Displaying 100 of 1,400 tasks