minnesotanlp/aar

The Amazing Agent Race (AAR): 1400 multi-step scavenger-hunt puzzles for evaluating LLM agents on tool use, web navigation, and arithmetic reasoning. Includes linear (800) and DAG (600) variants across 4 difficulty levels.

harbor run -d minnesotanlp/aar

⌘K

minnesotanlp/aar-linear-extreme-sample_191

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_085

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_020

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_053

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_169

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_213

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_164

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_185

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_152

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_082

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_244

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_211

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_189

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_119

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_121

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_099

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_038

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_152

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_171

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_223

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_163

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_188

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_146

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_032

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_043

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_173

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_231

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_159

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_099

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_088

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_216

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_015

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_089

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_117

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_226

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_173

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_052

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_179

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_091

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_204

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_156

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_110

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_106

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_100

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_129

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_239

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_092

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_128

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_170

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_055

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_095

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_146

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_248

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_073

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_089

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_241

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_202

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_121

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_192

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_124

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_002

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_089

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_190

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_207

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_046

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_113

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_182

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_214

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_135

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_092

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_105

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_067

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_178

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_098

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_185

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_088

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_077

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_218

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_063

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_051

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_222

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_094

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_060

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-easy-sample_062

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_216

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_213

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_230

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_004

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_128

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_024

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_026

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-extreme-sample_097

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-medium-sample_081

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_109

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-easy-sample_122

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_102

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-hard-sample_078

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-linear-extreme-sample_109

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-hard-sample_010

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

minnesotanlp/aar-dag-medium-sample_167

1.0latestrev. 1

aar, tool-use, multi-step

Zae Myung Kim

Displaying 100 of 1,400 tasks