The benchmark was designed to evaluate whether vision-language models could play games under latency constraints.

“really cool i guess the general premise of this benchmark was we just want to see if vision language models are good enough at just like plugging into games with the latency constraint included because this this actually i came out with this right after clawed plays pokemon came out”