The speaker characterizes existing benchmarks such as Gorilla as primarily evaluating whether an LLM can perform tool calling well.

“So they're like few, right? Like Gorilla was one, but then like a bunch of others emerged,”

“And they were all sort of more on the face of like,”

“is an LLM sort of good with tool calling,”