The speaker considers existing function-calling and multi-tool-calling evaluations unreliable for production because they do not represent orchestration problems in complex real-world scenarios.

“uh so again tool calling has a lot of different evals you have seen the function tool calling evals you have multifunction like like multi multi-time tool calling ones but like none of them are actual like production quality like like none of them are actually something that we are able to rely on truthfully because like they don't really represent the situations that occur like the orchestration issues that occur uh in a complicated sort of scenarios that that you often see in production and so this like getting the feedback through mcp”